[xen-4.12-testing test] 160767: regressions - trouble: blocked/broken/fail/pass

[osstest service owner]

flight 160767 xen-4.12-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/160767/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf  broken
 build-armhf   4 host-install(4)broken REGR. vs. 159418
 test-amd64-amd64-xl-qcow2 19 guest-localmigrate/x10 fail in 160709 REGR. vs. 
159418

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-libvirt-pair 22 guests-nbd-mirror/debian fail in 160709 pass 
in 160767
 test-amd64-amd64-xl-qcow218 guest-saverestore.2fail pass in 160709

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-cubietruck  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-rtds  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop fail like 159418
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 159418
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 159418
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stopfail like 159418
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop fail like 159418
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stopfail like 159418
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 159418
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop fail like 159418
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop fail like 159418
 test-amd64-i386-xl-pvshim14 guest-start  fail   never pass
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-checkfail   never pass

version targeted for testing:
 xen  5b280a59c4dd8dad6cc8da28db981b193d10acee
baseline version:
 xen  4cf5929606adc2fb1ab4e2921c14ba4b8046ecd1

Last test of basis   159418  2021-02-16 15:06:11 Z   49 days
Failing since160128  2021-03-18 14:36:18 Z   19 days   24 attempts
Testing same since   160150  2021-03-20 04:11:48 Z   18 days   22 attempts


People who touched revisions under test:
  Andrew Cooper 
  Anthony PERARD 
  Christian Lindig 
  Edwin Török 
  Igor Druzhinin 
  Jan Beulich 
  Julien Grall 
  Olaf Hering 
  Stefano Stabellini 

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  pass
 build-i386-xsm   pass
 build-amd64-xtf  pass
 build-amd64  

[qemu-mainline test] 160763: regressions - trouble: broken/fail/pass

[osstest service owner]

flight 160763 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/160763/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-libvirt broken
 test-armhf-armhf-libvirt-raw broken
 test-armhf-armhf-xl-arndale  broken
 test-armhf-armhf-xl-credit1  broken
 test-armhf-armhf-xl-rtds broken
 test-armhf-armhf-xl-vhd  broken
 test-armhf-armhf-xl-cubietruck broken
 test-armhf-armhf-xl-credit2  broken
 test-amd64-amd64-qemuu-freebsd11-amd64 16 guest-saverestore fail REGR. vs. 
152631
 test-amd64-i386-libvirt  14 guest-start  fail REGR. vs. 152631
 test-amd64-amd64-libvirt 14 guest-start  fail REGR. vs. 152631
 test-amd64-i386-libvirt-xsm  14 guest-start  fail REGR. vs. 152631
 test-amd64-amd64-qemuu-freebsd12-amd64 16 guest-saverestore fail REGR. vs. 
152631
 test-amd64-amd64-libvirt-xsm 14 guest-start  fail REGR. vs. 152631
 test-amd64-i386-freebsd10-i386 16 guest-saverestore  fail REGR. vs. 152631
 test-amd64-i386-freebsd10-amd64 16 guest-saverestore fail REGR. vs. 152631
 test-amd64-amd64-libvirt-pair 25 guest-start/debian  fail REGR. vs. 152631
 test-amd64-i386-libvirt-pair 25 guest-start/debian   fail REGR. vs. 152631
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 15 guest-saverestore fail 
REGR. vs. 152631
 test-amd64-i386-xl-qemuu-debianhvm-amd64 15 guest-saverestore fail REGR. vs. 
152631
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 15 guest-saverestore fail REGR. 
vs. 152631
 test-amd64-i386-xl-qemuu-ovmf-amd64 15 guest-saverestore fail REGR. vs. 152631
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 15 guest-saverestore fail 
REGR. vs. 152631
 test-amd64-amd64-xl-qemuu-debianhvm-amd64 15 guest-saverestore fail REGR. vs. 
152631
 test-amd64-amd64-xl-qemuu-win7-amd64 15 guest-saverestore fail REGR. vs. 152631
 test-amd64-amd64-xl-qemuu-ovmf-amd64 15 guest-saverestore fail REGR. vs. 152631
 test-arm64-arm64-libvirt-xsm 14 guest-start  fail REGR. vs. 152631
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 15 guest-saverestore fail REGR. 
vs. 152631
 test-amd64-amd64-qemuu-nested-intel 20 debian-hvm-install/l1/l2 fail REGR. vs. 
152631
 test-amd64-i386-xl-qemuu-win7-amd64 15 guest-saverestore fail REGR. vs. 152631
 test-armhf-armhf-xl-multivcpu 12 debian-install  fail REGR. vs. 152631
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 12 debian-hvm-install fail 
REGR. vs. 152631
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 12 debian-hvm-install fail 
REGR. vs. 152631
 test-amd64-i386-xl-qemuu-ws16-amd64 15 guest-saverestore fail REGR. vs. 152631
 test-amd64-amd64-xl-qemuu-ws16-amd64 15 guest-saverestore fail REGR. vs. 152631
 test-armhf-armhf-xl  12 debian-install   fail REGR. vs. 152631

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-vhd   5 host-install(5)   broken blocked in 152631
 test-armhf-armhf-xl-arndale   5 host-install(5)   broken blocked in 152631
 test-armhf-armhf-xl-credit1   5 host-install(5)   broken blocked in 152631
 test-armhf-armhf-libvirt  5 host-install(5)   broken blocked in 152631
 test-armhf-armhf-xl-rtds  5 host-install(5)   broken blocked in 152631
 test-armhf-armhf-libvirt-raw  5 host-install(5)   broken blocked in 152631
 test-armhf-armhf-xl-credit2   5 host-install(5)   broken blocked in 152631
 test-armhf-armhf-xl-cubietruck  5 host-install(5) broken blocked in 152631
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152631
 test-arm64-arm64-xl-seattle  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-pvshim14 guest-start  fail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-checkfail   never pass

version targeted for testing:
 qemuuee82c086baaa534d1af26cb8b86e86fb047af918

[xen-unstable-smoke test] 160777: tolerable all pass - PUSHED

[osstest service owner]

flight 160777 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/160777/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  19be4d56a1f7aa65eb4d92276fa5d386e9cb2a62
baseline version:
 xen  6425fcf37122b1f1e4ceaacadd00a4c280a58f57

Last test of basis   160772  2021-04-06 15:00:34 Z0 days
Testing same since   160777  2021-04-06 21:02:47 Z0 days1 attempts


People who touched revisions under test:
  George Dunlap 
  Ian Jackson 
  Jan Beulich 
  Julien Grall 
  Tim Deegan 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   6425fcf371..19be4d56a1  19be4d56a1f7aa65eb4d92276fa5d386e9cb2a62 -> smoke

[xen-4.15-testing test] 160774: tolerable FAIL - PUSHED

[osstest service owner]

flight 160774 xen-4.15-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/160774/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 160455
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stopfail like 160455
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 160455
 test-armhf-armhf-libvirt 16 saverestore-support-checkfail  like 160455
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 160455
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stopfail like 160455
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop fail like 160455
 test-armhf-armhf-libvirt-raw 15 saverestore-support-checkfail  like 160455
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop fail like 160455
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop fail like 160455
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop fail like 160455
 test-armhf-armhf-xl-multivcpu 15 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-checkfail  never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-checkfail   never pass
 test-amd64-i386-xl-pvshim14 guest-start  fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt 15 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  15 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  e25aa9939ae0cd8317605be3d5c5611b76bc4ab4
baseline version:
 xen  7fa14f3f525b4a2d660794424fd787cfdf9c904b

Last test of basis   160455  2021-03-26 20:09:16 Z   11 days
Testing same since   160774  2021-04-06 17:17:31 Z0 days1 attempts


People who touched revisions under test:
  George Dunlap 
  Ian Jackson 
  Ian Jackson 
  Jan Beulich 
  Julien Grall 
  Stefano Stabellini 

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  pass
 build-i386-xsm   

[qemu-mainline bisection] complete test-amd64-i386-libvirt-pair

[osstest service owner]

branch xen-unstable
xenbranch xen-unstable
job test-amd64-i386-libvirt-pair
testid guest-start/debian

Tree: libvirt git://xenbits.xen.org/libvirt.git
Tree: libvirt_keycodemapdb https://gitlab.com/keycodemap/keycodemapdb.git
Tree: linux git://xenbits.xen.org/linux-pvops.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: ovmf git://xenbits.xen.org/osstest/ovmf.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://git.qemu.org/qemu.git
Tree: seabios git://xenbits.xen.org/osstest/seabios.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

  Bug is in tree:  qemuu git://git.qemu.org/qemu.git
  Bug introduced:  8d17adf34f501ded65a106572740760f0a75577c
  Bug not present: e67d8e2928200e24ecb47c7be3ea8270077f2996
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/160775/


  commit 8d17adf34f501ded65a106572740760f0a75577c
  Author: Daniel P. Berrangé 
  Date:   Mon Feb 22 11:16:32 2021 +
  
  block: remove support for using "file" driver with block/char devices
  
  The 'host_device' and 'host_cdrom' drivers must be used instead.
  
  Reviewed-by: Eric Blake 
  Signed-off-by: Daniel P. Berrangé 


For bisection revision-tuple graph see:
   
http://logs.test-lab.xenproject.org/osstest/results/bisect/qemu-mainline/test-amd64-i386-libvirt-pair.guest-start--debian.html
Revision IDs in each graph node refer, respectively, to the Trees above.


Running cs-bisection-step 
--graph-out=/home/logs/results/bisect/qemu-mainline/test-amd64-i386-libvirt-pair.guest-start--debian
 --summary-out=tmp/160775.bisection-summary --basis-template=152631 
--blessings=real,real-bisect,real-retry qemu-mainline 
test-amd64-i386-libvirt-pair guest-start/debian
Searching for failure / basis pass:
 160748 fail [dst_host=chardonnay1,src_host=chardonnay0] / 160125 
[dst_host=albana1,src_host=albana0] 160119 
[dst_host=elbling0,src_host=elbling1] 160113 
[dst_host=chardonnay0,src_host=chardonnay1] 160104 
[dst_host=albana0,src_host=albana1] 160097 [dst_host=fiano0,src_host=fiano1] 
160091 ok.
Failure / basis pass flights: 160748 / 160091
(tree with no url: minios)
Tree: libvirt git://xenbits.xen.org/libvirt.git
Tree: libvirt_keycodemapdb https://gitlab.com/keycodemap/keycodemapdb.git
Tree: linux git://xenbits.xen.org/linux-pvops.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: ovmf git://xenbits.xen.org/osstest/ovmf.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://git.qemu.org/qemu.git
Tree: seabios git://xenbits.xen.org/osstest/seabios.git
Tree: xen git://xenbits.xen.org/xen.git
Latest 2c846fa6bcc11929c9fb857a22430fb9945654ad 
27acf0ef828bf719b2053ba398b195829413dbdd 
c3038e718a19fc596f7b1baba0f83d5146dc7784 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
f95cdd316c3d56e8f76b5044be54b9645e1dc60f 
3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 
11577d85b1a6939380bd16ed9a861653194de044 
b0d61ecef66eb05bd7a4eb7ada88ec5dab06dfee 
b0976d5c0441378b6348f5bfedbf431055bd0147
Basis pass 2c846fa6bcc11929c9fb857a22430fb9945654ad 
27acf0ef828bf719b2053ba398b195829413dbdd 
c3038e718a19fc596f7b1baba0f83d5146dc7784 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
4751a48aeb2ab828b0a5cbdc585fd3642967cda1 
3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 
e7c6a8cf9f5c82aa152273e1c9e80d07b1b0c32c 
b0d61ecef66eb05bd7a4eb7ada88ec5dab06dfee 
14b95b3b8546db201e7efd0636ae0e215fae98f3
Generating revisions with ./adhoc-revtuple-generator  
git://xenbits.xen.org/libvirt.git#2c846fa6bcc11929c9fb857a22430fb9945654ad-2c846fa6bcc11929c9fb857a22430fb9945654ad
 
https://gitlab.com/keycodemap/keycodemapdb.git#27acf0ef828bf719b2053ba398b195829413dbdd-27acf0ef828bf719b2053ba398b195829413dbdd
 
git://xenbits.xen.org/linux-pvops.git#c3038e718a19fc596f7b1baba0f83d5146dc7784-c3038e718a19fc596f7b1baba0f83d5146dc7784
 
git://xenbits.xen.org/osstest/linux-firmware.git#c530a75c1e6a472b0eb9558310b518f0\
 dfcd8860-c530a75c1e6a472b0eb9558310b518f0dfcd8860 
git://xenbits.xen.org/osstest/ovmf.git#4751a48aeb2ab828b0a5cbdc585fd3642967cda1-f95cdd316c3d56e8f76b5044be54b9645e1dc60f
 
git://xenbits.xen.org/qemu-xen-traditional.git#3d273dd05e51e5a1ffba3d98c7437ee84e8f8764-3d273dd05e51e5a1ffba3d98c7437ee84e8f8764
 
git://git.qemu.org/qemu.git#e7c6a8cf9f5c82aa152273e1c9e80d07b1b0c32c-11577d85b1a6939380bd16ed9a861653194de044
 
git://xenbits.xen.org/osstest/seabios.git#b0d61ecef66eb05bd7a4eb7ada88ec5dab06dfee-b0d61ec\
 ef66eb05bd7a4eb7ada88ec5dab06dfee 
git://xenbits.xen.org/xen.git#14b95b3b8546db201e7efd0636ae0e215fae98f3-b0976d5c0441378b6348f5bfedbf431055bd0147
Loaded 35125 nodes in revision graph
Searching for test results:
 160091 pass 2c846fa6bcc11929c9fb857a22430fb9945654ad 
27acf0ef828bf719b2053ba398b195829413dbdd 
c3038e718a19fc596f7b1baba0f83d5146dc7784 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
4751a48aeb2ab828b0a5cbdc585fd3642967cda1 
3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 

Re: [PATCH v3 0/3] Generic SMMU Bindings

[Stefano Stabellini]

On Mon, 5 Apr 2021, Julien Grall wrote:
> On 26/01/2021 22:58, Stefano Stabellini wrote:
> > Hi all,
> 
> Hi Stefano,
> 
> > This series introduces support for the generic SMMU bindings to
> > xen/drivers/passthrough/arm/smmu.c.
> > 
> > The last version of the series was
> > https://marc.info/?l=xen-devel=159539053406643
> Some changes in the SMMU drivers went in recently. I believe this touched
> similar area to this series. Would you be able to check that this series still
> work as intented?

Thanks for the heads up, no, unfortunately they don't work :-(

They badly clash. I did the forward port of the three patches but they
fail at runtime in my tests. I ran out of time to figure out what is the
problem, and I'll have to pick it up at some point in the future (it
might not be any time soon unfortunately).

Rahul, if you have any ideas about what the problem is please let me
know. This is the branch with the forward port:

http://xenbits.xenproject.org/git-http/people/sstabellini/xen-unstable.git 
smmu-generic

Re: [PATCH v2] xen/iommu: smmu: Silence clang in arm_smmu_device_dt_probe()

[Stefano Stabellini]

On Fri, 2 Apr 2021, Julien Grall wrote:
> From: Julien Grall 
> 
> Clang 11 will throw the following error:
> 
> smmu.c:2284:18: error: cast to smaller integer type 'enum 
> arm_smmu_arch_version' from 'const void *' 
> [-Werror,-Wvoid-pointer-to-enum-cast]
> smmu->version = (enum arm_smmu_arch_version)of_id->data;
> ^~~
> 
> The error can be prevented by initially casting to (uintptr_t) and then
> enum.
> 
> Signed-off-by: Julien Grall 

Acked-by: Stefano Stabellini 


> ---
> 
> Changes in v2:
> - Cast to (uintptr_t) rather than introduce separate pointer
> (requested by Andrew).
> 
> Only build tested
> ---
>  xen/drivers/passthrough/arm/smmu.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/xen/drivers/passthrough/arm/smmu.c 
> b/xen/drivers/passthrough/arm/smmu.c
> index 20ac672e91b6..fab7be8b48bb 100644
> --- a/xen/drivers/passthrough/arm/smmu.c
> +++ b/xen/drivers/passthrough/arm/smmu.c
> @@ -2382,7 +2382,7 @@ static int arm_smmu_device_dt_probe(struct 
> platform_device *pdev)
>   smmu->dev = dev;
>  
>   of_id = of_match_node(arm_smmu_of_match, dev->of_node);
> - smmu->version = (enum arm_smmu_arch_version)of_id->data;
> + smmu->version = (enum arm_smmu_arch_version)(uintptr_t)of_id->data;
>  
>   res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
>   smmu->base = devm_ioremap_resource(dev, res);
> -- 
> 2.17.1
> 

Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h

[Stefano Stabellini]

On Tue, 6 Apr 2021, Jan Beulich wrote:
> On 06.04.2021 12:36, Luca Fancellu wrote:
> > Modification to include/public/grant_table.h:
> > 
> > 1) Change anonymous structure to be named structure,
> >because doxygen can't deal with them.
> 
> Especially in the form presented (adding further name space clutter
> for consumers to fall over) I object to this, most notably ...
> 
> > @@ -584,7 +599,7 @@ DEFINE_XEN_GUEST_HANDLE(gnttab_swap_grant_ref_t);
> >   * page granted to the calling domain by a foreign domain.
> >   */
> >  struct gnttab_cache_flush {
> > -union {
> > +union a {
> 
> ... this one.

Hi Jan,

It is unfortunate that none of these tools support anonymous structs or
unions well. (You might recall we also had issues with the older
kernel-doc series too, although a bit different.)

It is difficult to provide a good name here, a suggestion would be more
than welcome. I agree with you that calling it "a" is a bad idea: "a"
becomes a globally-visible union name.

Maybe we could call it: StructName_MemberName, so in this case:

  union gnttab_cache_flush_a

It makes sure it is unique and doesn't risk clashing with anything else.
We can follow this pattern elsewhere as well.

Any better suggestions?

Re: [PATCH 0/3] Use Doxygen and sphinx for html documentation

[Stefano Stabellini]

On Tue, 6 Apr 2021, Andrew Cooper wrote:
> On 06/04/2021 11:36, Luca Fancellu wrote:
> > This serie introduce doxygen in the sphinx html docs generation.
> > One benefit is to keep most of the documentation in the source
> > files of xen so that it's more maintainable, on the other hand
> > there are some limitation of doxygen that must be addressed
> > modifying the current codebase (for example doxygen can't parse
> > anonymous structure/union).
> >
> > To reproduce the documentation xen must be compiled because
> > most of the headers are generated on compilation time from
> > the makefiles.
> >
> > Here follows the steps to generate the sphinx html docs, some
> > package may be required on your machine, everything is suggested
> > by the autoconf script.
> > Here I'm building the arm64 docs (the only introduced for now by
> > this serie):
> >
> > ./configure
> > make -C xen XEN_TARGET_ARCH="arm64" CROSS_COMPILE="aarch64-linux-gnu-" 
> > menuconfig
> > make -C xen XEN_TARGET_ARCH="arm64" CROSS_COMPILE="aarch64-linux-gnu-"
> > make -C docs XEN_TARGET_ARCH="arm64" sphinx-html
> >
> > now in docs/sphinx/html/ we have the generated docs starting
> > from the index.html page.
> 
> Do you have a sample rendered output?
> 
> The plan was to try and use Linux's kernel-doc plugin for Sphinx, which
> is very doxygen-like.  Did you experiment with this option?

As you probably know the end goal for Luca (and the Xen FuSa SIG as a
whole) is to generate all FuSa documents, including requirements docs,
interface docs, etc.

FuSa requires us to follow the famous V model, where the high level
requirements are linked to the lower level requirements, which are
linked to the interface docs, which are linked all the way down to the
code.

Maintaining the linking is difficult and typically done with expensive
proprietary FuSa tools.

Fortunately, an engineer working with the Zephyr project developed a set
of scripts for Doxygen that are able to generate the required FuSa docs
and also links from in-code comments and markdown/rst docs in the tree.

This is great work, and in the FuSa SIG we thought it would be best to
align ourselves with Zephyr to be able to pull our efforts together on
the tooling side instead of doing the same thing again on our own.

This is the reason why we ended up with Doxygen.

[xen-unstable-smoke test] 160772: tolerable all pass - PUSHED

[osstest service owner]

flight 160772 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/160772/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  6425fcf37122b1f1e4ceaacadd00a4c280a58f57
baseline version:
 xen  0435784cc75dcfef3b5f59c29deb1dbb84265ddb

Last test of basis   160648  2021-04-01 18:00:27 Z5 days
Testing same since   160772  2021-04-06 15:00:34 Z0 days1 attempts


People who touched revisions under test:
  Jan Beulich 
  Julien Grall 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   0435784cc7..6425fcf371  6425fcf37122b1f1e4ceaacadd00a4c280a58f57 -> smoke

Re: [PATCH 02/23] lib: move 64-bit div/mod compiler helpers

[Julien Grall]

Hi Jan,

On 01/04/2021 16:23, Jan Beulich wrote:

On 01.04.2021 16:56, Julien Grall wrote:

On 01/04/2021 11:19, Jan Beulich wrote:

--- a/xen/common/lib.c
+++ b/xen/lib/divmod.c
@@ -40,7 +40,6 @@
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*/
-#if BITS_PER_LONG == 32


In theory BITS_PER_LONG == 32 is not quite the same as 32-bit arch. Can
you clarify whether this code is necessary on 64-bit arch where long is
only 32-bit.


Likely the compiler can get away without invoking these helpers, so
the code would remain unused. I'm uncertain whether CONFIG_64BIT
ought to be set for such an architecture, as we assume sizeof(long)
== sizeof(void*), and hence pointers would then need to be 32-bit
as well there.


This is a fair point. Would you mind to add a sentence explaining that 
in the commit message?


With that:

Acked-by: Julien Grall 

Cheers,

--
Julien Grall

[PATCH] xen/page_alloc: Don't hold the heap_lock when clearing PGC_need_scrub

[Julien Grall]

From: Julien Grall 

Currently, the heap_lock is held when clearing PGC_need_scrub in
alloc_heap_pages(). However, this is unnecessary because the only caller
(mark_page_offline()) that can concurrently modify the count_info is
using cmpxchg() in a loop.

Therefore, rework the code to avoid holding the heap_lock and use
test_and_clear_bit() instead.

Signed-off-by: Julien Grall 
---
 xen/common/page_alloc.c | 6 +-
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 68e47d963842..70146a00ec8b 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -1038,16 +1038,12 @@ static struct page_info *alloc_heap_pages(
 {
 for ( i = 0; i < (1U << order); i++ )
 {
-if ( test_bit(_PGC_need_scrub, [i].count_info) )
+if ( test_and_clear_bit(_PGC_need_scrub, [i].count_info) )
 {
 if ( !(memflags & MEMF_no_scrub) )
 scrub_one_page([i]);
 
 dirty_cnt++;
-
-spin_lock(_lock);
-pg[i].count_info &= ~PGC_need_scrub;
-spin_unlock(_lock);
 }
 else if ( !(memflags & MEMF_no_scrub) )
 check_one_page([i]);
-- 
2.17.1

[PATCH] xen/page_alloc: Remove dead code in alloc_domheap_pages()

[Julien Grall]

From: Julien Grall 

Since commit 1aac966e24e9 "xen: support RAM at addresses 0 and 4096",
bits_to_zone() will never return 0 and it is expected that we have
minimum 2 zones.

Therefore the check in alloc_domheap_pages() is unnecessary and can
be removed. However, for sanity, it is replaced with an ASSERT().

Also take the opportunity to check atbuild time that NR_ZONES is minimum
2.

This bug was discovered and resolved using Coverity Static Analysis
Security Testing (SAST) by Synopsys, Inc.

Signed-off-by: Julien Grall 
---
 xen/common/page_alloc.c | 11 +--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 1744e6faa5c4..68e47d963842 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -457,6 +457,12 @@ static long total_avail_pages;
 static DEFINE_SPINLOCK(heap_lock);
 static long outstanding_claims; /* total outstanding claims by all domains */
 
+static void __init __maybe_unused build_assertions(void)
+{
+/* Zone 0 is reserved for Xen, so we at least need two zones to function.*/
+BUILD_BUG_ON(NR_ZONES < 2);
+}
+
 unsigned long domain_adjust_tot_pages(struct domain *d, long pages)
 {
 long dom_before, dom_after, dom_claimed, sys_before, sys_after;
@@ -2340,8 +2346,9 @@ struct page_info *alloc_domheap_pages(
 
 bits = domain_clamp_alloc_bitsize(memflags & MEMF_no_owner ? NULL : d,
   bits ? : (BITS_PER_LONG+PAGE_SHIFT));
-if ( (zone_hi = min_t(unsigned int, bits_to_zone(bits), zone_hi)) == 0 )
-return NULL;
+
+zone_hi = min_t(unsigned int, bits_to_zone(bits), zone_hi);
+ASSERT(zone_hi != 0);
 
 if ( memflags & MEMF_no_owner )
 memflags |= MEMF_no_refcount;
-- 
2.17.1

[PATCH v2] xen/arm: kernel: Propagate the error if we fail to decompress the kernel

[Julien Grall]

From: Julien Grall 

Currently, we are ignoring any error from perform_gunzip() and replacing
the compressed kernel with the "uncompressed" kernel.

If there is a gzip failure, then it means that the output buffer may
contain garbagge. So it can result to various sort of behavior that may
be difficult to root cause.

In case of failure, free the output buffer and propagate the error.
We also need to adjust the return check for kernel_compress() as
perform_gunzip() may return a positive value.

Take the opportunity to adjust the code style for the check.

Signed-off-by: Julien Grall 

---
Changes in v2:
- Fix build
---
 xen/arch/arm/kernel.c | 8 +++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/kernel.c b/xen/arch/arm/kernel.c
index ab78689ed2a6..8f43caa1866d 100644
--- a/xen/arch/arm/kernel.c
+++ b/xen/arch/arm/kernel.c
@@ -292,6 +292,12 @@ static __init int kernel_decompress(struct bootmodule *mod)
 iounmap(input);
 vunmap(output);
 
+if ( rc )
+{
+free_domheap_pages(pages, kernel_order_out);
+return rc;
+}
+
 mod->start = page_to_maddr(pages);
 mod->size = output_size;
 
@@ -503,7 +509,7 @@ int __init kernel_probe(struct kernel_info *info,
 
 /* if it is a gzip'ed image, 32bit or 64bit, uncompress it */
 rc = kernel_decompress(mod);
-if (rc < 0 && rc != -EINVAL)
+if ( rc && rc != -EINVAL )
 return rc;
 
 #ifdef CONFIG_ARM_64
-- 
2.17.1

Re: [PATCH 1/2] xen/arm: kernel: Propagate the error if we fail to decompress the kernel

[Julien Grall]

On 02/04/2021 16:21, Julien Grall wrote:

From: Julien Grall 

Currently, we are ignoring any error from perform_gunzip() and replacing
the compressed kernel with the "uncompressed" kernel.

If there is a gzip failure, then it means that the output buffer may
contain garbagge. So it can result to various sort of behavior that may
be difficult to root cause.

In case of failure, free the output buffer and propagate the error.
We also need to adjust the return check for kernel_compress() as
perform_gunzip() may return a positive value.

Take the opportunity to adjust the code style for the check.

Signed-off-by: Julien Grall 
---
  xen/arch/arm/kernel.c | 8 +++-
  1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/kernel.c b/xen/arch/arm/kernel.c
index ab78689ed2a6..f6b60ab77355 100644
--- a/xen/arch/arm/kernel.c
+++ b/xen/arch/arm/kernel.c
@@ -292,6 +292,12 @@ static __init int kernel_decompress(struct bootmodule *mod)
  iounmap(input);
  vunmap(output);
  
+if ( rc )

+{
+free_domheap_pages(pages);


This breaks the build. I am not sure how I managed to test it before... 
I will send a v2.


Cheers,

--
Julien Grall

Re: [PATCH 00/14] Use const whether we point to literal strings (take 1)

[Julien Grall]

Hi,

On 05/04/2021 16:56, Julien Grall wrote:

Julien Grall (14):
   xen: Constify the second parameter of rangeset_new()
   xen/sched: Constify name and opt_name in struct scheduler
   xen/x86: shadow: The return type of sh_audit_flags() should be const >
tools/firmware: hvmloader: Use const in __bug() and __assert_failed()
   tools/kdd: Use const whenever we point to literal strings
   tools/xentrace: Use const whenever we point to literal strings


I have merged the above patches. The rest still needs some reviews or 
respin (for patch #4).


Cheers,

--
Julien Grall

Re: [PATCH 08/14] tools/firmware: hvmloader: Use const in __bug() and __assert_failed()

[Julien Grall]

Hi Roger,

On 06/04/2021 08:29, Roger Pau Monné wrote:

On Mon, Apr 05, 2021 at 04:57:07PM +0100, Julien Grall wrote:

From: Julien Grall 

__bug() and __assert_failed() are not meant to modify the string
parameters. So mark them as const.

Signed-off-by: Julien Grall 


Reviewed-by: Roger Pau Monné 


Thanks!



While looking at this I think we should also make the line parameter
unsigned, but again doesn't need to be part of this patch.

I would prefer if this is done in a follow-up patch.

Cheers,

--
Julien Grall

Re: multiboot2 and module2 boot issues via GRUB2

[Andrew Cooper]

On 06/04/2021 19:03, Roman Shaposhnik wrote:
> On Tue, Apr 6, 2021 at 10:51 AM Andrew Cooper
> mailto:andrew.coop...@citrix.com>> wrote:
>
> On 06/04/2021 09:19, Jan Beulich wrote:
> > On 01.04.2021 21:43, Andrew Cooper wrote:
> >> On 01/04/2021 09:44, Roger Pau Monné wrote:
> >>> On Thu, Apr 01, 2021 at 09:31:07AM +0200, Jan Beulich wrote:
>  On 01.04.2021 03:06, Roman Shaposhnik wrote:
> > And the obvious next question: is my EVE usecase esoteric
> enough that
> > I should just go ahead and do a custom GRUB patch or is
> there a more
> > general interest in this?
>  Not sure if it ought to be a grub patch - the issue could as well
>  be dealt with in Xen, by concatenating modules to form a
> monolithic
>  initrd.
> >>> I would rather have it done in the loader than Xen, mostly because
> >>> it's a Linux boot specific format, and hence I don't think Xen
> should
> >>> have any knowledge about it.
> >>>
> >>> If it turns out to be impossible to implement on the loader
> side we
> >>> should consider doing it in Xen, but that's not my first option.
> >> Concatenating random things which may or may not be initrds is
> >> absolutely not something Xen should do.  We don't have enough
> context to
> >> do it safely/sensibly.
> > Well, I wasn't suggesting anywhere to concatenate random things.
> > Instead I was envisioning a command line option giving us the
> > context we need (e.g. "initrd=3+5").
>
> That's a massive layering violation, and incredibly fragile to the
> order
> of lines in the boot stanza.
>
>
> Don't have enough karma to argue Xen architectural side of things, but...
>  
>
> Either fix it by using a single monolithic initrd, which has worked
> perfectly well for the past 2 decades
>
>
> ...just to point out the obvious here:  even Debian who can HARDLY be
> blamed for tracking the bleeding edge has been recommending this
> for quite some time:
>  
>   
> https://wiki.debian.org/DebianInstaller/NetbootFirmware#The_Solution:_Add_Firmware_to_Initramfs
> 
> Obviously there's no way to do that with Xen today out of the box.

?

Those instructions do work out of the box with Xen.

It seems that pxelinux gained support for multiple initrd fragments in
3.05, but the sum total of documentation I can find is in the
changelog.  iPXE might have this ability, but it's documentation is
self-contradictory on the matter.

>
> Now, you can turn around and say "well, how hard could it be to
> concatenate?". That's fair. But it is also fair to point out that
> everytime
> we do that we portray Xen as "not quite as user friendly as X" (and
> this is, of course, pure perception -- but if we want users to stick
> perception matters a lot).

It's distinctly not trivial to do correctly at the Xen level, and fairly
invasive in at least two areas of logic.  Specific firmware layouts and
multiple fragments might even be impossible to concatenate, and its
better for the bootloaders to bail if they can't make the memory layout
work, than to leave Xen holding the the pieces and unable to boot. 
Either way, it would appear that common bootloaders already have logic
for this, which gets you a lot further than starting from scratch in Xen.


Furthermore, I don't think it fair to claim that this is a usability bug
in Xen, when what the Linux people have done is upstream Linux-specific
hacks to bootloaders.  Fixing the bootloaders to not have useful
features be Linux specific benefits everyone using Multiboot, not just
the Xen ecosystem.  If you're looking for general betterness to all OSS,
then fixing the bootloaders is clearly the better approach.

~Andrew

Re: [PATCH 0/2] xen/arm: Couple of bug fixes when decompressing kernels

[Julien Grall]

Hi,

On 02/04/2021 16:21, Julien Grall wrote:

From: Julien Grall 

Hi all,

The main goal of this series is to address the bug report [1]. It is not
possible

While testing the series, I also noticed that it is not possible to
re-use the same compressed kernel for multiple domains as the module
will be overwritten during the first decompression.

I am still a bit undecided how to fix it. We can either create a new
module for the uncompressed kernel and link with the compressed kernel.
Or we could decompress everytime.

One inconvenience to kernel to only decompress once is we have to keep
it until all the domains have booted. This may means a lot of memory to be
wasted just for keeping the uncompressed kernel (one my setup, it takes
~3 times more space).

Any opinions on how to approach it?

Cheers,

[1] https://lists.xenproject.org/archives/html/xen-users/2021-02/msg00027.html

Julien Grall (2):
   xen/arm: kernel: Propagate the error if we fail to decompress the
 kernel
   xen/gunzip: Allow perform_gunzip() to be called multiple times


I have committed the second patch as it is independent. The first patch 
still need some reviews.


Cheers,

--
Julien Grall

[xen-unstable test] 160758: regressions - FAIL

[osstest service owner]

flight 160758 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/160758/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-libvirt broken  in 160665
 test-armhf-armhf-xl-cubietruckbroken in 160715
 test-armhf-armhf-xl-arndale  broken  in 160715
 test-armhf-armhf-xl-arndale  19 guest-start.2  fail in 160733 REGR. vs. 160646
 test-armhf-armhf-xl-multivcpu 18 guest-start/debian.repeat fail in 160733 
REGR. vs. 160646
 test-armhf-armhf-xl-vhd 17 guest-start/debian.repeat fail in 160733 REGR. vs. 
160646

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt 5 host-install(5) broken in 160665 pass in 160758
 test-armhf-armhf-xl-arndale  5 host-install(5) broken in 160715 pass in 160758
 test-armhf-armhf-xl-cubietruck 5 host-install(5) broken in 160715 pass in 
160758
 test-arm64-arm64-xl-seattle   8 xen-boot fail in 160665 pass in 160758
 test-armhf-armhf-xl-rtds 18 guest-start/debian.repeat fail in 160665 pass in 
160758
 test-armhf-armhf-examine  8 reboot   fail in 160665 pass in 160758
 test-armhf-armhf-xl-arndale   8 xen-boot fail in 160665 pass in 160758
 test-armhf-armhf-xl-vhd 12 debian-di-install fail in 160715 pass in 160745
 test-armhf-armhf-xl   8 xen-boot fail in 160715 pass in 160758
 test-armhf-armhf-libvirt 14 guest-start  fail in 160715 pass in 160758
 test-armhf-armhf-xl-credit1 18 guest-start/debian.repeat fail in 160733 pass 
in 160715
 test-armhf-armhf-libvirt 18 guest-start/debian.repeat fail in 160733 pass in 
160758
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail in 
160733 pass in 160758
 test-armhf-armhf-libvirt-raw 12 debian-di-install fail in 160733 pass in 160758
 test-armhf-armhf-xl-vhd  13 guest-start  fail in 160745 pass in 160733
 test-armhf-armhf-xl-credit1  14 guest-start  fail in 160745 pass in 160733
 test-armhf-armhf-xl-arndale 18 guest-start/debian.repeat fail in 160745 pass 
in 160733
 test-armhf-armhf-xl-credit2  14 guest-start  fail in 160745 pass in 160758
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 14 
guest-start/debianhvm.repeat fail in 160745 pass in 160758
 test-armhf-armhf-libvirt  8 xen-boot fail in 160745 pass in 160758
 test-armhf-armhf-xl-rtds  8 xen-boot fail in 160745 pass in 160758
 test-armhf-armhf-libvirt-raw 13 guest-start  fail in 160745 pass in 160758
 test-armhf-armhf-xl-credit2  18 guest-start/debian.repeat  fail pass in 160665
 test-armhf-armhf-libvirt-raw 17 guest-start/debian.repeat  fail pass in 160715
 test-armhf-armhf-xl-multivcpu 14 guest-start   fail pass in 160733
 test-armhf-armhf-xl-vhd   8 xen-boot   fail pass in 160745
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 
160745
 test-armhf-armhf-xl-credit1   8 xen-boot   fail pass in 160745
 test-armhf-armhf-xl-arndale  14 guest-startfail pass in 160745

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check fail in 160733 never 
pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check fail in 160733 
never pass
 test-armhf-armhf-xl-vhd 14 migrate-support-check fail in 160733 never pass
 test-armhf-armhf-xl-vhd 15 saverestore-support-check fail in 160733 never pass
 test-armhf-armhf-xl-credit1 15 migrate-support-check fail in 160733 never pass
 test-armhf-armhf-xl-credit1 16 saverestore-support-check fail in 160733 never 
pass
 test-armhf-armhf-xl-arndale 15 migrate-support-check fail in 160745 never pass
 test-armhf-armhf-xl-arndale 16 saverestore-support-check fail in 160745 never 
pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stopfail like 160646
 test-armhf-armhf-libvirt 16 saverestore-support-checkfail  like 160646
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 160646
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 160646
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 160646
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop fail like 160646
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop fail like 160646
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stopfail like 160646
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop fail like 160646
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop fail like 160646
 test-armhf-armhf-libvirt-raw 15 saverestore-support-checkfail  like 160646
 test-amd64-i386-xl-pvshim14 guest-start  fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt   

Re: [PATCH 04/14] xen/char: console: Use const whenever we point to literal strings

[Julien Grall]

Hi Jan,

On 06/04/2021 09:10, Jan Beulich wrote:

On 05.04.2021 17:57, Julien Grall wrote:

--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -168,7 +168,7 @@ static int parse_guest_loglvl(const char *s);
  static char xenlog_val[LOGLVL_VAL_SZ];
  static char xenlog_guest_val[LOGLVL_VAL_SZ];
  
-static char *lvl2opt[] = { "none", "error", "warning", "info", "all" };

+static const char *lvl2opt[] = { "none", "error", "warning", "info", "all" };


If you add any const here, then I think you should go to full way
and also add the second missing one. Then
Reviewed-by: Jan Beulich 

Arguably the array should also be local to xenlog_update_val().


I will look at it and send a new version.

Cheers,

--
Julien Grall

Re: [PATCH 03/14] xen/x86: shadow: The return type of sh_audit_flags() should be const

[Julien Grall]

Hi Roger,

On 06/04/2021 08:24, Roger Pau Monné wrote:

On Mon, Apr 05, 2021 at 04:57:02PM +0100, Julien Grall wrote:

From: Julien Grall 

The function sh_audit_flags() is returning pointer to literal strings.
They should not be modified, so the return is now const and this is
propagated to the callers.

Take the opportunity to fix the coding style in the declaration of
sh_audit_flags.

Signed-off-by: Julien Grall 


While doing the cleanup I think you could narrow the scope of the 's'
variables also, but doesn't need to be part of this patch:


I think you are right. I will look at it as a follow-up.


Reviewed-by: Roger Pau Monné 


Thanks for the review!

Cheers,

--
Julien Grall

Re: [PATCH 02/14] xen/sched: Constify name and opt_name in struct scheduler

[Julien Grall]

Hi Jan,

On 06/04/2021 09:07, Jan Beulich wrote:

On 05.04.2021 17:57, Julien Grall wrote:

From: Julien Grall 

Both name and opt_name are pointing to literal string. So mark both of
the fields as const.

Signed-off-by: Julien Grall 


Reviewed-by: Jan Beulich 
albeit ...


--- a/xen/common/sched/private.h
+++ b/xen/common/sched/private.h
@@ -272,8 +272,8 @@ static inline spinlock_t *pcpu_schedule_trylock(unsigned 
int cpu)
  }
  
  struct scheduler {

-char *name; /* full name for this scheduler  */
-char *opt_name; /* option name for this scheduler*/
+const char *name;   /* full name for this scheduler  */
+const char *opt_name;   /* option name for this scheduler*/


... I'd like to suggest considering at least the latter to become
an array instead of a pointer - there's little point wasting 8
bytes of storage for the pointer when the strings pointed to are
all at most 9 bytes long (right now; I don't expect much longer
ones to appear).


I have tried this simple/dumb change on top of my patch:

diff --git a/xen/common/sched/private.h b/xen/common/sched/private.h
index a870320146ef..ab2236874217 100644
--- a/xen/common/sched/private.h
+++ b/xen/common/sched/private.h
@@ -273,7 +273,7 @@ static inline spinlock_t 
*pcpu_schedule_trylock(unsigned int cpu)


 struct scheduler {
 const char *name;   /* full name for this scheduler  */
-const char *opt_name;   /* option name for this scheduler*/
+const char opt_name[9]; /* option name for this scheduler*/
 unsigned int sched_id;  /* ID for this scheduler */
 void *sched_data;   /* global data pointer   */
 struct cpupool *cpupool;/* points to this scheduler's pool   */

GCC will throw an error:

core.c: In function ‘scheduler_init’:
core.c:2987:17: error: assignment of read-only variable ‘ops’
 ops = *schedulers[i];
 ^
core.c:2997:21: error: assignment of read-only variable ‘ops’
 ops = *schedulers[i];
 ^

I don't particularly want to drop the const. So the code would probably 
need some rework.


My patch doesn't change the size of the structure, so I would prefer to 
keep this patch as-is.


Cheers,

--
Julien Grall

Re: multiboot2 and module2 boot issues via GRUB2

[Roman Shaposhnik]

On Tue, Apr 6, 2021 at 10:51 AM Andrew Cooper 
wrote:

> On 06/04/2021 09:19, Jan Beulich wrote:
> > On 01.04.2021 21:43, Andrew Cooper wrote:
> >> On 01/04/2021 09:44, Roger Pau Monné wrote:
> >>> On Thu, Apr 01, 2021 at 09:31:07AM +0200, Jan Beulich wrote:
>  On 01.04.2021 03:06, Roman Shaposhnik wrote:
> > And the obvious next question: is my EVE usecase esoteric enough that
> > I should just go ahead and do a custom GRUB patch or is there a more
> > general interest in this?
>  Not sure if it ought to be a grub patch - the issue could as well
>  be dealt with in Xen, by concatenating modules to form a monolithic
>  initrd.
> >>> I would rather have it done in the loader than Xen, mostly because
> >>> it's a Linux boot specific format, and hence I don't think Xen should
> >>> have any knowledge about it.
> >>>
> >>> If it turns out to be impossible to implement on the loader side we
> >>> should consider doing it in Xen, but that's not my first option.
> >> Concatenating random things which may or may not be initrds is
> >> absolutely not something Xen should do.  We don't have enough context to
> >> do it safely/sensibly.
> > Well, I wasn't suggesting anywhere to concatenate random things.
> > Instead I was envisioning a command line option giving us the
> > context we need (e.g. "initrd=3+5").
>
> That's a massive layering violation, and incredibly fragile to the order
> of lines in the boot stanza.
>

Don't have enough karma to argue Xen architectural side of things, but...


> Either fix it by using a single monolithic initrd, which has worked
> perfectly well for the past 2 decades
>

...just to point out the obvious here:  even Debian who can HARDLY be
blamed for tracking the bleeding edge has been recommending this
for quite some time:

https://wiki.debian.org/DebianInstaller/NetbootFirmware#The_Solution:_Add_Firmware_to_Initramfs
Obviously there's no way to do that with Xen today out of the box.

Now, you can turn around and say "well, how hard could it be to
concatenate?". That's fair. But it is also fair to point out that everytime
we do that we portray Xen as "not quite as user friendly as X" (and
this is, of course, pure perception -- but if we want users to stick
perception matters a lot).

Thanks,
Roman.

P.S. Curiously enough, saying that -- I'm undermining my own case -- it is
actually
*easier* for me to hack GRUB in EVE. But I felt like stating the obvious
anyway.

Re: [PATCH 01/14] xen: Constify the second parameter of rangeset_new()

[Julien Grall]

Hi Jan,

On 06/04/2021 08:57, Jan Beulich wrote:

On 05.04.2021 17:57, Julien Grall wrote:

From: Julien Grall 

The string 'name' will never get modified by the function, so mark it
as const.

Signed-off-by: Julien Grall 


Reviewed-by: Jan Beulich 


Thanks!




--- a/xen/common/rangeset.c
+++ b/xen/common/rangeset.c
@@ -421,7 +421,7 @@ bool_t rangeset_is_empty(
  }
  
  struct rangeset *rangeset_new(

-struct domain *d, char *name, unsigned int flags)
+struct domain *d, const char *name, unsigned int flags)
  {
  struct rangeset *r;


Remotely along these lines the function also has no need anymore to
use snprintf() - safe_strcpy() very well fits both purposes. But
quite likely for another patch.


I saw you already sent the patch for that. So I am assuming there is no 
action for me here.


Cheers,

--
Julien Grall

Re: [PATCH 00/14] Use const whether we point to literal strings (take 1)

[Julien Grall]

Hi Elliott,

Thanks for the input!

On 05/04/2021 18:01, Elliott Mitchell wrote:

On Mon, Apr 05, 2021 at 04:56:59PM +0100, Julien Grall wrote:

I am not aware of code trying to modify literal strings in Xen.
However, there is a frequent use of non-const char * to point to
literal strings. Given the size of the codebase, there is a risk
to involuntarily introduce code that will modify literal strings.

Therefore it would be better to enforce using const when pointing
to such strings. Both GCC and Clang provide an option to warn
for such case (see -Wwrite-strings) and therefore could be used
by Xen.

This series doesn't yet make use of -Wwrite-strings because
the tree is not fully converted. Instead, it contains some easy
and likely non-controversial use const in the code.

The major blockers to enable -Wwrite-strings are the following:
 - xen/common/efi: union string is used in both const and
 non-const situation. It doesn't feel right to specific one member
 const and the other non-const.
 - libxl: the major block is the flexarray framework as we would use
 it with string (now const char*). I thought it would be possible to
 make the interface const, but it looks like there are a couple of
 places where we need to modify the content (such as in
 libxl_json.c).

Ideally, I would like to have -Wwrite-strings unconditionally used
tree-wide. But, some of the area may required some heavy refactoring.

One solution would be to enable it tree-wide but turned it off at a
directroy/file level.

Any opinions?


I think doing such is a Good Idea(tm).  Adding consts adds opportunities
for greater optimization.  Both by compilers which can avoid extra
copies, and developers who then know they don't need to generate extra
copies to sacrific them to an API.  In particular the consts also
function as documentation.

So you're certainly not the only person who thinks additional consts
would be a good thing:

https://lists.xenproject.org/archives/html/xen-devel/2021-01/msg00132.html


Alas merely getting the two const patches into the latest release
apparently wasn't even worthy of response:

https://lists.xenproject.org/archives/html/xen-devel/2021-02/msg01040.html


Sory to hear you had no response. Would you be able to give a nudge to 
the maintainers?





I agree this should be done, though I'm aware many developers hate
bothering with constants.




Cheers,

--
Julien Grall

Re: multiboot2 and module2 boot issues via GRUB2

[Andrew Cooper]

On 06/04/2021 09:19, Jan Beulich wrote:
> On 01.04.2021 21:43, Andrew Cooper wrote:
>> On 01/04/2021 09:44, Roger Pau Monné wrote:
>>> On Thu, Apr 01, 2021 at 09:31:07AM +0200, Jan Beulich wrote:
 On 01.04.2021 03:06, Roman Shaposhnik wrote:
> And the obvious next question: is my EVE usecase esoteric enough that
> I should just go ahead and do a custom GRUB patch or is there a more
> general interest in this?
 Not sure if it ought to be a grub patch - the issue could as well
 be dealt with in Xen, by concatenating modules to form a monolithic
 initrd.
>>> I would rather have it done in the loader than Xen, mostly because
>>> it's a Linux boot specific format, and hence I don't think Xen should
>>> have any knowledge about it.
>>>
>>> If it turns out to be impossible to implement on the loader side we
>>> should consider doing it in Xen, but that's not my first option.
>> Concatenating random things which may or may not be initrds is
>> absolutely not something Xen should do.  We don't have enough context to
>> do it safely/sensibly.
> Well, I wasn't suggesting anywhere to concatenate random things.
> Instead I was envisioning a command line option giving us the
> context we need (e.g. "initrd=3+5").

That's a massive layering violation, and incredibly fragile to the order
of lines in the boot stanza.

Either fix it by using a single monolithic initrd, which has worked
perfectly well for the past 2 decades, or add a feature to grub to get
initrd-like behaviour for MB2 too.  I will object very strongly to any
Xen patches trying to do this.

~Andrew

Re: multiboot2 and module2 boot issues via GRUB2

[Roman Shaposhnik]

On Tue, Apr 6, 2021 at 1:19 AM Jan Beulich  wrote:

> On 01.04.2021 21:43, Andrew Cooper wrote:
> > On 01/04/2021 09:44, Roger Pau Monné wrote:
> >> On Thu, Apr 01, 2021 at 09:31:07AM +0200, Jan Beulich wrote:
> >>> On 01.04.2021 03:06, Roman Shaposhnik wrote:
>  And the obvious next question: is my EVE usecase esoteric enough that
>  I should just go ahead and do a custom GRUB patch or is there a more
>  general interest in this?
> >>> Not sure if it ought to be a grub patch - the issue could as well
> >>> be dealt with in Xen, by concatenating modules to form a monolithic
> >>> initrd.
> >> I would rather have it done in the loader than Xen, mostly because
> >> it's a Linux boot specific format, and hence I don't think Xen should
> >> have any knowledge about it.
> >>
> >> If it turns out to be impossible to implement on the loader side we
> >> should consider doing it in Xen, but that's not my first option.
> >
> > Concatenating random things which may or may not be initrds is
> > absolutely not something Xen should do.  We don't have enough context to
> > do it safely/sensibly.
>
> Well, I wasn't suggesting anywhere to concatenate random things.
> Instead I was envisioning a command line option giving us the
> context we need (e.g. "initrd=3+5").
>

That's actually not a bad idea at all -- I may look into how feasible it
would
be to add on Xen side. GRUB side is trivial (but I'm not sure upstream folks
would take it).

Thanks,
Roman.

Re: [PATCH 5/5] x86: don't build unused entry code when !PV32

[Andrew Cooper]

On 01/04/2021 15:37, Jan Beulich wrote:
> On 01.04.2021 16:31, Andrew Cooper wrote:
>> On 25/11/2020 08:51, Jan Beulich wrote:
>>> @@ -102,19 +102,21 @@ void __dummy__(void)
>>>  BLANK();
>>>  #endif
>>>  
>>> -#ifdef CONFIG_PV
>>> +#ifdef CONFIG_PV32
>>>  OFFSET(DOMAIN_is_32bit_pv, struct domain, arch.pv.is_32bit);
>> Even if PV32 is compiled out, the is_32bit field still exists, and is
>> still necessary for crash analysis.  XCA parses this offset information
>> as part of dissecting /proc/vmcore.
>>
>> It's one single bool in a fixed size allocation which we've got plenty
>> of room in.  It can and should stay to avoid impacting the existing
>> diagnostic tools.
> I'm afraid I don't understand at all: I'm not removing the field.

You talked about removing it in the commit message.

> All I'm removing is the entry for it in asm-offsets.h.

Yes, and that will break XCA, which is used by several downstreams, not
just XenServer.

For RPM package reasons, you can't use debuginfo packages, because
what's on disk doesn't match what's in memory until you've rebooted. 
Livepatching adds an extra dimension of fun here.  There's not enough
space in the vmcoreinfo page to pass enough structure information, so
asm offsets is appended to the symbol table.  Yes its a gross hack, but
its how things currently work.

~Andrew

Re: [PATCH v2 3/3] x86: avoid building COMPAT code when !HVM && !PV32

[Wei Liu]

On Tue, Apr 06, 2021 at 04:02:08PM +0200, Jan Beulich wrote:
> It was probably a mistake to, over time, drop various CONFIG_COMPAT
> conditionals from x86-specific code, as we now have a build
> configuration again where we'd prefer this to be unset. Arrange for
> CONFIG_COMPAT to actually be off in this case, dealing with fallout.
> 
> Signed-off-by: Jan Beulich 

Reviewed-by: Wei Liu 

Re: [PATCH v2 2/3] x86: slim down hypercall handling when !PV32

[Wei Liu]

On Tue, Apr 06, 2021 at 04:01:41PM +0200, Jan Beulich wrote:
> In such a build various of the compat handlers aren't needed. Don't
> reference them from the hypercall table, and compile out those which
> aren't needed for HVM. Also compile out switch_compat(), which has no
> purpose in such a build.
> 
> Signed-off-by: Jan Beulich 

Reviewed-by: Wei Liu 

Re: [PATCH v2 1/3] x86: don't build unused entry code when !PV32

[Wei Liu]

On Tue, Apr 06, 2021 at 04:01:22PM +0200, Jan Beulich wrote:
> Except for the initial part of cstar_enter compat/entry.S is all dead
> code in this case. Further, along the lines of the PV conditionals we
> already have in entry.S, make code PV32-conditional there too (to a
> fair part because this code actually references compat/entry.S).
> 
> This has the side effect of moving the tail part (now at compat_syscall)
> of the code out of .text.entry (in line with e.g. compat_sysenter).
> 
> Signed-off-by: Jan Beulich 

Reviewed-by: Wei Liu 

Re: [PATCH for-4.15 1/7] CHANGELOG.md: Mention XEN_SCRIPT_DIR

[Ian Jackson]

George Dunlap writes ("[PATCH for-4.15 1/7] CHANGELOG.md: Mention 
XEN_SCRIPT_DIR"):
> Signed-off-by: George Dunlap 

Reviewed-by: Ian Jackson 

Re: [PATCH 2/2] xen/pci: Gate all MSI code in common code with CONFIG_HAS_PCI_MSI

[Roger Pau Monné]

On Tue, Apr 06, 2021 at 04:09:34PM +0100, Julien Grall wrote:
> 
> 
> On 06/04/2021 15:59, Roger Pau Monné wrote:
> > On Tue, Apr 06, 2021 at 03:30:01PM +0100, Julien Grall wrote:
> > > So I think we want to be able to compile out the code if not used. That
> > > said, I think providing stub would be better to avoid multiple #ifdef in 
> > > the
> > > same function.
> > 
> > I think providing stubs is the way to go, that should allow to remove
> > the unneeded code without having to explicitly drop MSI support. As
> > said before, I think it's fine to provide those unimplemented for Arm
> > ATM, can be filled later if there's more pressing PCI work to do
> > first.
> 
> We should remove unneeded and *avoid* allocation. Providing stub for
> existing functions will only address the first problem.
> 
> For the allocation (see alloc_pdev()) , we will need to move it in separate
> function and gate them to prevent the allocation.
> 
> It would be wrong to gate the code with #ifdef CONFIG_X86. So I think
> Rahul's idea to provide the new #ifdef is correct.

I think all this needs to be in the commit message then, because from
my reading of the current message it seems like MSI code is only
removed because MSI support is not implemented on Arm, rather than Arm
not requiring such strict tracking of MSI accesses and MSI interrupt
setup. Likely the naming of the option needs to be adjusted also
together with the reduction of it's scope to stuff that explicitly
needs to be removed in the preprocessor opposed to adding arch
specific stubs.

Thanks, Roger.

block script performance

[Marek Marczykowski-Górecki]

Hi,

Some time ago I've done an analysis what's the easiest way to improve VM
startup time, and one of the things that really stands out is block
script. VM with 4 disks can take up to 2s on waiting for block scripts.
A while ago I proposed a patch[1] that short-circuits the block script for
the simplest case - already existing block device. There is also similar
slightly more complex but common case - a file-based disk, needing a
loop device.

While there are several ways I can fix this in Qubes-specific
configuration, I'd like to have a solution useful for upstream too. And
for loop devices it would be especially useful, as the Linux API here
has a _huge_ potential for all kind of races (resulting for example in a
wrong disk being connected to a VM). Currently those races are avoided
by the block script by taking a global lock, which also adds to the
performance issues...
Furthermore, I'd like a solution that is compatible with driver domains
(backend in a distinct place than the toolstack), preferably using the
same code path for both (to reduce regression risk, because most test
only dom0 case...).

Recently I've talked with Demi (in cc) about it, and we came to a conclusion
that it should be somewhere in libxl, in a place that is also called in
`xl devd` (to cover non-dom0 case). 
I think I know libxl well enough to find the right place to plug this
in, but I'd like to ask first, if this would be acceptable in upstream
libxl. Note this will be Linux-specific (both because of loop devices
part, but also because of writing xen-blkback specific xenstore entry).

[1] https://xen.markmail.org/thread/ytikjjbs4kpihzi5

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab


signature.asc
Description: PGP signature

Re: [PATCH 00/11] Rid W=1 warnings from Block

[Lee Jones]

On Tue, 06 Apr 2021, Jens Axboe wrote:

> On 3/12/21 3:55 AM, Lee Jones wrote:
> > This set is part of a larger effort attempting to clean-up W=1
> > kernel builds, which are currently overwhelmingly riddled with
> > niggly little warnings.
> 
> Applied 2-11, 1 is already in the my tree.

Superstar, thanks Jens.

-- 
Lee Jones [李琼斯]
Senior Technical Lead - Developer Services
Linaro.org │ Open source software for Arm SoCs
Follow Linaro: Facebook | Twitter | Blog

Re: [PATCH 2/2] xen/pci: Gate all MSI code in common code with CONFIG_HAS_PCI_MSI

[Jan Beulich]

On 06.04.2021 16:30, Julien Grall wrote:
> Hi Roger,
> 
> On 06/04/2021 15:13, Roger Pau Monné wrote:
>> On Tue, Apr 06, 2021 at 12:39:11PM +0100, Rahul Singh wrote:
>>> MSI support is not implemented for ARM architecture but it is enabled
>>> for x86 architecture and referenced in common passthrough/pci.c code.
>>>
>>> Therefore introducing the new flag to gate the MSI code for ARM in
>>> common code to avoid compilation error when HAS_PCI is enabled for ARM.
>>
>> Is such option really interesting long term?
>>
>> IIRC PCI Express mandates MSI support, at which point I don't see much
>> value in being able to compile out the MSI support.
> 
> I am pretty sure there are board out with PCI support but no MSI 
> support. Anyway, even if the spec may mandate it...
> 
>>
>> So while maybe helpful for Arm PCI efforts ATM, I'm not sure it
>> warrants a Kconfig option, I would rather see Arm introduce dummy
>> helpers for the missing functionality, even if unimplemented at the
>> moment.
> 
> ... from my understanding, most of (if not all) the MSI code is not very 
> useful on Arm when using the GICv3 ITS.
> 
> The GICv3 ITS will do the isolation for you and therefore we should not 
> need to keep track of the state at the vPCI level.

But that's then not "has PCI MSI" but "need to intercept PCI MSI
accesses", i.e. I don't think the Kconfig option is correctly
named. If a device with MSI support is used, you can't make that
MSI support go away, after all.

And of course I agree with the desire to have less #ifdef-ary
here.

Jan

Re: [PATCH 00/11] Rid W=1 warnings from Block

[Jens Axboe]

On 3/12/21 3:55 AM, Lee Jones wrote:
> This set is part of a larger effort attempting to clean-up W=1
> kernel builds, which are currently overwhelmingly riddled with
> niggly little warnings.

Applied 2-11, 1 is already in the my tree.

-- 
Jens Axboe

Re: [PATCH 1/2] xen/pci: Move PCI ATS code to common directory

[Jan Beulich]

On 06.04.2021 13:39, Rahul Singh wrote:
> PCI ATS code is common for all architecture, move code to common
> directory to be usable for other architectures.
> 
> No functional change intended.
> 
> Signed-off-by: Rahul Singh 

Acked-by: Jan Beulich 

Re: [PATCH 2/2] xen/pci: Gate all MSI code in common code with CONFIG_HAS_PCI_MSI

[Julien Grall]

On 06/04/2021 15:59, Roger Pau Monné wrote:

On Tue, Apr 06, 2021 at 03:30:01PM +0100, Julien Grall wrote:

Hi Roger,

On 06/04/2021 15:13, Roger Pau Monné wrote:

On Tue, Apr 06, 2021 at 12:39:11PM +0100, Rahul Singh wrote:

MSI support is not implemented for ARM architecture but it is enabled
for x86 architecture and referenced in common passthrough/pci.c code.

Therefore introducing the new flag to gate the MSI code for ARM in
common code to avoid compilation error when HAS_PCI is enabled for ARM.


Is such option really interesting long term?

IIRC PCI Express mandates MSI support, at which point I don't see much
value in being able to compile out the MSI support.


I am pretty sure there are board out with PCI support but no MSI support.
Anyway, even if the spec may mandate it...



So while maybe helpful for Arm PCI efforts ATM, I'm not sure it
warrants a Kconfig option, I would rather see Arm introduce dummy
helpers for the missing functionality, even if unimplemented at the
moment.


... from my understanding, most of (if not all) the MSI code is not very
useful on Arm when using the GICv3 ITS.

The GICv3 ITS will do the isolation for you and therefore we should not need
to keep track of the state at the vPCI level.


Right, but MSI has nothing to do with isolation, is just the
capability to setup interrupts from PCI devices. What about systems
without GICv3 ITS, is there an aim to support those also? (as from my
reading of your reply those would require more auditing of the MSI
accesses by the guests)


I am not aware of any plan for them so far.




So I think we want to be able to compile out the code if not used. That
said, I think providing stub would be better to avoid multiple #ifdef in the
same function.


I think providing stubs is the way to go, that should allow to remove
the unneeded code without having to explicitly drop MSI support. 
As

said before, I think it's fine to provide those unimplemented for Arm
ATM, can be filled later if there's more pressing PCI work to do
first.


We should remove unneeded and *avoid* allocation. Providing stub for 
existing functions will only address the first problem.


For the allocation (see alloc_pdev()) , we will need to move it in 
separate function and gate them to prevent the allocation.


It would be wrong to gate the code with #ifdef CONFIG_X86. So I think 
Rahul's idea to provide the new #ifdef is correct.


Cheers,

--
Julien Grall

Re: [PATCH 2/2] xen/pci: Gate all MSI code in common code with CONFIG_HAS_PCI_MSI

[Roger Pau Monné]

On Tue, Apr 06, 2021 at 03:30:01PM +0100, Julien Grall wrote:
> Hi Roger,
> 
> On 06/04/2021 15:13, Roger Pau Monné wrote:
> > On Tue, Apr 06, 2021 at 12:39:11PM +0100, Rahul Singh wrote:
> > > MSI support is not implemented for ARM architecture but it is enabled
> > > for x86 architecture and referenced in common passthrough/pci.c code.
> > > 
> > > Therefore introducing the new flag to gate the MSI code for ARM in
> > > common code to avoid compilation error when HAS_PCI is enabled for ARM.
> > 
> > Is such option really interesting long term?
> > 
> > IIRC PCI Express mandates MSI support, at which point I don't see much
> > value in being able to compile out the MSI support.
> 
> I am pretty sure there are board out with PCI support but no MSI support.
> Anyway, even if the spec may mandate it...
> 
> > 
> > So while maybe helpful for Arm PCI efforts ATM, I'm not sure it
> > warrants a Kconfig option, I would rather see Arm introduce dummy
> > helpers for the missing functionality, even if unimplemented at the
> > moment.
> 
> ... from my understanding, most of (if not all) the MSI code is not very
> useful on Arm when using the GICv3 ITS.
> 
> The GICv3 ITS will do the isolation for you and therefore we should not need
> to keep track of the state at the vPCI level.

Right, but MSI has nothing to do with isolation, is just the
capability to setup interrupts from PCI devices. What about systems
without GICv3 ITS, is there an aim to support those also? (as from my
reading of your reply those would require more auditing of the MSI
accesses by the guests)

> So I think we want to be able to compile out the code if not used. That
> said, I think providing stub would be better to avoid multiple #ifdef in the
> same function.

I think providing stubs is the way to go, that should allow to remove
the unneeded code without having to explicitly drop MSI support. As
said before, I think it's fine to provide those unimplemented for Arm
ATM, can be filled later if there's more pressing PCI work to do
first.

Thanks, Roger.

Re: [PATCH 2/2] xen/pci: Gate all MSI code in common code with CONFIG_HAS_PCI_MSI

[Julien Grall]

Hi Roger,

On 06/04/2021 15:13, Roger Pau Monné wrote:

On Tue, Apr 06, 2021 at 12:39:11PM +0100, Rahul Singh wrote:

MSI support is not implemented for ARM architecture but it is enabled
for x86 architecture and referenced in common passthrough/pci.c code.

Therefore introducing the new flag to gate the MSI code for ARM in
common code to avoid compilation error when HAS_PCI is enabled for ARM.


Is such option really interesting long term?

IIRC PCI Express mandates MSI support, at which point I don't see much
value in being able to compile out the MSI support.


I am pretty sure there are board out with PCI support but no MSI 
support. Anyway, even if the spec may mandate it...




So while maybe helpful for Arm PCI efforts ATM, I'm not sure it
warrants a Kconfig option, I would rather see Arm introduce dummy
helpers for the missing functionality, even if unimplemented at the
moment.


... from my understanding, most of (if not all) the MSI code is not very 
useful on Arm when using the GICv3 ITS.


The GICv3 ITS will do the isolation for you and therefore we should not 
need to keep track of the state at the vPCI level.


So I think we want to be able to compile out the code if not used. That 
said, I think providing stub would be better to avoid multiple #ifdef in 
the same function.


Cheers,

--
Julien Grall

Re: [PATCH 02/14] xen/sched: Constify name and opt_name in struct scheduler

[George Dunlap]

> On Apr 5, 2021, at 4:57 PM, Julien Grall  wrote:
> 
> From: Julien Grall 
> 
> Both name and opt_name are pointing to literal string. So mark both of
> the fields as const.
> 
> Signed-off-by: Julien Grall 

Acked-by: George Dunlap 

Re: [PATCH 14/14] tools/xentrace: Use const whenever we point to literal strings

[George Dunlap]

> On Apr 5, 2021, at 4:57 PM, Julien Grall  wrote:
> 
> From: Julien Grall 
> 
> literal strings are not meant to be modified. So we should use const
> char * rather than char * when we want to store a pointer to them.
> 
> Signed-off-by: Julien Grall 

Acked-by: George Dunlap 

Re: [PATCH 2/2] xen/pci: Gate all MSI code in common code with CONFIG_HAS_PCI_MSI

[Roger Pau Monné]

On Tue, Apr 06, 2021 at 12:39:11PM +0100, Rahul Singh wrote:
> MSI support is not implemented for ARM architecture but it is enabled
> for x86 architecture and referenced in common passthrough/pci.c code.
> 
> Therefore introducing the new flag to gate the MSI code for ARM in
> common code to avoid compilation error when HAS_PCI is enabled for ARM.

Is such option really interesting long term?

IIRC PCI Express mandates MSI support, at which point I don't see much
value in being able to compile out the MSI support.

So while maybe helpful for Arm PCI efforts ATM, I'm not sure it
warrants a Kconfig option, I would rather see Arm introduce dummy
helpers for the missing functionality, even if unimplemented at the
moment.

Thanks, Roger.

Re: [PATCH 0/2] xen/arm: Couple of bug fixes when decompressing kernels

[Julien Grall]

Hi Jan,

On 06/04/2021 08:45, Jan Beulich wrote:

On 02.04.2021 17:21, Julien Grall wrote:

From: Julien Grall 

Hi all,

The main goal of this series is to address the bug report [1]. It is not
possible


...?


Urgh, I forgot to add the rest of the sentence :/. I was meant to say 
that it is not possible to decompress multiple kernels (doesn't need to 
be the same) today.





While testing the series, I also noticed that it is not possible to
re-use the same compressed kernel for multiple domains as the module
will be overwritten during the first decompression.

I am still a bit undecided how to fix it. We can either create a new
module for the uncompressed kernel and link with the compressed kernel.
Or we could decompress everytime.

One inconvenience to kernel to only decompress once is we have to keep
it until all the domains have booted. This may means a lot of memory to be
wasted just for keeping the uncompressed kernel (one my setup, it takes
~3 times more space).

Any opinions on how to approach it?


Well, it's not "until all the domains have booted", but until all the
domains have had their kernel image placed in the designated piece of
memory. So while for the time being multiple decompression may indeed
be a reasonable approach, longer term one could populate all the
domains' memory in two steps - first just the kernel space for all of
them, then load the kernel(s), then populate the rest of the memory.


Right, I am not sure this brings us to a better position. We would want 
to use superpage mapping for the guest memory as much as possible for 
performance reason.


So we may end up to populate the full memory of each guests. Therefore, 
I am not sure the split is that worth it.


Cheers,

--
Julien Grall

Re: [PATCH 10/14] tools/kdd: Use const whenever we point to literal strings

[Tim Deegan]

At 16:57 +0100 on 05 Apr (1617641829), Julien Grall wrote:
> From: Julien Grall 
> 
> literal strings are not meant to be modified. So we should use const
> char * rather than char * when we want to shore a pointer to them.
> 
> Signed-off-by: Julien Grall 

Acked-by: Tim Deegan 

Thanks,

Tim.

[PATCH v2 3/3] x86: avoid building COMPAT code when !HVM && !PV32

[Jan Beulich]

It was probably a mistake to, over time, drop various CONFIG_COMPAT
conditionals from x86-specific code, as we now have a build
configuration again where we'd prefer this to be unset. Arrange for
CONFIG_COMPAT to actually be off in this case, dealing with fallout.

Signed-off-by: Jan Beulich 
---
v2: New.

--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -6,7 +6,6 @@ config X86
select ACPI
select ACPI_LEGACY_TABLES_LOOKUP
select ARCH_SUPPORTS_INT128
-   select COMPAT
select CORE_PARKING
select HAS_ALTERNATIVE
select HAS_CPUFREQ
@@ -57,6 +56,7 @@ config PV32
bool "Support for 32bit PV guests"
depends on PV
default y
+   select COMPAT
---help---
  The 32bit PV ABI uses Ring1, an area of the x86 architecture which
  was deprecated and mostly removed in the AMD64 spec.  As a result,
@@ -91,6 +91,7 @@ config PV_LINEAR_PT
 
 config HVM
def_bool !PV_SHIM_EXCLUSIVE
+   select COMPAT
select IOREQ_SERVER
prompt "HVM support"
---help---
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -50,7 +50,8 @@ obj-y += nmi.o
 obj-y += numa.o
 obj-y += pci.o
 obj-y += percpu.o
-obj-y += physdev.o x86_64/physdev.o
+obj-y += physdev.o
+obj-$(CONFIG_COMPAT) += x86_64/physdev.o
 obj-y += psr.o
 obj-y += setup.o
 obj-y += shutdown.o
@@ -72,7 +73,8 @@ obj-y += xstate.o
 
 ifneq ($(CONFIG_PV_SHIM_EXCLUSIVE),y)
 obj-y += domctl.o
-obj-y += platform_hypercall.o x86_64/platform_hypercall.o
+obj-y += platform_hypercall.o
+obj-$(CONFIG_COMPAT) += x86_64/platform_hypercall.o
 obj-y += sysctl.o
 endif
 
--- a/xen/arch/x86/cpu/mcheck/mce.c
+++ b/xen/arch/x86/cpu/mcheck/mce.c
@@ -1291,6 +1291,8 @@ static void x86_mc_mceinject(void *data)
 #error BITS_PER_LONG definition absent
 #endif
 
+# ifdef CONFIG_COMPAT
+
 # include 
 
 # define xen_mcinfo_msr  mcinfo_msr
@@ -1343,6 +1345,11 @@ CHECK_mcinfo_recovery;
 # undef xen_page_offline_action
 # undef xen_mcinfo_recovery
 
+# else
+#  define compat_handle_is_null(h) true
+#  define copy_to_compat(h, p, n)  true /* really (-EFAULT), but gcc chokes */
+# endif /* CONFIG_COMPAT */
+
 /* Machine Check Architecture Hypercall */
 long do_mca(XEN_GUEST_HANDLE_PARAM(xen_mc_t) u_xen_mc)
 {
@@ -1351,11 +1358,15 @@ long do_mca(XEN_GUEST_HANDLE_PARAM(xen_m
 struct vcpu *v = current;
 union {
 struct xen_mc_fetch *nat;
+#ifdef CONFIG_COMPAT
 struct compat_mc_fetch *cmp;
+#endif
 } mc_fetch;
 union {
 struct xen_mc_physcpuinfo *nat;
+#ifdef CONFIG_COMPAT
 struct compat_mc_physcpuinfo *cmp;
+#endif
 } mc_physcpuinfo;
 uint32_t flags, cmdflags;
 int nlcpu;
--- a/xen/arch/x86/cpu/vpmu.c
+++ b/xen/arch/x86/cpu/vpmu.c
@@ -39,10 +39,12 @@
 #include 
 #include 
 
+#ifdef CONFIG_COMPAT
 #include 
 CHECK_pmu_cntr_pair;
 CHECK_pmu_data;
 CHECK_pmu_params;
+#endif
 
 static unsigned int __read_mostly opt_vpmu_enabled;
 unsigned int __read_mostly vpmu_mode = XENPMU_MODE_OFF;
@@ -232,6 +234,7 @@ void vpmu_do_interrupt(struct cpu_user_r
 domid = sampled->domain->domain_id;
 
 /* Store appropriate registers in xenpmu_data */
+#ifdef CONFIG_COMPAT
 /* FIXME: 32-bit PVH should go here as well */
 if ( is_pv_32bit_vcpu(sampling) )
 {
@@ -254,6 +257,7 @@ void vpmu_do_interrupt(struct cpu_user_r
 *flags |= PMU_SAMPLE_USER;
 }
 else
+#endif
 {
 struct xen_pmu_regs *r = >xenpmu_data->pmu.r.regs;
 
@@ -448,7 +452,9 @@ static int vpmu_arch_initialise(struct v
 BUILD_BUG_ON(sizeof(struct xen_pmu_intel_ctxt) > XENPMU_CTXT_PAD_SZ);
 BUILD_BUG_ON(sizeof(struct xen_pmu_amd_ctxt) > XENPMU_CTXT_PAD_SZ);
 BUILD_BUG_ON(sizeof(struct xen_pmu_regs) > XENPMU_REGS_PAD_SZ);
+#ifdef CONFIG_COMPAT
 BUILD_BUG_ON(sizeof(struct compat_pmu_regs) > XENPMU_REGS_PAD_SZ);
+#endif
 
 ASSERT(!(vpmu->flags & ~VPMU_AVAILABLE) && !vpmu->context);
 
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -64,7 +64,9 @@
 #include 
 #include 
 #include 
+#ifdef CONFIG_COMPAT
 #include 
+#endif
 #include 
 #include 
 #include 
@@ -1020,11 +1022,13 @@ void arch_domain_creation_finished(struc
 hvm_domain_creation_finished(d);
 }
 
+#ifdef CONFIG_COMPAT
 #define xen_vcpu_guest_context vcpu_guest_context
 #define fpu_ctxt fpu_ctxt.x
 CHECK_FIELD_(struct, vcpu_guest_context, fpu_ctxt);
 #undef fpu_ctxt
 #undef xen_vcpu_guest_context
+#endif
 
 /* Called by XEN_DOMCTL_setvcpucontext and VCPUOP_initialise. */
 int arch_set_info_guest(
@@ -1045,7 +1049,11 @@ int arch_set_info_guest(
  * we expect the tools to DTRT even in compat-mode callers. */
 compat = is_pv_32bit_domain(d);
 
+#ifdef CONFIG_COMPAT
 #define c(fld) (compat ? (c.cmp->fld) : (c.nat->fld))
+#else
+#define c(fld) (c.nat->fld)
+#endif
 flags = c(flags);
 
 if ( is_pv_domain(d) )
@@ -1078,6 +1086,7 @@ int arch_set_info_guest(
 if ( 

[PATCH v2 2/3] x86: slim down hypercall handling when !PV32

[Jan Beulich]

In such a build various of the compat handlers aren't needed. Don't
reference them from the hypercall table, and compile out those which
aren't needed for HVM. Also compile out switch_compat(), which has no
purpose in such a build.

Signed-off-by: Jan Beulich 
---
v2: New.

--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -17,7 +17,8 @@ obj-bin-y += bzimage.init.o
 obj-bin-y += clear_page.o
 obj-bin-y += copy_page.o
 obj-y += cpuid.o
-obj-$(CONFIG_PV) += compat.o x86_64/compat.o
+obj-$(CONFIG_PV) += compat.o
+obj-$(CONFIG_PV32) += x86_64/compat.o
 obj-$(CONFIG_KEXEC) += crash.o
 obj-$(CONFIG_GDBSX) += debug.o
 obj-y += delay.o
--- a/xen/arch/x86/hvm/hypercall.c
+++ b/xen/arch/x86/hvm/hypercall.c
@@ -121,7 +121,9 @@ static long hvm_physdev_op(int cmd, XEN_
 
 #define do_arch_1 paging_domctl_continuation
 
-static const hypercall_table_t hvm_hypercall_table[] = {
+static const struct {
+hypercall_fn_t *native, *compat;
+} hvm_hypercall_table[] = {
 HVM_CALL(memory_op),
 #ifdef CONFIG_GRANT_TABLE
 HVM_CALL(grant_table_op),
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -4498,7 +4498,9 @@ long do_update_va_mapping_otherdomain(un
 
 return rc;
 }
+#endif /* CONFIG_PV */
 
+#ifdef CONFIG_PV32
 int compat_update_va_mapping(unsigned int va, uint32_t lo, uint32_t hi,
  unsigned int flags)
 {
@@ -4533,7 +4535,7 @@ int compat_update_va_mapping_otherdomain
 
 return rc;
 }
-#endif /* CONFIG_PV */
+#endif /* CONFIG_PV32 */
 
 typedef struct e820entry e820entry_t;
 DEFINE_XEN_GUEST_HANDLE(e820entry_t);
--- a/xen/arch/x86/pv/callback.c
+++ b/xen/arch/x86/pv/callback.c
@@ -19,12 +19,11 @@
 #include 
 #include 
 #include 
-#include 
-#include 
 
 #include 
 
 #include 
+#include 
 
 static int register_guest_nmi_callback(unsigned long address)
 {
@@ -203,6 +202,11 @@ long do_set_callbacks(unsigned long even
 return 0;
 }
 
+#ifdef CONFIG_PV32
+
+#include 
+#include 
+
 static long compat_register_guest_callback(struct compat_callback_register 
*reg)
 {
 long ret = 0;
@@ -343,6 +347,8 @@ long compat_set_callbacks(unsigned long
 return 0;
 }
 
+#endif /* CONFIG_PV32 */
+
 long do_set_trap_table(XEN_GUEST_HANDLE_PARAM(const_trap_info_t) traps)
 {
 struct trap_info cur;
@@ -388,6 +394,7 @@ long do_set_trap_table(XEN_GUEST_HANDLE_
 return rc;
 }
 
+#ifdef CONFIG_PV32
 int compat_set_trap_table(XEN_GUEST_HANDLE(trap_info_compat_t) traps)
 {
 struct vcpu *curr = current;
@@ -429,6 +436,7 @@ int compat_set_trap_table(XEN_GUEST_HAND
 
 return rc;
 }
+#endif
 
 long do_nmi_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
 {
@@ -455,6 +463,7 @@ long do_nmi_op(unsigned int cmd, XEN_GUE
 return rc;
 }
 
+#ifdef CONFIG_PV32
 int compat_nmi_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
 {
 struct compat_nmi_callback cb;
@@ -479,6 +488,7 @@ int compat_nmi_op(unsigned int cmd, XEN_
 
 return rc;
 }
+#endif
 
 /*
  * Local variables:
--- a/xen/arch/x86/pv/descriptor-tables.c
+++ b/xen/arch/x86/pv/descriptor-tables.c
@@ -149,6 +149,8 @@ long do_set_gdt(XEN_GUEST_HANDLE_PARAM(x
 return ret;
 }
 
+#ifdef CONFIG_PV32
+
 int compat_set_gdt(XEN_GUEST_HANDLE_PARAM(uint) frame_list,
unsigned int entries)
 {
@@ -185,6 +187,18 @@ int compat_set_gdt(XEN_GUEST_HANDLE_PARA
 return ret;
 }
 
+int compat_update_descriptor(uint32_t pa_lo, uint32_t pa_hi,
+ uint32_t desc_lo, uint32_t desc_hi)
+{
+seg_desc_t d;
+
+d.raw = ((uint64_t)desc_hi << 32) | desc_lo;
+
+return do_update_descriptor(pa_lo | ((uint64_t)pa_hi << 32), d);
+}
+
+#endif /* CONFIG_PV32 */
+
 static bool check_descriptor(const struct domain *dom, seg_desc_t *d)
 {
 unsigned int a = d->a, b = d->b, cs, dpl;
@@ -334,16 +348,6 @@ long do_update_descriptor(uint64_t gaddr
 return ret;
 }
 
-int compat_update_descriptor(uint32_t pa_lo, uint32_t pa_hi,
- uint32_t desc_lo, uint32_t desc_hi)
-{
-seg_desc_t d;
-
-d.raw = ((uint64_t)desc_hi << 32) | desc_lo;
-
-return do_update_descriptor(pa_lo | ((uint64_t)pa_hi << 32), d);
-}
-
 /*
  * Local variables:
  * mode: C
--- a/xen/arch/x86/pv/domain.c
+++ b/xen/arch/x86/pv/domain.c
@@ -212,6 +212,7 @@ unsigned long pv_make_cr4(const struct v
 return cr4;
 }
 
+#ifdef CONFIG_PV32
 int switch_compat(struct domain *d)
 {
 struct vcpu *v;
@@ -256,6 +257,7 @@ int switch_compat(struct domain *d)
 
 return rc;
 }
+#endif
 
 static int pv_create_gdt_ldt_l1tab(struct vcpu *v)
 {
--- a/xen/arch/x86/pv/hypercall.c
+++ b/xen/arch/x86/pv/hypercall.c
@@ -25,12 +25,18 @@
 #include 
 #include 
 
+#ifdef CONFIG_PV32
 #define HYPERCALL(x)\
 [ __HYPERVISOR_ ## x ] = { (hypercall_fn_t *) do_ ## x, \
(hypercall_fn_t *) do_ ## x }
 #define COMPAT_CALL(x)  \
 [ __HYPERVISOR_ ## x ] = { 

[PATCH v2 1/3] x86: don't build unused entry code when !PV32

[Jan Beulich]

Except for the initial part of cstar_enter compat/entry.S is all dead
code in this case. Further, along the lines of the PV conditionals we
already have in entry.S, make code PV32-conditional there too (to a
fair part because this code actually references compat/entry.S).

This has the side effect of moving the tail part (now at compat_syscall)
of the code out of .text.entry (in line with e.g. compat_sysenter).

Signed-off-by: Jan Beulich 
---
v2: Avoid #ifdef-ary in compat/entry.S.
---
TBD: I'm on the fence of whether (in a separate patch) to also make
 conditional struct pv_domain's is_32bit field.

--- a/xen/arch/x86/x86_64/asm-offsets.c
+++ b/xen/arch/x86/x86_64/asm-offsets.c
@@ -9,7 +9,7 @@
 #include 
 #endif
 #include 
-#ifdef CONFIG_PV
+#ifdef CONFIG_PV32
 #include 
 #endif
 #include 
@@ -102,19 +102,21 @@ void __dummy__(void)
 BLANK();
 #endif
 
-#ifdef CONFIG_PV
+#ifdef CONFIG_PV32
 OFFSET(DOMAIN_is_32bit_pv, struct domain, arch.pv.is_32bit);
 BLANK();
 
-OFFSET(VCPUINFO_upcall_pending, struct vcpu_info, evtchn_upcall_pending);
-OFFSET(VCPUINFO_upcall_mask, struct vcpu_info, evtchn_upcall_mask);
-BLANK();
-
 OFFSET(COMPAT_VCPUINFO_upcall_pending, struct compat_vcpu_info, 
evtchn_upcall_pending);
 OFFSET(COMPAT_VCPUINFO_upcall_mask, struct compat_vcpu_info, 
evtchn_upcall_mask);
 BLANK();
 #endif
 
+#ifdef CONFIG_PV
+OFFSET(VCPUINFO_upcall_pending, struct vcpu_info, evtchn_upcall_pending);
+OFFSET(VCPUINFO_upcall_mask, struct vcpu_info, evtchn_upcall_mask);
+BLANK();
+#endif
+
 OFFSET(CPUINFO_guest_cpu_user_regs, struct cpu_info, guest_cpu_user_regs);
 OFFSET(CPUINFO_verw_sel, struct cpu_info, verw_sel);
 OFFSET(CPUINFO_current_vcpu, struct cpu_info, current_vcpu);
--- a/xen/arch/x86/x86_64/compat/entry.S
+++ b/xen/arch/x86/x86_64/compat/entry.S
@@ -11,8 +11,6 @@
 #include 
 #include 
 
-#ifdef CONFIG_PV32
-
 ENTRY(entry_int82)
 ALTERNATIVE "", clac, X86_FEATURE_XEN_SMAP
 pushq $0
@@ -29,8 +27,6 @@ ENTRY(entry_int82)
 mov   %rsp, %rdi
 call  do_entry_int82
 
-#endif /* CONFIG_PV32 */
-
 /* %rbx: struct vcpu */
 ENTRY(compat_test_all_events)
 ASSERT_NOT_IN_ATOMIC
@@ -197,43 +193,7 @@ ENTRY(cr4_pv32_restore)
 xor   %eax, %eax
 ret
 
-.section .text.entry, "ax", @progbits
-
-/* See lstar_enter for entry register state. */
-ENTRY(cstar_enter)
-#ifdef CONFIG_XEN_SHSTK
-ALTERNATIVE "", "setssbsy", X86_FEATURE_XEN_SHSTK
-#endif
-/* sti could live here when we don't switch page tables below. */
-CR4_PV32_RESTORE
-movq  8(%rsp),%rax /* Restore %rax. */
-movq  $FLAT_USER_SS32, 8(%rsp) /* Assume a 64bit domain.  Compat 
handled lower. */
-pushq %r11
-pushq $FLAT_USER_CS32
-pushq %rcx
-pushq $0
-movl  $TRAP_syscall, 4(%rsp)
-SAVE_ALL
-
-SPEC_CTRL_ENTRY_FROM_PV /* Req: %rsp=regs/cpuinfo, Clob: acd */
-/* WARNING! `ret`, `call *`, `jmp *` not safe before this point. */
-
-GET_STACK_END(bx)
-mov   STACK_CPUINFO_FIELD(xen_cr3)(%rbx), %rcx
-test  %rcx, %rcx
-jz.Lcstar_cr3_okay
-movb  $0, STACK_CPUINFO_FIELD(use_pv_cr3)(%rbx)
-mov   %rcx, %cr3
-/* %r12 is still zero at this point. */
-mov   %r12, STACK_CPUINFO_FIELD(xen_cr3)(%rbx)
-.Lcstar_cr3_okay:
-sti
-
-movq  STACK_CPUINFO_FIELD(current_vcpu)(%rbx), %rbx
-movq  VCPU_domain(%rbx),%rcx
-cmpb  $0,DOMAIN_is_32bit_pv(%rcx)
-jeswitch_to_kernel
-
+ENTRY(compat_syscall)
 /* Fix up reported %cs/%ss for compat domains. */
 movl  $FLAT_COMPAT_USER_SS, UREGS_ss(%rsp)
 movl  $FLAT_COMPAT_USER_CS, UREGS_cs(%rsp)
@@ -262,8 +222,6 @@ UNLIKELY_END(compat_syscall_gpf)
 movb  %cl,TRAPBOUNCE_flags(%rdx)
 jmp   .Lcompat_bounce_exception
 
-.text
-
 ENTRY(compat_sysenter)
 CR4_PV32_RESTORE
 movq  VCPU_trap_ctxt(%rbx),%rcx
--- a/xen/arch/x86/x86_64/Makefile
+++ b/xen/arch/x86/x86_64/Makefile
@@ -1,4 +1,4 @@
-obj-$(CONFIG_PV) += compat/
+obj-$(CONFIG_PV32) += compat/
 
 obj-bin-y += entry.o
 obj-y += traps.o
--- a/xen/arch/x86/x86_64/entry.S
+++ b/xen/arch/x86/x86_64/entry.S
@@ -24,7 +24,7 @@
 
 #ifdef CONFIG_PV
 /* %rbx: struct vcpu */
-ENTRY(switch_to_kernel)
+switch_to_kernel:
 leaq  VCPU_trap_bounce(%rbx),%rdx
 
 /* TB_eip = 32-bit syscall ? syscall32_addr : syscall_addr */
@@ -283,6 +283,45 @@ ENTRY(lstar_enter)
 call  pv_hypercall
 jmp   test_all_events
 
+/* See lstar_enter for entry register state. */
+ENTRY(cstar_enter)
+#ifdef CONFIG_XEN_SHSTK
+ALTERNATIVE "", "setssbsy", X86_FEATURE_XEN_SHSTK
+#endif
+/* sti could live here when we don't switch page tables below. */
+CR4_PV32_RESTORE
+movq  8(%rsp), %rax /* Restore %rax. */
+movq  $FLAT_USER_SS32, 8(%rsp) /* Assume a 64bit domain.  Compat 
handled 

Re: [PATCH 03/14] xen/x86: shadow: The return type of sh_audit_flags() should be const

[Tim Deegan]

At 16:57 +0100 on 05 Apr (1617641822), Julien Grall wrote:
> From: Julien Grall 
> 
> The function sh_audit_flags() is returning pointer to literal strings.
> They should not be modified, so the return is now const and this is
> propagated to the callers.
> 
> Take the opportunity to fix the coding style in the declaration of
> sh_audit_flags.
> 
> Signed-off-by: Julien Grall 

Acked-by: Tim Deegan 

Thanks,

Tim.

[PATCH v2 0/3] x86: asm-offsets.h and !PV32 adjustments

[Jan Beulich]

This is a rework of the previously posted (now) 1st patch plus two
new ones.

1: don't build unused entry code when !PV32
2: x86: slim down hypercall handling when !PV32
3: x86: avoid building COMPAT code when !HVM && !PV32

Jan

Re: [PATCH] rangeset: no need to use snprintf()

[Jan Beulich]

On 06.04.2021 15:44, Julien Grall wrote:
> On 06/04/2021 09:50, Jan Beulich wrote:
>> As of the conversion to safe_strcpy() years ago there has been no need
>> anymore to use snprintf() to prevent storing a not-nul-terminated string.
>>
>> Signed-off-by: Jan Beulich 
> 
> Acked-by: Julien Grall 

Thanks.

>> --- a/xen/common/rangeset.c
>> +++ b/xen/common/rangeset.c
>> @@ -436,14 +436,7 @@ struct rangeset *rangeset_new(
>>   BUG_ON(flags & ~RANGESETF_prettyprint_hex);
>>   r->flags = flags;
>>   
>> -if ( name != NULL )
>> -{
>> -safe_strcpy(r->name, name);
>> -}
>> -else
>> -{
>> -snprintf(r->name, sizeof(r->name), "(no name)");
>> -}
>> +safe_strcpy(r->name, name ?: "(no name)");
> 
> I realize the current code is not checking the return, but I wonder we 
> should rather than silently truncating the string.

The name field is used only for display purposes, so I guess truncation
wouldn't really be a problem here.

Jan

Re: [PATCH] xen/evtchn: Change irq_info lock to raw_spinlock_t

[Julien Grall]

Hi Luca,

On 06/04/2021 11:51, Luca Fancellu wrote:

Unmask operation must be called with interrupt disabled,
on preempt_rt spin_lock_irqsave/spin_unlock_irqrestore
don't disable/enable interrupts, so use raw_* implementation
and change lock variable in struct irq_info from spinlock_t
to raw_spinlock_t

Cc: sta...@vger.kernel.org
Fixes: 25da4618af24 ("xen/events: don't unmask an event channel
when an eoi is pending")

Signed-off-by: Luca Fancellu 


Reviewed-by: Julien Grall 

Cheers,

--
Julien Grall

Re: [PATCH] rangeset: no need to use snprintf()

[Julien Grall]

Hi Jan,

On 06/04/2021 09:50, Jan Beulich wrote:

As of the conversion to safe_strcpy() years ago there has been no need
anymore to use snprintf() to prevent storing a not-nul-terminated string.

Signed-off-by: Jan Beulich 


Acked-by: Julien Grall 



--- a/xen/common/rangeset.c
+++ b/xen/common/rangeset.c
@@ -436,14 +436,7 @@ struct rangeset *rangeset_new(
  BUG_ON(flags & ~RANGESETF_prettyprint_hex);
  r->flags = flags;
  
-if ( name != NULL )

-{
-safe_strcpy(r->name, name);
-}
-else
-{
-snprintf(r->name, sizeof(r->name), "(no name)");
-}
+safe_strcpy(r->name, name ?: "(no name)");


I realize the current code is not checking the return, but I wonder we 
should rather than silently truncating the string.


This is not a new issue, so it can dealt separately if we decide to 
check the return.


Cheers,

--
Julien Grall

[linux-linus test] 160754: regressions - FAIL

[osstest service owner]

flight 160754 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/160754/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-xsm7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 
152332
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. 
vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-examine   6 xen-install  fail REGR. vs. 152332
 test-amd64-i386-libvirt   7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-pair 10 xen-install/src_host fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-installfail REGR. vs. 152332
 test-amd64-i386-pair 11 xen-install/dst_host fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-installfail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 
152332
 test-amd64-i386-freebsd10-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim 7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-installfail REGR. vs. 152332
 test-amd64-i386-xl-raw7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-shadow 7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. 
vs. 152332
 test-amd64-amd64-amd64-pvgrub 20 guest-stop  fail REGR. vs. 152332
 test-amd64-amd64-i386-pvgrub 20 guest-stop   fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stopfail like 152332
 test-armhf-armhf-libvirt 16 saverestore-support-checkfail  like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stopfail like 152332
 test-armhf-armhf-libvirt-raw 15 saverestore-support-checkfail  like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 152332
 test-amd64-amd64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  15 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check  

Re: [PATCH 0/3] Use Doxygen and sphinx for html documentation

[Andrew Cooper]

On 06/04/2021 11:36, Luca Fancellu wrote:
> This serie introduce doxygen in the sphinx html docs generation.
> One benefit is to keep most of the documentation in the source
> files of xen so that it's more maintainable, on the other hand
> there are some limitation of doxygen that must be addressed
> modifying the current codebase (for example doxygen can't parse
> anonymous structure/union).
>
> To reproduce the documentation xen must be compiled because
> most of the headers are generated on compilation time from
> the makefiles.
>
> Here follows the steps to generate the sphinx html docs, some
> package may be required on your machine, everything is suggested
> by the autoconf script.
> Here I'm building the arm64 docs (the only introduced for now by
> this serie):
>
> ./configure
> make -C xen XEN_TARGET_ARCH="arm64" CROSS_COMPILE="aarch64-linux-gnu-" 
> menuconfig
> make -C xen XEN_TARGET_ARCH="arm64" CROSS_COMPILE="aarch64-linux-gnu-"
> make -C docs XEN_TARGET_ARCH="arm64" sphinx-html
>
> now in docs/sphinx/html/ we have the generated docs starting
> from the index.html page.

Do you have a sample rendered output?

The plan was to try and use Linux's kernel-doc plugin for Sphinx, which
is very doxygen-like.  Did you experiment with this option?

~Andrew

[PATCH 2/2] xen/pci: Gate all MSI code in common code with CONFIG_HAS_PCI_MSI

[Rahul Singh]

MSI support is not implemented for ARM architecture but it is enabled
for x86 architecture and referenced in common passthrough/pci.c code.

Therefore introducing the new flag to gate the MSI code for ARM in
common code to avoid compilation error when HAS_PCI is enabled for ARM.

Signed-off-by: Rahul Singh 
---
 xen/drivers/passthrough/pci.c | 17 +
 xen/drivers/pci/Kconfig   |  4 
 xen/drivers/vpci/Makefile |  3 ++-
 xen/drivers/vpci/header.c |  6 ++
 xen/drivers/vpci/vpci.c   |  2 ++
 xen/include/xen/vpci.h|  4 
 xen/xsm/flask/hooks.c |  6 +++---
 7 files changed, 38 insertions(+), 4 deletions(-)

diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index 705137f8be..390b960d83 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -33,7 +33,9 @@
 #include 
 #include 
 #include 
+#ifdef CONFIG_HAS_PCI_MSI
 #include 
+#endif
 #include "ats.h"
 
 struct pci_seg {
@@ -327,6 +329,8 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 
bus, u8 devfn)
 *((u8*) >bus) = bus;
 *((u8*) >devfn) = devfn;
 pdev->domain = NULL;
+
+#ifdef CONFIG_HAS_PCI_MSI
 INIT_LIST_HEAD(>msi_list);
 
 pos = pci_find_cap_offset(pseg->nr, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
@@ -357,6 +361,7 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 
bus, u8 devfn)
 
 pdev->msix = msix;
 }
+#endif
 
 list_add(>alldevs_list, >alldevs_list);
 
@@ -449,7 +454,9 @@ static void free_pdev(struct pci_seg *pseg, struct pci_dev 
*pdev)
 }
 
 list_del(>alldevs_list);
+#ifdef CONFIG_HAS_PCI_MSI
 xfree(pdev->msix);
+#endif
 xfree(pdev);
 }
 
@@ -827,7 +834,9 @@ int pci_remove_device(u16 seg, u8 bus, u8 devfn)
 list_for_each_entry ( pdev, >alldevs_list, alldevs_list )
 if ( pdev->bus == bus && pdev->devfn == devfn )
 {
+#ifdef CONFIG_HAS_PCI_MSI
 pci_cleanup_msi(pdev);
+#endif
 ret = iommu_remove_device(pdev);
 if ( pdev->domain )
 list_del(>domain_list);
@@ -1271,7 +1280,9 @@ bool_t pcie_aer_get_firmware_first(const struct pci_dev 
*pdev)
 static int _dump_pci_devices(struct pci_seg *pseg, void *arg)
 {
 struct pci_dev *pdev;
+#ifdef CONFIG_HAS_PCI_MSI
 struct msi_desc *msi;
+#endif
 
 printk(" segment %04x \n", pseg->nr);
 
@@ -1280,8 +1291,10 @@ static int _dump_pci_devices(struct pci_seg *pseg, void 
*arg)
 printk("%pp - %pd - node %-3d - MSIs < ",
>sbdf, pdev->domain,
(pdev->node != NUMA_NO_NODE) ? pdev->node : -1);
+#ifdef CONFIG_HAS_PCI_MSI
 list_for_each_entry ( msi, >msi_list, list )
printk("%d ", msi->irq);
+#endif
 printk(">\n");
 }
 
@@ -1303,12 +1316,14 @@ static int __init setup_dump_pcidevs(void)
 }
 __initcall(setup_dump_pcidevs);
 
+#ifdef CONFIG_HAS_PCI_MSI
 int iommu_update_ire_from_msi(
 struct msi_desc *msi_desc, struct msi_msg *msg)
 {
 return iommu_intremap
? iommu_call(_ops, update_ire_from_msi, msi_desc, msg) : 0;
 }
+#endif
 
 static int iommu_add_device(struct pci_dev *pdev)
 {
@@ -1429,6 +1444,7 @@ static int assign_device(struct domain *d, u16 seg, u8 
bus, u8 devfn, u32 flag)
 ASSERT(pdev && (pdev->domain == hardware_domain ||
 pdev->domain == dom_io));
 
+#ifdef CONFIG_HAS_PCI_MSI
 if ( pdev->msix )
 {
 rc = pci_reset_msix_state(pdev);
@@ -1436,6 +1452,7 @@ static int assign_device(struct domain *d, u16 seg, u8 
bus, u8 devfn, u32 flag)
 goto done;
 msixtbl_init(d);
 }
+#endif
 
 pdev->fault.count = 0;
 
diff --git a/xen/drivers/pci/Kconfig b/xen/drivers/pci/Kconfig
index 7da03fa13b..695781ce3a 100644
--- a/xen/drivers/pci/Kconfig
+++ b/xen/drivers/pci/Kconfig
@@ -1,3 +1,7 @@
 
 config HAS_PCI
bool
+
+config HAS_PCI_MSI
+   def_bool y
+   depends on X86 && HAS_PCI
diff --git a/xen/drivers/vpci/Makefile b/xen/drivers/vpci/Makefile
index 55d1bdfda0..1a1413b93e 100644
--- a/xen/drivers/vpci/Makefile
+++ b/xen/drivers/vpci/Makefile
@@ -1 +1,2 @@
-obj-y += vpci.o header.o msi.o msix.o
+obj-y += vpci.o header.o
+obj-$(CONFIG_HAS_PCI_MSI) += msi.o msix.o
diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index ba9a036202..b90c345612 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -96,8 +96,10 @@ static void modify_decoding(const struct pci_dev *pdev, 
uint16_t cmd,
  * FIXME: punching holes after the p2m has been set up might be racy for
  * DomU usage, needs to be revisited.
  */
+#ifdef CONFIG_HAS_PCI_MSI
 if ( map && !rom_only && vpci_make_msix_hole(pdev) )
 return;
+#endif
 
 for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
 {
@@ -206,7 +208,9 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t 
cmd, bool rom_only)
 struct vpci_header *header = >vpci->header;
 struct rangeset *mem = 

[PATCH 1/2] xen/pci: Move PCI ATS code to common directory

[Rahul Singh]

PCI ATS code is common for all architecture, move code to common
directory to be usable for other architectures.

No functional change intended.

Signed-off-by: Rahul Singh 
---
 xen/drivers/passthrough/Makefile| 1 +
 xen/drivers/passthrough/{x86 => }/ats.c | 2 +-
 xen/drivers/passthrough/x86/Makefile| 1 -
 3 files changed, 2 insertions(+), 2 deletions(-)
 rename xen/drivers/passthrough/{x86 => }/ats.c (99%)

diff --git a/xen/drivers/passthrough/Makefile b/xen/drivers/passthrough/Makefile
index cc646612c7..445690e3e5 100644
--- a/xen/drivers/passthrough/Makefile
+++ b/xen/drivers/passthrough/Makefile
@@ -6,3 +6,4 @@ obj-$(CONFIG_ARM) += arm/
 obj-y += iommu.o
 obj-$(CONFIG_HAS_PCI) += pci.o
 obj-$(CONFIG_HAS_DEVICE_TREE) += device_tree.o
+obj-$(CONFIG_HAS_PCI) += ats.o
diff --git a/xen/drivers/passthrough/x86/ats.c b/xen/drivers/passthrough/ats.c
similarity index 99%
rename from xen/drivers/passthrough/x86/ats.c
rename to xen/drivers/passthrough/ats.c
index 4628ffde45..7f7b16dc49 100644
--- a/xen/drivers/passthrough/x86/ats.c
+++ b/xen/drivers/passthrough/ats.c
@@ -16,7 +16,7 @@
 #include 
 #include 
 #include 
-#include "../ats.h"
+#include "ats.h"
 
 bool_t __read_mostly ats_enabled = 0;
 boolean_param("ats", ats_enabled);
diff --git a/xen/drivers/passthrough/x86/Makefile 
b/xen/drivers/passthrough/x86/Makefile
index 69284a5d19..75b2885336 100644
--- a/xen/drivers/passthrough/x86/Makefile
+++ b/xen/drivers/passthrough/x86/Makefile
@@ -1,3 +1,2 @@
-obj-y += ats.o
 obj-y += iommu.o
 obj-$(CONFIG_HVM) += hvm.o
-- 
2.17.1

[PATCH 0/2] xen/pci: Make PCI passthrough code non-x86 specific

[Rahul Singh]

This patch series is preparatory work to implement the PCI passthrough support
for the ARM architecture.

Rahul Singh (2):
  xen/pci: Move PCI ATS code to common directory
  xen/pci: Gate all MSI code in common code with CONFIG_HAS_PCI_MSI

 xen/drivers/passthrough/Makefile|  1 +
 xen/drivers/passthrough/{x86 => }/ats.c |  2 +-
 xen/drivers/passthrough/pci.c   | 17 +
 xen/drivers/passthrough/x86/Makefile|  1 -
 xen/drivers/pci/Kconfig |  4 
 xen/drivers/vpci/Makefile   |  3 ++-
 xen/drivers/vpci/header.c   |  6 ++
 xen/drivers/vpci/vpci.c |  2 ++
 xen/include/xen/vpci.h  |  4 
 xen/xsm/flask/hooks.c   |  6 +++---
 10 files changed, 40 insertions(+), 6 deletions(-)
 rename xen/drivers/passthrough/{x86 => }/ats.c (99%)

-- 
2.17.1

Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h

[Jan Beulich]

On 06.04.2021 12:36, Luca Fancellu wrote:
> Modification to include/public/grant_table.h:
> 
> 1) Change anonymous structure to be named structure,
>because doxygen can't deal with them.

Especially in the form presented (adding further name space clutter
for consumers to fall over) I object to this, most notably ...

> @@ -584,7 +599,7 @@ DEFINE_XEN_GUEST_HANDLE(gnttab_swap_grant_ref_t);
>   * page granted to the calling domain by a foreign domain.
>   */
>  struct gnttab_cache_flush {
> -union {
> +union a {

... this one.

Jan

[PATCH v9 10/13] x86/smpboot: switch clone_mapping() to new APIs

[Hongyan Xia]

From: Wei Liu 

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
Reviewed-by: Jan Beulich 

---
Changed in v7:
- change patch title
- remove initialiser of pl3e.
- combine the initialisation of pl3e into a single assignment.
- use the new alloc_map_clear() helper.
- use the normal map_domain_page() in the error path.
---
 xen/arch/x86/smpboot.c | 44 ++
 1 file changed, 27 insertions(+), 17 deletions(-)

diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index e90c4dfa8a88..9c5323977b25 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -694,8 +694,8 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
 unsigned long linear = (unsigned long)ptr, pfn;
 unsigned int flags;
 l3_pgentry_t *pl3e;
-l2_pgentry_t *pl2e;
-l1_pgentry_t *pl1e;
+l2_pgentry_t *pl2e = NULL;
+l1_pgentry_t *pl1e = NULL;
 int rc = 0;
 
 /*
@@ -710,7 +710,7 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
  (linear >= XEN_VIRT_END && linear < DIRECTMAP_VIRT_START) )
 return -EINVAL;
 
-pl3e = l4e_to_l3e(idle_pg_table[root_table_offset(linear)]) +
+pl3e = map_l3t_from_l4e(idle_pg_table[root_table_offset(linear)]) +
 l3_table_offset(linear);
 
 flags = l3e_get_flags(*pl3e);
@@ -723,7 +723,7 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
 }
 else
 {
-pl2e = l3e_to_l2e(*pl3e) + l2_table_offset(linear);
+pl2e = map_l2t_from_l3e(*pl3e) + l2_table_offset(linear);
 flags = l2e_get_flags(*pl2e);
 ASSERT(flags & _PAGE_PRESENT);
 if ( flags & _PAGE_PSE )
@@ -734,7 +734,7 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
 }
 else
 {
-pl1e = l2e_to_l1e(*pl2e) + l1_table_offset(linear);
+pl1e = map_l1t_from_l2e(*pl2e) + l1_table_offset(linear);
 flags = l1e_get_flags(*pl1e);
 if ( !(flags & _PAGE_PRESENT) )
 goto out;
@@ -742,51 +742,58 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
 }
 }
 
+UNMAP_DOMAIN_PAGE(pl1e);
+UNMAP_DOMAIN_PAGE(pl2e);
+UNMAP_DOMAIN_PAGE(pl3e);
+
 if ( !(root_get_flags(rpt[root_table_offset(linear)]) & _PAGE_PRESENT) )
 {
-pl3e = alloc_xen_pagetable();
+mfn_t l3mfn;
+
+pl3e = alloc_map_clear_xen_pt();
 rc = -ENOMEM;
 if ( !pl3e )
 goto out;
-clear_page(pl3e);
 l4e_write([root_table_offset(linear)],
-  l4e_from_paddr(__pa(pl3e), __PAGE_HYPERVISOR));
+  l4e_from_mfn(l3mfn, __PAGE_HYPERVISOR));
 }
 else
-pl3e = l4e_to_l3e(rpt[root_table_offset(linear)]);
+pl3e = map_l3t_from_l4e(rpt[root_table_offset(linear)]);
 
 pl3e += l3_table_offset(linear);
 
 if ( !(l3e_get_flags(*pl3e) & _PAGE_PRESENT) )
 {
-pl2e = alloc_xen_pagetable();
+mfn_t l2mfn;
+
+pl2e = alloc_map_clear_xen_pt();
 rc = -ENOMEM;
 if ( !pl2e )
 goto out;
-clear_page(pl2e);
-l3e_write(pl3e, l3e_from_paddr(__pa(pl2e), __PAGE_HYPERVISOR));
+l3e_write(pl3e, l3e_from_mfn(l2mfn, __PAGE_HYPERVISOR));
 }
 else
 {
 ASSERT(!(l3e_get_flags(*pl3e) & _PAGE_PSE));
-pl2e = l3e_to_l2e(*pl3e);
+pl2e = map_l2t_from_l3e(*pl3e);
 }
 
 pl2e += l2_table_offset(linear);
 
 if ( !(l2e_get_flags(*pl2e) & _PAGE_PRESENT) )
 {
-pl1e = alloc_xen_pagetable();
+mfn_t l1mfn;
+
+pl1e = alloc_map_clear_xen_pt();
 rc = -ENOMEM;
 if ( !pl1e )
 goto out;
-clear_page(pl1e);
-l2e_write(pl2e, l2e_from_paddr(__pa(pl1e), __PAGE_HYPERVISOR));
+l2e_write(pl2e, l2e_from_mfn(l1mfn, __PAGE_HYPERVISOR));
 }
 else
 {
 ASSERT(!(l2e_get_flags(*pl2e) & _PAGE_PSE));
-pl1e = l2e_to_l1e(*pl2e);
+pl1e = map_l1t_from_l2e(*pl2e);
 }
 
 pl1e += l1_table_offset(linear);
@@ -802,6 +809,9 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
 
 rc = 0;
  out:
+unmap_domain_page(pl1e);
+unmap_domain_page(pl2e);
+unmap_domain_page(pl3e);
 return rc;
 }
 
-- 
2.23.3

[PATCH v9 12/13] x86: switch to use domheap page for page tables

[Hongyan Xia]

From: Hongyan Xia 

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
Reviewed-by: Jan Beulich 

---
Changed in v8:
- const qualify pg in alloc_xen_pagetable_new().
---
 xen/arch/x86/mm.c | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index a1ea1835d49b..03362448bd05 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -4885,10 +4885,10 @@ mfn_t alloc_xen_pagetable_new(void)
 {
 if ( system_state != SYS_STATE_early_boot )
 {
-void *ptr = alloc_xenheap_page();
+const struct page_info *pg = alloc_domheap_page(NULL, 0);
 
-BUG_ON(!hardware_domain && !ptr);
-return ptr ? virt_to_mfn(ptr) : INVALID_MFN;
+BUG_ON(!hardware_domain && !pg);
+return pg ? page_to_mfn(pg) : INVALID_MFN;
 }
 
 return alloc_boot_pages(1, 1);
@@ -4898,7 +4898,7 @@ mfn_t alloc_xen_pagetable_new(void)
 void free_xen_pagetable_new(mfn_t mfn)
 {
 if ( system_state != SYS_STATE_early_boot && !mfn_eq(mfn, INVALID_MFN) )
-free_xenheap_page(mfn_to_virt(mfn_x(mfn)));
+free_domheap_page(mfn_to_page(mfn));
 }
 
 void *alloc_map_clear_xen_pt(mfn_t *pmfn)
-- 
2.23.3

[PATCH v9 11/13] x86/mm: drop old page table APIs

[Hongyan Xia]

From: Hongyan Xia 

Two sets of old APIs, alloc/free_xen_pagetable() and lXe_to_lYe(), are
now dropped to avoid the dependency on direct map.

There are two special cases which still have not been re-written into
the new APIs, thus need special treatment:

rpt in smpboot.c cannot use ephemeral mappings yet. The problem is that
rpt is read and written in context switch code, but the mapping
infrastructure is NOT context-switch-safe, meaning we cannot map rpt in
one domain and unmap in another. Before the mapping infrastructure
supports context switches, rpt has to be globally mapped.

Also, lXe_to_lYe() during Xen image relocation cannot be converted into
map/unmap pairs. We cannot hold on to mappings while the mapping
infrastructure is being relocated! It is enough to remove the direct map
in the second e820 pass, so we still use the direct map (<4GiB) in Xen
relocation (which is during the first e820 pass).

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
Reviewed-by: Jan Beulich 
---
 xen/arch/x86/mm.c  | 14 --
 xen/arch/x86/setup.c   |  4 ++--
 xen/arch/x86/smpboot.c |  4 ++--
 xen/include/asm-x86/mm.h   |  2 --
 xen/include/asm-x86/page.h |  5 -
 5 files changed, 4 insertions(+), 25 deletions(-)

diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index ababfffb3afc..a1ea1835d49b 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -4875,20 +4875,6 @@ int mmcfg_intercept_write(
 return X86EMUL_OKAY;
 }
 
-void *alloc_xen_pagetable(void)
-{
-mfn_t mfn = alloc_xen_pagetable_new();
-
-return mfn_eq(mfn, INVALID_MFN) ? NULL : mfn_to_virt(mfn_x(mfn));
-}
-
-void free_xen_pagetable(void *v)
-{
-mfn_t mfn = v ? virt_to_mfn(v) : INVALID_MFN;
-
-free_xen_pagetable_new(mfn);
-}
-
 /*
  * For these PTE APIs, the caller must follow the alloc-map-unmap-free
  * lifecycle, which means explicitly mapping the PTE pages before accessing
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 68454df8ed67..84f015bfa949 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1245,7 +1245,7 @@ void __init noreturn __start_xen(unsigned long mbi_p)
 continue;
 *pl4e = l4e_from_intpte(l4e_get_intpte(*pl4e) +
 xen_phys_start);
-pl3e = l4e_to_l3e(*pl4e);
+pl3e = __va(l4e_get_paddr(*pl4e));
 for ( j = 0; j < L3_PAGETABLE_ENTRIES; j++, pl3e++ )
 {
 /* Not present, 1GB mapping, or already relocated? */
@@ -1255,7 +1255,7 @@ void __init noreturn __start_xen(unsigned long mbi_p)
 continue;
 *pl3e = l3e_from_intpte(l3e_get_intpte(*pl3e) +
 xen_phys_start);
-pl2e = l3e_to_l2e(*pl3e);
+pl2e = __va(l3e_get_paddr(*pl3e));
 for ( k = 0; k < L2_PAGETABLE_ENTRIES; k++, pl2e++ )
 {
 /* Not present, PSE, or already relocated? */
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index 9c5323977b25..b91d1f37e223 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -830,7 +830,7 @@ static int setup_cpu_root_pgt(unsigned int cpu)
 if ( !opt_xpti_hwdom && !opt_xpti_domu )
 return 0;
 
-rpt = alloc_xen_pagetable();
+rpt = alloc_xenheap_page();
 if ( !rpt )
 return -ENOMEM;
 
@@ -933,7 +933,7 @@ static void cleanup_cpu_root_pgt(unsigned int cpu)
 free_xen_pagetable_new(l3mfn);
 }
 
-free_xen_pagetable(rpt);
+free_xenheap_page(rpt);
 
 /* Also zap the stub mapping for this CPU. */
 if ( stub_linear )
diff --git a/xen/include/asm-x86/mm.h b/xen/include/asm-x86/mm.h
index 681aac5b7ac2..b05ede721377 100644
--- a/xen/include/asm-x86/mm.h
+++ b/xen/include/asm-x86/mm.h
@@ -573,8 +573,6 @@ int vcpu_destroy_pagetables(struct vcpu *);
 void *do_page_walk(struct vcpu *v, unsigned long addr);
 
 /* Allocator functions for Xen pagetables. */
-void *alloc_xen_pagetable(void);
-void free_xen_pagetable(void *v);
 mfn_t alloc_xen_pagetable_new(void);
 void free_xen_pagetable_new(mfn_t mfn);
 void *alloc_map_clear_xen_pt(mfn_t *pmfn);
diff --git a/xen/include/asm-x86/page.h b/xen/include/asm-x86/page.h
index 4c7f2cb70c69..1d080cffbe84 100644
--- a/xen/include/asm-x86/page.h
+++ b/xen/include/asm-x86/page.h
@@ -180,11 +180,6 @@ static inline l4_pgentry_t l4e_from_paddr(paddr_t pa, 
unsigned int flags)
 #define l4e_has_changed(x,y,flags) \
 ( !!(((x).l4 ^ (y).l4) & ((PADDR_MASK_MASK)|put_pte_flags(flags))) )
 
-/* Pagetable walking. */
-#define l2e_to_l1e(x)  ((l1_pgentry_t *)__va(l2e_get_paddr(x)))
-#define l3e_to_l2e(x)  ((l2_pgentry_t *)__va(l3e_get_paddr(x)))
-#define l4e_to_l3e(x)  ((l3_pgentry_t *)__va(l4e_get_paddr(x)))
-
 #define map_l1t_from_l2e(x)(l1_pgentry_t 
*)map_domain_page(l2e_get_mfn(x))
 #define 

[PATCH v9 13/13] x86/mm: drop _new suffix for page table APIs

[Hongyan Xia]

From: Wei Liu 

No functional change.

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
Acked-by: Jan Beulich 
---
 xen/arch/x86/mm.c| 44 
 xen/arch/x86/smpboot.c   |  6 +++---
 xen/arch/x86/x86_64/mm.c |  2 +-
 xen/include/asm-x86/mm.h |  4 ++--
 4 files changed, 28 insertions(+), 28 deletions(-)

diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 03362448bd05..b90c2d5f8911 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -369,7 +369,7 @@ void __init arch_init_memory(void)
 ASSERT(root_pgt_pv_xen_slots < ROOT_PAGETABLE_PV_XEN_SLOTS);
 if ( l4_table_offset(split_va) == l4_table_offset(split_va - 1) )
 {
-mfn_t l3mfn = alloc_xen_pagetable_new();
+mfn_t l3mfn = alloc_xen_pagetable();
 
 if ( !mfn_eq(l3mfn, INVALID_MFN) )
 {
@@ -4881,7 +4881,7 @@ int mmcfg_intercept_write(
  * them. The caller must check whether the allocation has succeeded, and only
  * pass valid MFNs to map_domain_page().
  */
-mfn_t alloc_xen_pagetable_new(void)
+mfn_t alloc_xen_pagetable(void)
 {
 if ( system_state != SYS_STATE_early_boot )
 {
@@ -4895,7 +4895,7 @@ mfn_t alloc_xen_pagetable_new(void)
 }
 
 /* mfn can be INVALID_MFN */
-void free_xen_pagetable_new(mfn_t mfn)
+void free_xen_pagetable(mfn_t mfn)
 {
 if ( system_state != SYS_STATE_early_boot && !mfn_eq(mfn, INVALID_MFN) )
 free_domheap_page(mfn_to_page(mfn));
@@ -4903,7 +4903,7 @@ void free_xen_pagetable_new(mfn_t mfn)
 
 void *alloc_map_clear_xen_pt(mfn_t *pmfn)
 {
-mfn_t mfn = alloc_xen_pagetable_new();
+mfn_t mfn = alloc_xen_pagetable();
 void *ret;
 
 if ( mfn_eq(mfn, INVALID_MFN) )
@@ -4949,7 +4949,7 @@ static l3_pgentry_t *virt_to_xen_l3e(unsigned long v)
 }
 if ( locking )
 spin_unlock(_pgdir_lock);
-free_xen_pagetable_new(l3mfn);
+free_xen_pagetable(l3mfn);
 }
 
 return map_l3t_from_l4e(*pl4e) + l3_table_offset(v);
@@ -4984,7 +4984,7 @@ static l2_pgentry_t *virt_to_xen_l2e(unsigned long v)
 }
 if ( locking )
 spin_unlock(_pgdir_lock);
-free_xen_pagetable_new(l2mfn);
+free_xen_pagetable(l2mfn);
 }
 
 BUG_ON(l3e_get_flags(*pl3e) & _PAGE_PSE);
@@ -5023,7 +5023,7 @@ l1_pgentry_t *virt_to_xen_l1e(unsigned long v)
 }
 if ( locking )
 spin_unlock(_pgdir_lock);
-free_xen_pagetable_new(l1mfn);
+free_xen_pagetable(l1mfn);
 }
 
 BUG_ON(l2e_get_flags(*pl2e) & _PAGE_PSE);
@@ -5210,10 +5210,10 @@ int map_pages_to_xen(
 ol2e = l2t[i];
 if ( (l2e_get_flags(ol2e) & _PAGE_PRESENT) &&
  !(l2e_get_flags(ol2e) & _PAGE_PSE) )
-free_xen_pagetable_new(l2e_get_mfn(ol2e));
+free_xen_pagetable(l2e_get_mfn(ol2e));
 }
 unmap_domain_page(l2t);
-free_xen_pagetable_new(l3e_get_mfn(ol3e));
+free_xen_pagetable(l3e_get_mfn(ol3e));
 }
 }
 
@@ -5252,7 +5252,7 @@ int map_pages_to_xen(
 continue;
 }
 
-l2mfn = alloc_xen_pagetable_new();
+l2mfn = alloc_xen_pagetable();
 if ( mfn_eq(l2mfn, INVALID_MFN) )
 goto out;
 
@@ -5280,7 +5280,7 @@ int map_pages_to_xen(
 spin_unlock(_pgdir_lock);
 flush_area(virt, flush_flags);
 
-free_xen_pagetable_new(l2mfn);
+free_xen_pagetable(l2mfn);
 }
 
 pl2e = virt_to_xen_l2e(virt);
@@ -5314,7 +5314,7 @@ int map_pages_to_xen(
 flush_flags(l1e_get_flags(l1t[i]));
 flush_area(virt, flush_flags);
 unmap_domain_page(l1t);
-free_xen_pagetable_new(l2e_get_mfn(ol2e));
+free_xen_pagetable(l2e_get_mfn(ol2e));
 }
 }
 
@@ -5359,7 +5359,7 @@ int map_pages_to_xen(
 goto check_l3;
 }
 
-l1mfn = alloc_xen_pagetable_new();
+l1mfn = alloc_xen_pagetable();
 if ( mfn_eq(l1mfn, INVALID_MFN) )
 goto out;
 
@@ -5386,7 +5386,7 @@ int map_pages_to_xen(
 spin_unlock(_pgdir_lock);
 flush_area(virt, flush_flags);
 
-free_xen_pagetable_new(l1mfn);
+free_xen_pagetable(l1mfn);
 }
 
 pl1e  = map_l1t_from_l2e(*pl2e) + l1_table_offset(virt);
@@ -5452,7 +5452,7 @@ int map_pages_to_xen(
 flush_area(virt - PAGE_SIZE,
FLUSH_TLB_GLOBAL |
FLUSH_ORDER(PAGETABLE_ORDER));
-free_xen_pagetable_new(l2e_get_mfn(ol2e));
+

[PATCH v9 09/13] x86/smpboot: add exit path for clone_mapping()

[Hongyan Xia]

From: Wei Liu 

We will soon need to clean up page table mappings in the exit path.

No functional change.

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 

---
Changed in v7:
- edit commit message.
- begin with rc = 0 and set it to -ENOMEM ahead of if().
---
 xen/arch/x86/smpboot.c | 16 +++-
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index 82c1012e892f..e90c4dfa8a88 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -696,6 +696,7 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
 l3_pgentry_t *pl3e;
 l2_pgentry_t *pl2e;
 l1_pgentry_t *pl1e;
+int rc = 0;
 
 /*
  * Sanity check 'linear'.  We only allow cloning from the Xen virtual
@@ -736,7 +737,7 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
 pl1e = l2e_to_l1e(*pl2e) + l1_table_offset(linear);
 flags = l1e_get_flags(*pl1e);
 if ( !(flags & _PAGE_PRESENT) )
-return 0;
+goto out;
 pfn = l1e_get_pfn(*pl1e);
 }
 }
@@ -744,8 +745,9 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
 if ( !(root_get_flags(rpt[root_table_offset(linear)]) & _PAGE_PRESENT) )
 {
 pl3e = alloc_xen_pagetable();
+rc = -ENOMEM;
 if ( !pl3e )
-return -ENOMEM;
+goto out;
 clear_page(pl3e);
 l4e_write([root_table_offset(linear)],
   l4e_from_paddr(__pa(pl3e), __PAGE_HYPERVISOR));
@@ -758,8 +760,9 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
 if ( !(l3e_get_flags(*pl3e) & _PAGE_PRESENT) )
 {
 pl2e = alloc_xen_pagetable();
+rc = -ENOMEM;
 if ( !pl2e )
-return -ENOMEM;
+goto out;
 clear_page(pl2e);
 l3e_write(pl3e, l3e_from_paddr(__pa(pl2e), __PAGE_HYPERVISOR));
 }
@@ -774,8 +777,9 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
 if ( !(l2e_get_flags(*pl2e) & _PAGE_PRESENT) )
 {
 pl1e = alloc_xen_pagetable();
+rc = -ENOMEM;
 if ( !pl1e )
-return -ENOMEM;
+goto out;
 clear_page(pl1e);
 l2e_write(pl2e, l2e_from_paddr(__pa(pl1e), __PAGE_HYPERVISOR));
 }
@@ -796,7 +800,9 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
 else
 l1e_write(pl1e, l1e_from_pfn(pfn, flags));
 
-return 0;
+rc = 0;
+ out:
+return rc;
 }
 
 DEFINE_PER_CPU(root_pgentry_t *, root_pgt);
-- 
2.23.3

[PATCH v9 07/13] efi: use new page table APIs in copy_mapping

[Hongyan Xia]

From: Wei Liu 

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
Reviewed-by: Jan Beulich 

---
Changed in v8:
- remove redundant commit message.
- unmap l3src based on va instead of mfn.
- re-structure if condition around l3dst.

Changed in v7:
- hoist l3 variables out of the loop to avoid repetitive mappings.
---
 xen/common/efi/boot.c | 28 +---
 1 file changed, 21 insertions(+), 7 deletions(-)

diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
index 63e289ab8506..64b319d0013b 100644
--- a/xen/common/efi/boot.c
+++ b/xen/common/efi/boot.c
@@ -6,6 +6,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -1439,29 +1440,42 @@ static __init void copy_mapping(unsigned long mfn, 
unsigned long end,
  unsigned long emfn))
 {
 unsigned long next;
+l3_pgentry_t *l3src = NULL, *l3dst = NULL;
 
 for ( ; mfn < end; mfn = next )
 {
 l4_pgentry_t l4e = efi_l4_pgtable[l4_table_offset(mfn << PAGE_SHIFT)];
-l3_pgentry_t *l3src, *l3dst;
 unsigned long va = (unsigned long)mfn_to_virt(mfn);
 
+if ( !(mfn & ((1UL << (L4_PAGETABLE_SHIFT - PAGE_SHIFT)) - 1)) )
+UNMAP_DOMAIN_PAGE(l3dst);
+if ( !(va & ((1UL << L4_PAGETABLE_SHIFT) - 1)) )
+UNMAP_DOMAIN_PAGE(l3src);
 next = mfn + (1UL << (L3_PAGETABLE_SHIFT - PAGE_SHIFT));
 if ( !is_valid(mfn, min(next, end)) )
 continue;
-if ( !(l4e_get_flags(l4e) & _PAGE_PRESENT) )
+
+if ( l3dst )
+/* nothing */;
+else if ( !(l4e_get_flags(l4e) & _PAGE_PRESENT) )
 {
-l3dst = alloc_xen_pagetable();
+mfn_t l3mfn;
+
+l3dst = alloc_map_clear_xen_pt();
 BUG_ON(!l3dst);
-clear_page(l3dst);
 efi_l4_pgtable[l4_table_offset(mfn << PAGE_SHIFT)] =
-l4e_from_paddr(virt_to_maddr(l3dst), __PAGE_HYPERVISOR);
+l4e_from_mfn(l3mfn, __PAGE_HYPERVISOR);
 }
 else
-l3dst = l4e_to_l3e(l4e);
-l3src = l4e_to_l3e(idle_pg_table[l4_table_offset(va)]);
+l3dst = map_l3t_from_l4e(l4e);
+
+if ( !l3src )
+l3src = map_l3t_from_l4e(idle_pg_table[l4_table_offset(va)]);
 l3dst[l3_table_offset(mfn << PAGE_SHIFT)] = l3src[l3_table_offset(va)];
 }
+
+unmap_domain_page(l3src);
+unmap_domain_page(l3dst);
 }
 
 static bool __init ram_range_valid(unsigned long smfn, unsigned long emfn)
-- 
2.23.3

[PATCH v9 08/13] efi: switch to new APIs in EFI code

[Hongyan Xia]

From: Wei Liu 

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
Reviewed-by: Jan Beulich 

---
Changed in v7:
- add blank line after declaration.
- rename efi_l4_pgtable into efi_l4t.
- pass the mapped efi_l4t to copy_mapping() instead of map it again.
- use the alloc_map_clear_xen_pt() API.
- unmap pl3e, pl2e, l1t earlier.
---
 xen/arch/x86/efi/runtime.h | 13 ++---
 xen/common/efi/boot.c  | 55 ++
 xen/common/efi/efi.h   |  3 ++-
 xen/common/efi/runtime.c   |  8 +++---
 4 files changed, 48 insertions(+), 31 deletions(-)

diff --git a/xen/arch/x86/efi/runtime.h b/xen/arch/x86/efi/runtime.h
index d9eb8f5c270f..77866c5f2178 100644
--- a/xen/arch/x86/efi/runtime.h
+++ b/xen/arch/x86/efi/runtime.h
@@ -1,12 +1,19 @@
+#include 
+#include 
 #include 
 #include 
 
 #ifndef COMPAT
-l4_pgentry_t *__read_mostly efi_l4_pgtable;
+mfn_t __read_mostly efi_l4_mfn = INVALID_MFN_INITIALIZER;
 
 void efi_update_l4_pgtable(unsigned int l4idx, l4_pgentry_t l4e)
 {
-if ( efi_l4_pgtable )
-l4e_write(efi_l4_pgtable + l4idx, l4e);
+if ( !mfn_eq(efi_l4_mfn, INVALID_MFN) )
+{
+l4_pgentry_t *efi_l4t = map_domain_page(efi_l4_mfn);
+
+l4e_write(efi_l4t + l4idx, l4e);
+unmap_domain_page(efi_l4t);
+}
 }
 #endif
diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
index 64b319d0013b..f21ad5030f41 100644
--- a/xen/common/efi/boot.c
+++ b/xen/common/efi/boot.c
@@ -1437,14 +1437,15 @@ custom_param("efi", parse_efi_param);
 
 static __init void copy_mapping(unsigned long mfn, unsigned long end,
 bool (*is_valid)(unsigned long smfn,
- unsigned long emfn))
+ unsigned long emfn),
+l4_pgentry_t *efi_l4t)
 {
 unsigned long next;
 l3_pgentry_t *l3src = NULL, *l3dst = NULL;
 
 for ( ; mfn < end; mfn = next )
 {
-l4_pgentry_t l4e = efi_l4_pgtable[l4_table_offset(mfn << PAGE_SHIFT)];
+l4_pgentry_t l4e = efi_l4t[l4_table_offset(mfn << PAGE_SHIFT)];
 unsigned long va = (unsigned long)mfn_to_virt(mfn);
 
 if ( !(mfn & ((1UL << (L4_PAGETABLE_SHIFT - PAGE_SHIFT)) - 1)) )
@@ -1463,7 +1464,7 @@ static __init void copy_mapping(unsigned long mfn, 
unsigned long end,
 
 l3dst = alloc_map_clear_xen_pt();
 BUG_ON(!l3dst);
-efi_l4_pgtable[l4_table_offset(mfn << PAGE_SHIFT)] =
+efi_l4t[l4_table_offset(mfn << PAGE_SHIFT)] =
 l4e_from_mfn(l3mfn, __PAGE_HYPERVISOR);
 }
 else
@@ -1496,6 +1497,7 @@ static bool __init rt_range_valid(unsigned long smfn, 
unsigned long emfn)
 void __init efi_init_memory(void)
 {
 unsigned int i;
+l4_pgentry_t *efi_l4t;
 struct rt_extra {
 struct rt_extra *next;
 unsigned long smfn, emfn;
@@ -1610,11 +1612,10 @@ void __init efi_init_memory(void)
  * Set up 1:1 page tables for runtime calls. See SetVirtualAddressMap() in
  * efi_exit_boot().
  */
-efi_l4_pgtable = alloc_xen_pagetable();
-BUG_ON(!efi_l4_pgtable);
-clear_page(efi_l4_pgtable);
+efi_l4t = alloc_map_clear_xen_pt(_l4_mfn);
+BUG_ON(!efi_l4t);
 
-copy_mapping(0, max_page, ram_range_valid);
+copy_mapping(0, max_page, ram_range_valid, efi_l4t);
 
 /* Insert non-RAM runtime mappings inside the direct map. */
 for ( i = 0; i < efi_memmap_size; i += efi_mdesc_size )
@@ -1630,58 +1631,64 @@ void __init efi_init_memory(void)
 copy_mapping(PFN_DOWN(desc->PhysicalStart),
  PFN_UP(desc->PhysicalStart +
 (desc->NumberOfPages << EFI_PAGE_SHIFT)),
- rt_range_valid);
+ rt_range_valid, efi_l4t);
 }
 
 /* Insert non-RAM runtime mappings outside of the direct map. */
 while ( (extra = extra_head) != NULL )
 {
 unsigned long addr = extra->smfn << PAGE_SHIFT;
-l4_pgentry_t l4e = efi_l4_pgtable[l4_table_offset(addr)];
+l4_pgentry_t l4e = efi_l4t[l4_table_offset(addr)];
 l3_pgentry_t *pl3e;
 l2_pgentry_t *pl2e;
 l1_pgentry_t *l1t;
 
 if ( !(l4e_get_flags(l4e) & _PAGE_PRESENT) )
 {
-pl3e = alloc_xen_pagetable();
+mfn_t l3mfn;
+
+pl3e = alloc_map_clear_xen_pt();
 BUG_ON(!pl3e);
-clear_page(pl3e);
-efi_l4_pgtable[l4_table_offset(addr)] =
-l4e_from_paddr(virt_to_maddr(pl3e), __PAGE_HYPERVISOR);
+efi_l4t[l4_table_offset(addr)] =
+l4e_from_mfn(l3mfn, __PAGE_HYPERVISOR);
 }
 else
-pl3e = l4e_to_l3e(l4e);
+pl3e = map_l3t_from_l4e(l4e);
 pl3e += l3_table_offset(addr);
 if ( !(l3e_get_flags(*pl3e) & _PAGE_PRESENT) )
 {
-pl2e = alloc_xen_pagetable();
+mfn_t 

[PATCH v9 06/13] x86_64/mm: switch to new APIs in setup_m2p_table

[Hongyan Xia]

From: Wei Liu 

While doing so, avoid repetitive mapping of l2_ro_mpt by keeping it
across loops, and only unmap and map it when crossing 1G boundaries.

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
Reviewed-by: Jan Beulich 

---
Changed in v8:
- re-structure if condition around l2_ro_mpt.
- reword the commit message.

Changed in v7:
- avoid repetitive mapping of l2_ro_mpt.
- edit commit message.
- switch to alloc_map_clear_xen_pt().
---
 xen/arch/x86/x86_64/mm.c | 32 +++-
 1 file changed, 19 insertions(+), 13 deletions(-)

diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c
index da239f097703..442f345b2a54 100644
--- a/xen/arch/x86/x86_64/mm.c
+++ b/xen/arch/x86/x86_64/mm.c
@@ -402,7 +402,8 @@ static int setup_m2p_table(struct mem_hotadd_info *info)
 
 ASSERT(l4e_get_flags(idle_pg_table[l4_table_offset(RO_MPT_VIRT_START)])
 & _PAGE_PRESENT);
-l3_ro_mpt = l4e_to_l3e(idle_pg_table[l4_table_offset(RO_MPT_VIRT_START)]);
+l3_ro_mpt = map_l3t_from_l4e(
+idle_pg_table[l4_table_offset(RO_MPT_VIRT_START)]);
 
 smap = (info->spfn & (~((1UL << (L2_PAGETABLE_SHIFT - 3)) -1)));
 emap = ((info->epfn + ((1UL << (L2_PAGETABLE_SHIFT - 3)) - 1 )) &
@@ -420,6 +421,10 @@ static int setup_m2p_table(struct mem_hotadd_info *info)
 i = smap;
 while ( i < emap )
 {
+if ( (RO_MPT_VIRT_START + i * sizeof(*machine_to_phys_mapping)) &
+ ((1UL << L3_PAGETABLE_SHIFT) - 1) )
+UNMAP_DOMAIN_PAGE(l2_ro_mpt);
+
 switch ( m2p_mapped(i) )
 {
 case M2P_1G_MAPPED:
@@ -455,32 +460,31 @@ static int setup_m2p_table(struct mem_hotadd_info *info)
 
 ASSERT(!(l3e_get_flags(l3_ro_mpt[l3_table_offset(va)]) &
   _PAGE_PSE));
-if ( l3e_get_flags(l3_ro_mpt[l3_table_offset(va)]) &
-  _PAGE_PRESENT )
-l2_ro_mpt = l3e_to_l2e(l3_ro_mpt[l3_table_offset(va)]) +
-  l2_table_offset(va);
+if ( l2_ro_mpt )
+/* nothing */;
+else if ( l3e_get_flags(l3_ro_mpt[l3_table_offset(va)]) &
+  _PAGE_PRESENT )
+l2_ro_mpt = map_l2t_from_l3e(l3_ro_mpt[l3_table_offset(va)]);
 else
 {
-l2_ro_mpt = alloc_xen_pagetable();
+mfn_t l2_ro_mpt_mfn;
+
+l2_ro_mpt = alloc_map_clear_xen_pt(_ro_mpt_mfn);
 if ( !l2_ro_mpt )
 {
 ret = -ENOMEM;
 goto error;
 }
 
-clear_page(l2_ro_mpt);
 l3e_write(_ro_mpt[l3_table_offset(va)],
-  l3e_from_paddr(__pa(l2_ro_mpt),
- __PAGE_HYPERVISOR_RO | _PAGE_USER));
-l2_ro_mpt += l2_table_offset(va);
+  l3e_from_mfn(l2_ro_mpt_mfn,
+   __PAGE_HYPERVISOR_RO | _PAGE_USER));
 }
 
 /* NB. Cannot be GLOBAL: guest user mode should not see it. */
-l2e_write(l2_ro_mpt, l2e_from_mfn(mfn,
+l2e_write(_ro_mpt[l2_table_offset(va)], l2e_from_mfn(mfn,
/*_PAGE_GLOBAL|*/_PAGE_PSE|_PAGE_USER|_PAGE_PRESENT));
 }
-if ( !((unsigned long)l2_ro_mpt & ~PAGE_MASK) )
-l2_ro_mpt = NULL;
 i += ( 1UL << (L2_PAGETABLE_SHIFT - 3));
 }
 #undef CNT
@@ -488,6 +492,8 @@ static int setup_m2p_table(struct mem_hotadd_info *info)
 
 ret = setup_compat_m2p_table(info);
 error:
+unmap_domain_page(l2_ro_mpt);
+unmap_domain_page(l3_ro_mpt);
 return ret;
 }
 
-- 
2.23.3

[PATCH v9 05/13] x86_64/mm: switch to new APIs in paging_init

[Hongyan Xia]

From: Wei Liu 

Map and unmap pages instead of relying on the direct map.

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
Reviewed-by: Jan Beulich 

---
Changed in v9:
- remove an unnecessary l3mfn variable.

Changed in v8:
- replace l3/2_ro_mpt_mfn with just mfn since their lifetimes do not
  overlap

Changed in v7:
- use the new alloc_map_clear_xen_pt() helper.
- move the unmap of pl3t up a bit.
- remove the unmaps in the nomem path.
---
 xen/arch/x86/x86_64/mm.c | 34 --
 1 file changed, 20 insertions(+), 14 deletions(-)

diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c
index c5a47df01bde..da239f097703 100644
--- a/xen/arch/x86/x86_64/mm.c
+++ b/xen/arch/x86/x86_64/mm.c
@@ -498,6 +498,7 @@ void __init paging_init(void)
 l3_pgentry_t *l3_ro_mpt;
 l2_pgentry_t *pl2e = NULL, *l2_ro_mpt = NULL;
 struct page_info *l1_pg;
+mfn_t mfn;
 
 /*
  * We setup the L3s for 1:1 mapping if host support memory hotplug
@@ -510,22 +511,22 @@ void __init paging_init(void)
 if ( !(l4e_get_flags(idle_pg_table[l4_table_offset(va)]) &
   _PAGE_PRESENT) )
 {
-l3_pgentry_t *pl3t = alloc_xen_pagetable();
+l3_pgentry_t *pl3t = alloc_map_clear_xen_pt();
 
 if ( !pl3t )
 goto nomem;
-clear_page(pl3t);
+UNMAP_DOMAIN_PAGE(pl3t);
 l4e_write(_pg_table[l4_table_offset(va)],
-  l4e_from_paddr(__pa(pl3t), __PAGE_HYPERVISOR_RW));
+  l4e_from_mfn(mfn, __PAGE_HYPERVISOR_RW));
 }
 }
 
 /* Create user-accessible L2 directory to map the MPT for guests. */
-if ( (l3_ro_mpt = alloc_xen_pagetable()) == NULL )
+l3_ro_mpt = alloc_map_clear_xen_pt();
+if ( !l3_ro_mpt )
 goto nomem;
-clear_page(l3_ro_mpt);
 l4e_write(_pg_table[l4_table_offset(RO_MPT_VIRT_START)],
-  l4e_from_paddr(__pa(l3_ro_mpt), __PAGE_HYPERVISOR_RO | 
_PAGE_USER));
+  l4e_from_mfn(mfn, __PAGE_HYPERVISOR_RO | _PAGE_USER));
 
 /*
  * Allocate and map the machine-to-phys table.
@@ -608,12 +609,14 @@ void __init paging_init(void)
 }
 if ( !((unsigned long)pl2e & ~PAGE_MASK) )
 {
-if ( (l2_ro_mpt = alloc_xen_pagetable()) == NULL )
+UNMAP_DOMAIN_PAGE(l2_ro_mpt);
+
+l2_ro_mpt = alloc_map_clear_xen_pt();
+if ( !l2_ro_mpt )
 goto nomem;
-clear_page(l2_ro_mpt);
+
 l3e_write(_ro_mpt[l3_table_offset(va)],
-  l3e_from_paddr(__pa(l2_ro_mpt),
- __PAGE_HYPERVISOR_RO | _PAGE_USER));
+  l3e_from_mfn(mfn, __PAGE_HYPERVISOR_RO | _PAGE_USER));
 pl2e = l2_ro_mpt;
 ASSERT(!l2_table_offset(va));
 }
@@ -625,15 +628,18 @@ void __init paging_init(void)
 }
 #undef CNT
 #undef MFN
+UNMAP_DOMAIN_PAGE(l2_ro_mpt);
+UNMAP_DOMAIN_PAGE(l3_ro_mpt);
 
 /* Create user-accessible L2 directory to map the MPT for compat guests. */
 if ( opt_pv32 )
 {
-if ( (l2_ro_mpt = alloc_xen_pagetable()) == NULL )
+mfn = alloc_xen_pagetable_new();
+if ( mfn_eq(mfn, INVALID_MFN) )
 goto nomem;
-compat_idle_pg_table_l2 = l2_ro_mpt;
-clear_page(l2_ro_mpt);
-pl2e = l2_ro_mpt;
+compat_idle_pg_table_l2 = map_domain_page_global(mfn);
+clear_page(compat_idle_pg_table_l2);
+pl2e = compat_idle_pg_table_l2;
 
 /* Allocate and map the compatibility mode machine-to-phys table. */
 mpt_size = (mpt_size >> 1) + (1UL << (L2_PAGETABLE_SHIFT - 1));
-- 
2.23.3

[PATCH v9 03/13] x86/mm: switch to new APIs in modify_xen_mappings

[Hongyan Xia]

From: Wei Liu 

Page tables allocated in that function should be mapped and unmapped
now.

Note that pl2e now maybe mapped and unmapped in different iterations, so
we need to add clean-ups for that.

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
Reviewed-by: Jan Beulich 

---
Changed in v7:
- use normal unmap in the error path.
---
 xen/arch/x86/mm.c | 57 ++-
 1 file changed, 36 insertions(+), 21 deletions(-)

diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index c49e8554f9f7..ababfffb3afc 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -5530,6 +5530,7 @@ int map_pages_to_xen(
 
  out:
 L3T_UNLOCK(current_l3page);
+unmap_domain_page(pl2e);
 unmap_domain_page(pl3e);
 return rc;
 }
@@ -,7 +5556,7 @@ int modify_xen_mappings(unsigned long s, unsigned long e, 
unsigned int nf)
 {
 bool locking = system_state > SYS_STATE_boot;
 l3_pgentry_t *pl3e = NULL;
-l2_pgentry_t *pl2e;
+l2_pgentry_t *pl2e = NULL;
 l1_pgentry_t *pl1e;
 unsigned int  i;
 unsigned long v = s;
@@ -5575,6 +5576,7 @@ int modify_xen_mappings(unsigned long s, unsigned long e, 
unsigned int nf)
 {
 /* Clean up the previous iteration. */
 L3T_UNLOCK(current_l3page);
+UNMAP_DOMAIN_PAGE(pl2e);
 UNMAP_DOMAIN_PAGE(pl3e);
 
 pl3e = virt_to_xen_l3e(v);
@@ -5597,6 +5599,7 @@ int modify_xen_mappings(unsigned long s, unsigned long e, 
unsigned int nf)
 if ( l3e_get_flags(*pl3e) & _PAGE_PSE )
 {
 l2_pgentry_t *l2t;
+mfn_t l2mfn;
 
 if ( l2_table_offset(v) == 0 &&
  l1_table_offset(v) == 0 &&
@@ -5613,35 +5616,38 @@ int modify_xen_mappings(unsigned long s, unsigned long 
e, unsigned int nf)
 }
 
 /* PAGE1GB: shatter the superpage and fall through. */
-l2t = alloc_xen_pagetable();
-if ( !l2t )
+l2mfn = alloc_xen_pagetable_new();
+if ( mfn_eq(l2mfn, INVALID_MFN) )
 goto out;
 
+l2t = map_domain_page(l2mfn);
 for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++ )
 l2e_write(l2t + i,
   l2e_from_pfn(l3e_get_pfn(*pl3e) +
(i << PAGETABLE_ORDER),
l3e_get_flags(*pl3e)));
+UNMAP_DOMAIN_PAGE(l2t);
+
 if ( locking )
 spin_lock(_pgdir_lock);
 if ( (l3e_get_flags(*pl3e) & _PAGE_PRESENT) &&
  (l3e_get_flags(*pl3e) & _PAGE_PSE) )
 {
-l3e_write_atomic(pl3e, l3e_from_mfn(virt_to_mfn(l2t),
-__PAGE_HYPERVISOR));
-l2t = NULL;
+l3e_write_atomic(pl3e,
+ l3e_from_mfn(l2mfn, __PAGE_HYPERVISOR));
+l2mfn = INVALID_MFN;
 }
 if ( locking )
 spin_unlock(_pgdir_lock);
-if ( l2t )
-free_xen_pagetable(l2t);
+
+free_xen_pagetable_new(l2mfn);
 }
 
 /*
  * The L3 entry has been verified to be present, and we've dealt with
  * 1G pages as well, so the L2 table cannot require allocation.
  */
-pl2e = l3e_to_l2e(*pl3e) + l2_table_offset(v);
+pl2e = map_l2t_from_l3e(*pl3e) + l2_table_offset(v);
 
 if ( !(l2e_get_flags(*pl2e) & _PAGE_PRESENT) )
 {
@@ -5669,41 +5675,45 @@ int modify_xen_mappings(unsigned long s, unsigned long 
e, unsigned int nf)
 else
 {
 l1_pgentry_t *l1t;
-
 /* PSE: shatter the superpage and try again. */
-l1t = alloc_xen_pagetable();
-if ( !l1t )
+mfn_t l1mfn = alloc_xen_pagetable_new();
+
+if ( mfn_eq(l1mfn, INVALID_MFN) )
 goto out;
 
+l1t = map_domain_page(l1mfn);
 for ( i = 0; i < L1_PAGETABLE_ENTRIES; i++ )
 l1e_write([i],
   l1e_from_pfn(l2e_get_pfn(*pl2e) + i,
l2e_get_flags(*pl2e) & ~_PAGE_PSE));
+UNMAP_DOMAIN_PAGE(l1t);
+
 if ( locking )
 spin_lock(_pgdir_lock);
 if ( (l2e_get_flags(*pl2e) & _PAGE_PRESENT) &&
  (l2e_get_flags(*pl2e) & _PAGE_PSE) )
 {
-l2e_write_atomic(pl2e, l2e_from_mfn(virt_to_mfn(l1t),
+l2e_write_atomic(pl2e, l2e_from_mfn(l1mfn,
 __PAGE_HYPERVISOR));
-l1t = NULL;
+l1mfn = INVALID_MFN;
 }
 if ( locking )
 spin_unlock(_pgdir_lock);
-if ( l1t )
-

[PATCH v9 04/13] x86_64/mm: introduce pl2e in paging_init

[Hongyan Xia]

From: Wei Liu 

We will soon map and unmap pages in paging_init(). Introduce pl2e so
that we can use l2_ro_mpt to point to the page table itself.

No functional change.

Signed-off-by: Wei Liu 
Acked-by: Jan Beulich 

---
Changed in v7:
- reword commit message.
---
 xen/arch/x86/x86_64/mm.c | 16 +---
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c
index bce1561e1a80..c5a47df01bde 100644
--- a/xen/arch/x86/x86_64/mm.c
+++ b/xen/arch/x86/x86_64/mm.c
@@ -496,7 +496,7 @@ void __init paging_init(void)
 unsigned long i, mpt_size, va;
 unsigned int n, memflags;
 l3_pgentry_t *l3_ro_mpt;
-l2_pgentry_t *l2_ro_mpt = NULL;
+l2_pgentry_t *pl2e = NULL, *l2_ro_mpt = NULL;
 struct page_info *l1_pg;
 
 /*
@@ -546,7 +546,7 @@ void __init paging_init(void)
 (L2_PAGETABLE_SHIFT - 3 + PAGE_SHIFT)));
 
 if ( cpu_has_page1gb &&
- !((unsigned long)l2_ro_mpt & ~PAGE_MASK) &&
+ !((unsigned long)pl2e & ~PAGE_MASK) &&
  (mpt_size >> L3_PAGETABLE_SHIFT) > (i >> PAGETABLE_ORDER) )
 {
 unsigned int k, holes;
@@ -606,7 +606,7 @@ void __init paging_init(void)
 memset((void *)(RDWR_MPT_VIRT_START + (i << L2_PAGETABLE_SHIFT)),
0xFF, 1UL << L2_PAGETABLE_SHIFT);
 }
-if ( !((unsigned long)l2_ro_mpt & ~PAGE_MASK) )
+if ( !((unsigned long)pl2e & ~PAGE_MASK) )
 {
 if ( (l2_ro_mpt = alloc_xen_pagetable()) == NULL )
 goto nomem;
@@ -614,13 +614,14 @@ void __init paging_init(void)
 l3e_write(_ro_mpt[l3_table_offset(va)],
   l3e_from_paddr(__pa(l2_ro_mpt),
  __PAGE_HYPERVISOR_RO | _PAGE_USER));
+pl2e = l2_ro_mpt;
 ASSERT(!l2_table_offset(va));
 }
 /* NB. Cannot be GLOBAL: guest user mode should not see it. */
 if ( l1_pg )
-l2e_write(l2_ro_mpt, l2e_from_page(
+l2e_write(pl2e, l2e_from_page(
 l1_pg, /*_PAGE_GLOBAL|*/_PAGE_PSE|_PAGE_USER|_PAGE_PRESENT));
-l2_ro_mpt++;
+pl2e++;
 }
 #undef CNT
 #undef MFN
@@ -632,6 +633,7 @@ void __init paging_init(void)
 goto nomem;
 compat_idle_pg_table_l2 = l2_ro_mpt;
 clear_page(l2_ro_mpt);
+pl2e = l2_ro_mpt;
 
 /* Allocate and map the compatibility mode machine-to-phys table. */
 mpt_size = (mpt_size >> 1) + (1UL << (L2_PAGETABLE_SHIFT - 1));
@@ -649,7 +651,7 @@ void __init paging_init(void)
  sizeof(*compat_machine_to_phys_mapping))
 BUILD_BUG_ON((sizeof(*frame_table) & ~sizeof(*frame_table)) % \
  sizeof(*compat_machine_to_phys_mapping));
-for ( i = 0; i < (mpt_size >> L2_PAGETABLE_SHIFT); i++, l2_ro_mpt++ )
+for ( i = 0; i < (mpt_size >> L2_PAGETABLE_SHIFT); i++, pl2e++ )
 {
 memflags = MEMF_node(phys_to_nid(i <<
 (L2_PAGETABLE_SHIFT - 2 + PAGE_SHIFT)));
@@ -671,7 +673,7 @@ void __init paging_init(void)
 (i << L2_PAGETABLE_SHIFT)),
0xFF, 1UL << L2_PAGETABLE_SHIFT);
 /* NB. Cannot be GLOBAL as the ptes get copied into per-VM space. */
-l2e_write(l2_ro_mpt, l2e_from_page(l1_pg, _PAGE_PSE|_PAGE_PRESENT));
+l2e_write(pl2e, l2e_from_page(l1_pg, _PAGE_PSE|_PAGE_PRESENT));
 }
 #undef CNT
 #undef MFN
-- 
2.23.3

[PATCH v9 02/13] x86/mm: switch to new APIs in map_pages_to_xen

[Hongyan Xia]

From: Wei Liu 

Page tables allocated in that function should be mapped and unmapped
now.

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
Reviewed-by: Jan Beulich 
---
 xen/arch/x86/mm.c | 60 ---
 1 file changed, 36 insertions(+), 24 deletions(-)

diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 9705fed195f1..c49e8554f9f7 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -5199,7 +5199,7 @@ int map_pages_to_xen(
 }
 else
 {
-l2_pgentry_t *l2t = l3e_to_l2e(ol3e);
+l2_pgentry_t *l2t = map_l2t_from_l3e(ol3e);
 
 for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++ )
 {
@@ -5211,10 +5211,11 @@ int map_pages_to_xen(
 else
 {
 unsigned int j;
-const l1_pgentry_t *l1t = l2e_to_l1e(ol2e);
+const l1_pgentry_t *l1t = map_l1t_from_l2e(ol2e);
 
 for ( j = 0; j < L1_PAGETABLE_ENTRIES; j++ )
 flush_flags(l1e_get_flags(l1t[j]));
+unmap_domain_page(l1t);
 }
 }
 flush_area(virt, flush_flags);
@@ -5223,9 +5224,10 @@ int map_pages_to_xen(
 ol2e = l2t[i];
 if ( (l2e_get_flags(ol2e) & _PAGE_PRESENT) &&
  !(l2e_get_flags(ol2e) & _PAGE_PSE) )
-free_xen_pagetable(l2e_to_l1e(ol2e));
+free_xen_pagetable_new(l2e_get_mfn(ol2e));
 }
-free_xen_pagetable(l2t);
+unmap_domain_page(l2t);
+free_xen_pagetable_new(l3e_get_mfn(ol3e));
 }
 }
 
@@ -5242,6 +5244,7 @@ int map_pages_to_xen(
 unsigned int flush_flags =
 FLUSH_TLB | FLUSH_ORDER(2 * PAGETABLE_ORDER);
 l2_pgentry_t *l2t;
+mfn_t l2mfn;
 
 /* Skip this PTE if there is no change. */
 if ( ((l3e_get_pfn(ol3e) & ~(L2_PAGETABLE_ENTRIES *
@@ -5263,15 +5266,17 @@ int map_pages_to_xen(
 continue;
 }
 
-l2t = alloc_xen_pagetable();
-if ( l2t == NULL )
+l2mfn = alloc_xen_pagetable_new();
+if ( mfn_eq(l2mfn, INVALID_MFN) )
 goto out;
 
+l2t = map_domain_page(l2mfn);
 for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++ )
 l2e_write(l2t + i,
   l2e_from_pfn(l3e_get_pfn(ol3e) +
(i << PAGETABLE_ORDER),
l3e_get_flags(ol3e)));
+UNMAP_DOMAIN_PAGE(l2t);
 
 if ( l3e_get_flags(ol3e) & _PAGE_GLOBAL )
 flush_flags |= FLUSH_TLB_GLOBAL;
@@ -5281,15 +5286,15 @@ int map_pages_to_xen(
 if ( (l3e_get_flags(*pl3e) & _PAGE_PRESENT) &&
  (l3e_get_flags(*pl3e) & _PAGE_PSE) )
 {
-l3e_write_atomic(pl3e, l3e_from_mfn(virt_to_mfn(l2t),
-__PAGE_HYPERVISOR));
-l2t = NULL;
+l3e_write_atomic(pl3e,
+ l3e_from_mfn(l2mfn, __PAGE_HYPERVISOR));
+l2mfn = INVALID_MFN;
 }
 if ( locking )
 spin_unlock(_pgdir_lock);
 flush_area(virt, flush_flags);
-if ( l2t )
-free_xen_pagetable(l2t);
+
+free_xen_pagetable_new(l2mfn);
 }
 
 pl2e = virt_to_xen_l2e(virt);
@@ -5317,12 +5322,13 @@ int map_pages_to_xen(
 }
 else
 {
-l1_pgentry_t *l1t = l2e_to_l1e(ol2e);
+l1_pgentry_t *l1t = map_l1t_from_l2e(ol2e);
 
 for ( i = 0; i < L1_PAGETABLE_ENTRIES; i++ )
 flush_flags(l1e_get_flags(l1t[i]));
 flush_area(virt, flush_flags);
-free_xen_pagetable(l1t);
+unmap_domain_page(l1t);
+free_xen_pagetable_new(l2e_get_mfn(ol2e));
 }
 }
 
@@ -5347,6 +5353,7 @@ int map_pages_to_xen(
 unsigned int flush_flags =
 FLUSH_TLB | FLUSH_ORDER(PAGETABLE_ORDER);
 l1_pgentry_t *l1t;
+mfn_t l1mfn;
 
 /* Skip this PTE if there is no change. */
 if ( (((l2e_get_pfn(*pl2e) & ~(L1_PAGETABLE_ENTRIES - 1)) +
@@ -5366,14 +5373,16 @@ int map_pages_to_xen(
 goto check_l3;
 }
 
-l1t = alloc_xen_pagetable();
-if ( l1t == NULL )
+l1mfn = 

[PATCH v9 01/13] x86/mm: rewrite virt_to_xen_l*e

[Hongyan Xia]

From: Wei Liu 

Rewrite those functions to use the new APIs. Modify its callers to unmap
the pointer returned. Since alloc_xen_pagetable_new() is almost never
useful unless accompanied by page clearing and a mapping, introduce a
helper alloc_map_clear_xen_pt() for this sequence.

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 

---
Changed in v9:
- use domain_page_map_to_mfn() around the L3 table locking logic.
- remove vmap_to_mfn() changes since we now use xen_map_to_mfn().

Changed in v8:
- s/virtual address/linear address/.
- BUG_ON() on NULL return in vmap_to_mfn().

Changed in v7:
- remove a comment.
- use l1e_get_mfn() instead of converting things back and forth.
- add alloc_map_clear_xen_pt().
- unmap before the next mapping to reduce mapcache pressure.
- use normal unmap calls instead of the macro in error paths because
  unmap can handle NULL now.
---
 xen/arch/x86/mm.c| 102 +++
 xen/common/vmap.c|   1 +
 xen/include/asm-x86/mm.h |   1 +
 3 files changed, 73 insertions(+), 31 deletions(-)

diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index afb4febf6f4e..9705fed195f1 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -4915,8 +4915,28 @@ void free_xen_pagetable_new(mfn_t mfn)
 free_xenheap_page(mfn_to_virt(mfn_x(mfn)));
 }
 
+void *alloc_map_clear_xen_pt(mfn_t *pmfn)
+{
+mfn_t mfn = alloc_xen_pagetable_new();
+void *ret;
+
+if ( mfn_eq(mfn, INVALID_MFN) )
+return NULL;
+
+if ( pmfn )
+*pmfn = mfn;
+ret = map_domain_page(mfn);
+clear_page(ret);
+
+return ret;
+}
+
 static DEFINE_SPINLOCK(map_pgdir_lock);
 
+/*
+ * For virt_to_xen_lXe() functions, they take a linear address and return a
+ * pointer to Xen's LX entry. Caller needs to unmap the pointer.
+ */
 static l3_pgentry_t *virt_to_xen_l3e(unsigned long v)
 {
 l4_pgentry_t *pl4e;
@@ -4925,33 +4945,33 @@ static l3_pgentry_t *virt_to_xen_l3e(unsigned long v)
 if ( !(l4e_get_flags(*pl4e) & _PAGE_PRESENT) )
 {
 bool locking = system_state > SYS_STATE_boot;
-l3_pgentry_t *l3t = alloc_xen_pagetable();
+mfn_t l3mfn;
+l3_pgentry_t *l3t = alloc_map_clear_xen_pt();
 
 if ( !l3t )
 return NULL;
-clear_page(l3t);
+UNMAP_DOMAIN_PAGE(l3t);
 if ( locking )
 spin_lock(_pgdir_lock);
 if ( !(l4e_get_flags(*pl4e) & _PAGE_PRESENT) )
 {
-l4_pgentry_t l4e = l4e_from_paddr(__pa(l3t), __PAGE_HYPERVISOR);
+l4_pgentry_t l4e = l4e_from_mfn(l3mfn, __PAGE_HYPERVISOR);
 
 l4e_write(pl4e, l4e);
 efi_update_l4_pgtable(l4_table_offset(v), l4e);
-l3t = NULL;
+l3mfn = INVALID_MFN;
 }
 if ( locking )
 spin_unlock(_pgdir_lock);
-if ( l3t )
-free_xen_pagetable(l3t);
+free_xen_pagetable_new(l3mfn);
 }
 
-return l4e_to_l3e(*pl4e) + l3_table_offset(v);
+return map_l3t_from_l4e(*pl4e) + l3_table_offset(v);
 }
 
 static l2_pgentry_t *virt_to_xen_l2e(unsigned long v)
 {
-l3_pgentry_t *pl3e;
+l3_pgentry_t *pl3e, l3e;
 
 pl3e = virt_to_xen_l3e(v);
 if ( !pl3e )
@@ -4960,31 +4980,37 @@ static l2_pgentry_t *virt_to_xen_l2e(unsigned long v)
 if ( !(l3e_get_flags(*pl3e) & _PAGE_PRESENT) )
 {
 bool locking = system_state > SYS_STATE_boot;
-l2_pgentry_t *l2t = alloc_xen_pagetable();
+mfn_t l2mfn;
+l2_pgentry_t *l2t = alloc_map_clear_xen_pt();
 
 if ( !l2t )
+{
+unmap_domain_page(pl3e);
 return NULL;
-clear_page(l2t);
+}
+UNMAP_DOMAIN_PAGE(l2t);
 if ( locking )
 spin_lock(_pgdir_lock);
 if ( !(l3e_get_flags(*pl3e) & _PAGE_PRESENT) )
 {
-l3e_write(pl3e, l3e_from_paddr(__pa(l2t), __PAGE_HYPERVISOR));
-l2t = NULL;
+l3e_write(pl3e, l3e_from_mfn(l2mfn, __PAGE_HYPERVISOR));
+l2mfn = INVALID_MFN;
 }
 if ( locking )
 spin_unlock(_pgdir_lock);
-if ( l2t )
-free_xen_pagetable(l2t);
+free_xen_pagetable_new(l2mfn);
 }
 
 BUG_ON(l3e_get_flags(*pl3e) & _PAGE_PSE);
-return l3e_to_l2e(*pl3e) + l2_table_offset(v);
+l3e = *pl3e;
+unmap_domain_page(pl3e);
+
+return map_l2t_from_l3e(l3e) + l2_table_offset(v);
 }
 
 l1_pgentry_t *virt_to_xen_l1e(unsigned long v)
 {
-l2_pgentry_t *pl2e;
+l2_pgentry_t *pl2e, l2e;
 
 pl2e = virt_to_xen_l2e(v);
 if ( !pl2e )
@@ -4993,26 +5019,32 @@ l1_pgentry_t *virt_to_xen_l1e(unsigned long v)
 if ( !(l2e_get_flags(*pl2e) & _PAGE_PRESENT) )
 {
 bool locking = system_state > SYS_STATE_boot;
-l1_pgentry_t *l1t = alloc_xen_pagetable();
+mfn_t l1mfn;
+l1_pgentry_t *l1t = alloc_map_clear_xen_pt();
 
 if ( !l1t )
+{
+unmap_domain_page(pl2e);
 return 

[PATCH v9 00/13] switch to domheap for Xen page tables

[Hongyan Xia]

From: Hongyan Xia 

This series rewrites all the remaining functions and finally makes the
switch from xenheap to domheap for Xen page tables, so that they no
longer need to rely on the direct map, which is a big step towards
removing the direct map.

---
Changed in v9:
- drop first 2 patches which have been merged in XSA-345.
- adjust code around L3 page locking in mm.c.

Hongyan Xia (2):
  x86/mm: drop old page table APIs
  x86: switch to use domheap page for page tables

Wei Liu (11):
  x86/mm: rewrite virt_to_xen_l*e
  x86/mm: switch to new APIs in map_pages_to_xen
  x86/mm: switch to new APIs in modify_xen_mappings
  x86_64/mm: introduce pl2e in paging_init
  x86_64/mm: switch to new APIs in paging_init
  x86_64/mm: switch to new APIs in setup_m2p_table
  efi: use new page table APIs in copy_mapping
  efi: switch to new APIs in EFI code
  x86/smpboot: add exit path for clone_mapping()
  x86/smpboot: switch clone_mapping() to new APIs
  x86/mm: drop _new suffix for page table APIs

 xen/arch/x86/efi/runtime.h |  13 +-
 xen/arch/x86/mm.c  | 247 ++---
 xen/arch/x86/setup.c   |   4 +-
 xen/arch/x86/smpboot.c |  70 +++
 xen/arch/x86/x86_64/mm.c   |  80 +++-
 xen/common/efi/boot.c  |  83 -
 xen/common/efi/efi.h   |   3 +-
 xen/common/efi/runtime.c   |   8 +-
 xen/common/vmap.c  |   1 +
 xen/include/asm-x86/mm.h   |   7 +-
 xen/include/asm-x86/page.h |   5 -
 11 files changed, 315 insertions(+), 206 deletions(-)

-- 
2.23.3

[PATCH] xen/evtchn: Change irq_info lock to raw_spinlock_t

[Luca Fancellu]

Unmask operation must be called with interrupt disabled,
on preempt_rt spin_lock_irqsave/spin_unlock_irqrestore
don't disable/enable interrupts, so use raw_* implementation
and change lock variable in struct irq_info from spinlock_t
to raw_spinlock_t

Cc: sta...@vger.kernel.org
Fixes: 25da4618af24 ("xen/events: don't unmask an event channel
when an eoi is pending")

Signed-off-by: Luca Fancellu 
---
 drivers/xen/events/events_base.c | 12 ++--
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/xen/events/events_base.c b/drivers/xen/events/events_base.c
index 8236e2364eeb..7bbfd58958bc 100644
--- a/drivers/xen/events/events_base.c
+++ b/drivers/xen/events/events_base.c
@@ -110,7 +110,7 @@ struct irq_info {
unsigned short eoi_cpu; /* EOI must happen on this cpu-1 */
unsigned int irq_epoch; /* If eoi_cpu valid: irq_epoch of event */
u64 eoi_time;   /* Time in jiffies when to EOI. */
-   spinlock_t lock;
+   raw_spinlock_t lock;
 
union {
unsigned short virq;
@@ -312,7 +312,7 @@ static int xen_irq_info_common_setup(struct irq_info *info,
info->evtchn = evtchn;
info->cpu = cpu;
info->mask_reason = EVT_MASK_REASON_EXPLICIT;
-   spin_lock_init(>lock);
+   raw_spin_lock_init(>lock);
 
ret = set_evtchn_to_irq(evtchn, irq);
if (ret < 0)
@@ -472,28 +472,28 @@ static void do_mask(struct irq_info *info, u8 reason)
 {
unsigned long flags;
 
-   spin_lock_irqsave(>lock, flags);
+   raw_spin_lock_irqsave(>lock, flags);
 
if (!info->mask_reason)
mask_evtchn(info->evtchn);
 
info->mask_reason |= reason;
 
-   spin_unlock_irqrestore(>lock, flags);
+   raw_spin_unlock_irqrestore(>lock, flags);
 }
 
 static void do_unmask(struct irq_info *info, u8 reason)
 {
unsigned long flags;
 
-   spin_lock_irqsave(>lock, flags);
+   raw_spin_lock_irqsave(>lock, flags);
 
info->mask_reason &= ~reason;
 
if (!info->mask_reason)
unmask_evtchn(info->evtchn);
 
-   spin_unlock_irqrestore(>lock, flags);
+   raw_spin_unlock_irqrestore(>lock, flags);
 }
 
 #ifdef CONFIG_X86
-- 
2.17.1

[PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h

[Luca Fancellu]

Modification to include/public/grant_table.h:

1) Change anonymous structure to be named structure,
   because doxygen can't deal with them.
2) Add doxygen tags to:
 - Create Grant tables section
 - include variables in the generated documentation
3) Add .rst file for grant table for Arm64

Signed-off-by: Luca Fancellu 
---
 docs/hypercall-interfaces/arm64.rst   |  1 +
 .../arm64/grant_tables.rst|  8 ++
 docs/xen-doxygen/doxy_input.list  |  1 +
 xen/include/public/grant_table.h  | 79 ---
 4 files changed, 59 insertions(+), 30 deletions(-)
 create mode 100644 docs/hypercall-interfaces/arm64/grant_tables.rst

diff --git a/docs/hypercall-interfaces/arm64.rst 
b/docs/hypercall-interfaces/arm64.rst
index 5e701a2adc..c30a7142b1 100644
--- a/docs/hypercall-interfaces/arm64.rst
+++ b/docs/hypercall-interfaces/arm64.rst
@@ -8,6 +8,7 @@ Starting points
 .. toctree::
:maxdepth: 2
 
+   arm64/grant_tables
 
 
 Functions
diff --git a/docs/hypercall-interfaces/arm64/grant_tables.rst 
b/docs/hypercall-interfaces/arm64/grant_tables.rst
new file mode 100644
index 00..8955ec5812
--- /dev/null
+++ b/docs/hypercall-interfaces/arm64/grant_tables.rst
@@ -0,0 +1,8 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
+Grant Tables
+
+
+.. doxygengroup:: grant_table
+   :project: Xen
+   :members:
diff --git a/docs/xen-doxygen/doxy_input.list b/docs/xen-doxygen/doxy_input.list
index e69de29bb2..233d692fa7 100644
--- a/docs/xen-doxygen/doxy_input.list
+++ b/docs/xen-doxygen/doxy_input.list
@@ -0,0 +1 @@
+xen/include/public/grant_table.h
diff --git a/xen/include/public/grant_table.h b/xen/include/public/grant_table.h
index 84b1d26b36..b506e09ed3 100644
--- a/xen/include/public/grant_table.h
+++ b/xen/include/public/grant_table.h
@@ -25,15 +25,19 @@
  * Copyright (c) 2004, K A Fraser
  */
 
-#ifndef __XEN_PUBLIC_GRANT_TABLE_H__
+/**
+ * @file
+ * @brief Interface for granting foreign access to page frames, and receiving
+ * page-ownership transfers.
+ */
+
+#if !defined(__XEN_PUBLIC_GRANT_TABLE_H__) || defined(DOXYGEN)
 #define __XEN_PUBLIC_GRANT_TABLE_H__
 
 #include "xen.h"
 
-/*
- * `incontents 150 gnttab Grant Tables
- *
- * Xen's grant tables provide a generic mechanism to memory sharing
+/**
+ * @brief Xen's grant tables provide a generic mechanism to memory sharing
  * between domains. This shared memory interface underpins the split
  * device drivers for block and network IO.
  *
@@ -51,13 +55,10 @@
  * know the real machine address of a page it is sharing. This makes
  * it possible to share memory correctly with domains running in
  * fully virtualised memory.
- */
-
-/***
+ *
  * GRANT TABLE REPRESENTATION
- */
-
-/* Some rough guidelines on accessing and updating grant-table entries
+ *
+ * Some rough guidelines on accessing and updating grant-table entries
  * in a concurrency-safe manner. For more information, Linux contains a
  * reference implementation for guest OSes (drivers/xen/grant_table.c, see
  * 
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=blob;f=drivers/xen/grant-table.c;hb=HEAD
@@ -66,6 +67,7 @@
  * compiler barrier will still be required.
  *
  * Introducing a valid entry into the grant table:
+ * @code
  *  1. Write ent->domid.
  *  2. Write ent->frame:
  *  GTF_permit_access:   Frame to which access is permitted.
@@ -73,20 +75,25 @@
  *   frame, or zero if none.
  *  3. Write memory barrier (WMB).
  *  4. Write ent->flags, inc. valid type.
+ * @endcode
  *
  * Invalidating an unused GTF_permit_access entry:
+ * @code
  *  1. flags = ent->flags.
  *  2. Observe that !(flags & (GTF_reading|GTF_writing)).
  *  3. Check result of SMP-safe CMPXCHG(>flags, flags, 0).
  *  NB. No need for WMB as reuse of entry is control-dependent on success of
  *  step 3, and all architectures guarantee ordering of ctrl-dep writes.
+ * @endcode
  *
  * Invalidating an in-use GTF_permit_access entry:
+ *
  *  This cannot be done directly. Request assistance from the domain controller
  *  which can set a timeout on the use of a grant entry and take necessary
  *  action. (NB. This is not yet implemented!).
  *
  * Invalidating an unused GTF_accept_transfer entry:
+ * @code
  *  1. flags = ent->flags.
  *  2. Observe that !(flags & GTF_transfer_committed). [*]
  *  3. Check result of SMP-safe CMPXCHG(>flags, flags, 0).
@@ -97,47 +104,55 @@
  *  transferred frame is written. It is safe for the guest to spin waiting
  *  for this to occur (detect by observing GTF_transfer_completed in
  *  ent->flags).
+ * @endcode
  *
  * Invalidating a committed GTF_accept_transfer entry:
  *  1. Wait for (ent->flags & GTF_transfer_completed).
  *
  * Changing a GTF_permit_access from writable to read-only:
+ *
  *  Use SMP-safe CMPXCHG to set GTF_readonly, while checking !GTF_writing.
  *
  * Changing a GTF_permit_access from read-only to writable:
+ *
  *  Use 

[PATCH 2/3] docs: hypercalls sphinx skeleton for generated html

[Luca Fancellu]

Create a skeleton for the documentation about hypercalls

Signed-off-by: Luca Fancellu 
---
 .gitignore |  1 +
 docs/Makefile  |  4 
 docs/hypercall-interfaces/arm32.rst|  4 
 docs/hypercall-interfaces/arm64.rst| 32 ++
 docs/hypercall-interfaces/index.rst.in |  7 ++
 docs/hypercall-interfaces/x86_64.rst   |  4 
 docs/index.rst |  8 +++
 7 files changed, 60 insertions(+)
 create mode 100644 docs/hypercall-interfaces/arm32.rst
 create mode 100644 docs/hypercall-interfaces/arm64.rst
 create mode 100644 docs/hypercall-interfaces/index.rst.in
 create mode 100644 docs/hypercall-interfaces/x86_64.rst

diff --git a/.gitignore b/.gitignore
index a5eebf1213..dd3ca96b6f 100644
--- a/.gitignore
+++ b/.gitignore
@@ -64,6 +64,7 @@ docs/xen.doxyfile
 docs/xen.doxyfile.tmp
 docs/doxygen_input.h
 docs/doxygen_input.h.tmp
+docs/hypercall-interfaces/index.rst
 extras/mini-os*
 install/*
 stubdom/*-minios-config.mk
diff --git a/docs/Makefile b/docs/Makefile
index dd651e9a93..2f29ca7e78 100644
--- a/docs/Makefile
+++ b/docs/Makefile
@@ -65,6 +65,9 @@ build: html txt pdf man-pages figs
 sphinx-html: $(DOXY_DEPS) $(DOXY_LIST_SOURCES)
 ifneq ($(SPHINXBUILD),)
$(DOXYGEN) xen.doxyfile
+   @echo "Generating hypercall-interfaces/index.rst"
+   @sed -e "s,@XEN_TARGET_ARCH@,$(XEN_TARGET_ARCH),g" \
+   hypercall-interfaces/index.rst.in > 
hypercall-interfaces/index.rst
XEN_ROOT=$(realpath $(XEN_ROOT)) $(SPHINXBUILD) -b html . sphinx/html
 else
@echo "Sphinx is not installed; skipping sphinx-html documentation."
@@ -113,6 +116,7 @@ clean: clean-man-pages
rm -f xen.doxyfile.tmp
rm -f doxygen_input.h
rm -f doxygen_input.h.tmp
+   rm -f hypercall-interfaces/index.rst
 
 .PHONY: distclean
 distclean: clean
diff --git a/docs/hypercall-interfaces/arm32.rst 
b/docs/hypercall-interfaces/arm32.rst
new file mode 100644
index 00..4e973fbbaf
--- /dev/null
+++ b/docs/hypercall-interfaces/arm32.rst
@@ -0,0 +1,4 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
+Hypercall Interfaces - arm32
+
diff --git a/docs/hypercall-interfaces/arm64.rst 
b/docs/hypercall-interfaces/arm64.rst
new file mode 100644
index 00..5e701a2adc
--- /dev/null
+++ b/docs/hypercall-interfaces/arm64.rst
@@ -0,0 +1,32 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
+Hypercall Interfaces - arm64
+
+
+Starting points
+---
+.. toctree::
+   :maxdepth: 2
+
+
+
+Functions
+-
+
+
+Structs
+---
+
+
+Enums and sets of #defines
+--
+
+
+Typedefs
+
+
+
+Enum values and individual #defines
+---
+
+
diff --git a/docs/hypercall-interfaces/index.rst.in 
b/docs/hypercall-interfaces/index.rst.in
new file mode 100644
index 00..e4dcc5db8d
--- /dev/null
+++ b/docs/hypercall-interfaces/index.rst.in
@@ -0,0 +1,7 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
+Hypercall Interfaces
+
+
+.. toctree::
+   @XEN_TARGET_ARCH@
diff --git a/docs/hypercall-interfaces/x86_64.rst 
b/docs/hypercall-interfaces/x86_64.rst
new file mode 100644
index 00..3ed70dff95
--- /dev/null
+++ b/docs/hypercall-interfaces/x86_64.rst
@@ -0,0 +1,4 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
+Hypercall Interfaces - x86_64
+=
diff --git a/docs/index.rst b/docs/index.rst
index b75487a05d..52226a42d8 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -53,6 +53,14 @@ kind of development environment.
hypervisor-guide/index
 
 
+Hypercall Interfaces documentation
+--
+
+.. toctree::
+   :maxdepth: 2
+
+   hypercall-interfaces/index
+
 Miscellanea
 ---
 
-- 
2.17.1

[PATCH 0/3] Use Doxygen and sphinx for html documentation

[Luca Fancellu]

This serie introduce doxygen in the sphinx html docs generation.
One benefit is to keep most of the documentation in the source
files of xen so that it's more maintainable, on the other hand
there are some limitation of doxygen that must be addressed
modifying the current codebase (for example doxygen can't parse
anonymous structure/union).

To reproduce the documentation xen must be compiled because
most of the headers are generated on compilation time from
the makefiles.

Here follows the steps to generate the sphinx html docs, some
package may be required on your machine, everything is suggested
by the autoconf script.
Here I'm building the arm64 docs (the only introduced for now by
this serie):

./configure
make -C xen XEN_TARGET_ARCH="arm64" CROSS_COMPILE="aarch64-linux-gnu-" 
menuconfig
make -C xen XEN_TARGET_ARCH="arm64" CROSS_COMPILE="aarch64-linux-gnu-"
make -C docs XEN_TARGET_ARCH="arm64" sphinx-html

now in docs/sphinx/html/ we have the generated docs starting
from the index.html page.


Luca Fancellu (3):
  docs: add doxygen support for html documentation
  docs: hypercalls sphinx skeleton for generated html
  docs/doxygen: doxygen documentation for grant_table.h

 .gitignore|7 +
 config/Docs.mk.in |2 +
 docs/Makefile |   51 +-
 docs/conf.py  |   48 +-
 docs/configure|  258 ++
 docs/configure.ac |   15 +
 docs/hypercall-interfaces/arm32.rst   |4 +
 docs/hypercall-interfaces/arm64.rst   |   33 +
 .../arm64/grant_tables.rst|8 +
 docs/hypercall-interfaces/index.rst.in|7 +
 docs/hypercall-interfaces/x86_64.rst  |4 +
 docs/index.rst|8 +
 docs/xen-doxygen/customdoxygen.css|   36 +
 docs/xen-doxygen/doxy_input.list  |1 +
 docs/xen-doxygen/doxy_input_template.in   |   21 +
 docs/xen-doxygen/footer.html  |   21 +
 docs/xen-doxygen/header.html  |   56 +
 docs/xen-doxygen/mainpage.md  |5 +
 docs/xen-doxygen/xen_project_logo_165x67.png  |  Bin 0 -> 18223 bytes
 docs/xen.doxyfile.in  | 2314 +
 m4/ax_python_module.m4|   56 +
 m4/docs_tool.m4   |9 +
 xen/include/public/grant_table.h  |   79 +-
 23 files changed, 3007 insertions(+), 36 deletions(-)
 create mode 100644 docs/hypercall-interfaces/arm32.rst
 create mode 100644 docs/hypercall-interfaces/arm64.rst
 create mode 100644 docs/hypercall-interfaces/arm64/grant_tables.rst
 create mode 100644 docs/hypercall-interfaces/index.rst.in
 create mode 100644 docs/hypercall-interfaces/x86_64.rst
 create mode 100644 docs/xen-doxygen/customdoxygen.css
 create mode 100644 docs/xen-doxygen/doxy_input.list
 create mode 100644 docs/xen-doxygen/doxy_input_template.in
 create mode 100644 docs/xen-doxygen/footer.html
 create mode 100644 docs/xen-doxygen/header.html
 create mode 100644 docs/xen-doxygen/mainpage.md
 create mode 100644 docs/xen-doxygen/xen_project_logo_165x67.png
 create mode 100644 docs/xen.doxyfile.in
 create mode 100644 m4/ax_python_module.m4

-- 
2.17.1

[xen-4.12-testing test] 160752: regressions - FAIL

[osstest service owner]

flight 160752 xen-4.12-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/160752/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf  broken  in 160709
 build-armhf4 host-install(4) broken in 160709 REGR. vs. 159418
 test-amd64-amd64-xl-qcow2 19 guest-localmigrate/x10 fail in 160709 REGR. vs. 
159418

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-libvirt-pair 22 guests-nbd-mirror/debian fail in 160709 pass 
in 160752
 test-amd64-amd64-xl-qcow218 guest-saverestore.2fail pass in 160709
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass 
in 160738

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt  1 build-check(1)   blocked in 160709 n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)   blocked in 160709 n/a
 test-armhf-armhf-xl-rtds  1 build-check(1)   blocked in 160709 n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked in 160709 n/a
 test-armhf-armhf-xl-cubietruck  1 build-check(1) blocked in 160709 n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)  blocked in 160709 n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked in 160709 n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)   blocked in 160709 n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked in 160709 n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked in 160709 n/a
 build-armhf-libvirt   1 build-check(1)   blocked in 160709 n/a
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop fail like 159418
 test-armhf-armhf-libvirt 16 saverestore-support-checkfail  like 159418
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 159418
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 159418
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stopfail like 159418
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop fail like 159418
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stopfail like 159418
 test-armhf-armhf-libvirt-raw 15 saverestore-support-checkfail  like 159418
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 159418
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop fail like 159418
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop fail like 159418
 test-amd64-i386-xl-pvshim14 guest-start  fail   never pass
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 15 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-checkfail   never pass
 

Re: [PATCH 5/5] x86: don't build unused entry code when !PV32

[Jan Beulich]

On 01.04.2021 16:01, Roger Pau Monné wrote:
> On Wed, Nov 25, 2020 at 09:51:33AM +0100, Jan Beulich wrote:
>> Except for the initial part of cstar_enter compat/entry.S is all dead
>> code in this case. Further, along the lines of the PV conditionals we
>> already have in entry.S, make code PV32-conditional there too (to a
>> fair part because this code actually references compat/entry.S).
>>
>> Signed-off-by: Jan Beulich 
>> ---
>> TBD: I'm on the fence of whether (in a separate patch) to also make
>>  conditional struct pv_domain's is_32bit field.
>>
>> --- a/xen/arch/x86/x86_64/asm-offsets.c
>> +++ b/xen/arch/x86/x86_64/asm-offsets.c
>> @@ -9,7 +9,7 @@
>>  #include 
>>  #endif
>>  #include 
>> -#ifdef CONFIG_PV
>> +#ifdef CONFIG_PV32
>>  #include 
>>  #endif
>>  #include 
>> @@ -102,19 +102,21 @@ void __dummy__(void)
>>  BLANK();
>>  #endif
>>  
>> -#ifdef CONFIG_PV
>> +#ifdef CONFIG_PV32
>>  OFFSET(DOMAIN_is_32bit_pv, struct domain, arch.pv.is_32bit);
>>  BLANK();
>>  
>> -OFFSET(VCPUINFO_upcall_pending, struct vcpu_info, 
>> evtchn_upcall_pending);
>> -OFFSET(VCPUINFO_upcall_mask, struct vcpu_info, evtchn_upcall_mask);
>> -BLANK();
>> -
>>  OFFSET(COMPAT_VCPUINFO_upcall_pending, struct compat_vcpu_info, 
>> evtchn_upcall_pending);
>>  OFFSET(COMPAT_VCPUINFO_upcall_mask, struct compat_vcpu_info, 
>> evtchn_upcall_mask);
>>  BLANK();
>>  #endif
>>  
>> +#ifdef CONFIG_PV
>> +OFFSET(VCPUINFO_upcall_pending, struct vcpu_info, 
>> evtchn_upcall_pending);
>> +OFFSET(VCPUINFO_upcall_mask, struct vcpu_info, evtchn_upcall_mask);
>> +BLANK();
>> +#endif
>> +
>>  OFFSET(CPUINFO_guest_cpu_user_regs, struct cpu_info, 
>> guest_cpu_user_regs);
>>  OFFSET(CPUINFO_verw_sel, struct cpu_info, verw_sel);
>>  OFFSET(CPUINFO_current_vcpu, struct cpu_info, current_vcpu);
>> --- a/xen/arch/x86/x86_64/compat/entry.S
>> +++ b/xen/arch/x86/x86_64/compat/entry.S
>> @@ -29,8 +29,6 @@ ENTRY(entry_int82)
>>  mov   %rsp, %rdi
>>  call  do_entry_int82
>>  
>> -#endif /* CONFIG_PV32 */
>> -
>>  /* %rbx: struct vcpu */
>>  ENTRY(compat_test_all_events)
>>  ASSERT_NOT_IN_ATOMIC
>> @@ -197,6 +195,8 @@ ENTRY(cr4_pv32_restore)
>>  xor   %eax, %eax
>>  ret
>>  
>> +#endif /* CONFIG_PV32 */
>> +
>>  .section .text.entry, "ax", @progbits
>>  
>>  /* See lstar_enter for entry register state. */
>> @@ -230,6 +230,13 @@ ENTRY(cstar_enter)
>>  sti
>>  
>>  movq  STACK_CPUINFO_FIELD(current_vcpu)(%rbx), %rbx
>> +
>> +#ifndef CONFIG_PV32
>> +
>> +jmp   switch_to_kernel
>> +
>> +#else
>> +
>>  movq  VCPU_domain(%rbx),%rcx
>>  cmpb  $0,DOMAIN_is_32bit_pv(%rcx)
>>  jeswitch_to_kernel
>> @@ -393,3 +400,5 @@ compat_crash_page_fault:
>>  jmp   .Lft14
>>  .previous
>>  _ASM_EXTABLE(.Lft14, .Lfx14)
>> +
>> +#endif /* CONFIG_PV32 */
> 
> Seeing this chunk, would it make sense to move the cstar_enter part
> relevant to !is_32bit_pv into the common entry.S and leave the rest
> here as compat_cstar_enter or some such?
> 
> AFAICT we only need a tiny part of the compat stuff when !CONFIG_PV32,
> so I think we could make the hole compat/entry.S depend on
> CONFIG_PV32.

While it grows the patch, doing things this way looks to work out
nicely. v2 in the works (making in fact compat/ as a whole build
only when PV32, as it's really only the one object file that gets
built there) ...

Jan

Re: Revert NR_CPUS=1 fix from 4.15 (was: Re: [PATCH] fix for_each_cpu() again for NR_CPUS=1)

[Ian Jackson]

Roger Pau Monné writes ("Re: Revert NR_CPUS=1 fix from 4.15 (was: Re: [PATCH] 
fix for_each_cpu() again for NR_CPUS=1)"):
> On Thu, Apr 01, 2021 at 11:26:03AM +0200, Jan Beulich wrote:
> > Well, I didn't propose reverting (or taking this fix) because I think
> > build breakage is better than runtime breakage. But in the end, Ian,
> > it's up to you.
> 
> Oh, right, sorry. The build issue only happens with NR_CPUS=1, in
> which case I agree, there's no need to do anything in 4.15 IMO.

Oh.  Right.  I had the impression that the build breakage broke other
configurations too.

Since you're saying that's not the case, please disregard my earlier
mail.

Ian.

Revert NR_CPUS=1 fix from 4.15 (was: Re: [PATCH] fix for_each_cpu() again for NR_CPUS=1)

[Ian Jackson]

Roger Pau Monné writes ("Revert NR_CPUS=1 fix from 4.15 (was: Re: [PATCH] fix 
for_each_cpu() again for NR_CPUS=1)"):
> At this point, should be consider reverting the original fix from the
> 4.15 branch, so that we don't release something that's build broken
> with gcc 10?

Yes.  I think so.

Release-Acked-by: Ian Jackson 

But please leave it to me to commit it.  I will do so at or after
around 14:00 UK time (13:00 UTC) today unless someone objects.

Thanks,
Ian.

Re: [PATCH 10/11] block: xen-blkfront: Demote kernel-doc abuses

[Roger Pau Monné]

On Fri, Mar 12, 2021 at 10:55:29AM +, Lee Jones wrote:
> Fixes the following W=1 kernel build warning(s):
> 
>  drivers/block/xen-blkfront.c:1960: warning: Function parameter or member 
> 'dev' not described in 'blkfront_probe'
>  drivers/block/xen-blkfront.c:1960: warning: Function parameter or member 
> 'id' not described in 'blkfront_probe'
>  drivers/block/xen-blkfront.c:1960: warning: expecting prototype for Allocate 
> the basic(). Prototype was for blkfront_probe() instead
>  drivers/block/xen-blkfront.c:2085: warning: Function parameter or member 
> 'dev' not described in 'blkfront_resume'
>  drivers/block/xen-blkfront.c:2085: warning: expecting prototype for or a 
> backend(). Prototype was for blkfront_resume() instead
>  drivers/block/xen-blkfront.c:2444: warning: wrong kernel-doc identifier on 
> line:
> 
> Cc: Konrad Rzeszutek Wilk 
> Cc: "Roger Pau Monné" 
> Cc: Boris Ostrovsky 
> Cc: Juergen Gross 
> Cc: Stefano Stabellini 
> Cc: Jens Axboe 
> Cc: xen-devel@lists.xenproject.org
> Cc: linux-bl...@vger.kernel.org
> Signed-off-by: Lee Jones 

Acked-by: Roger Pau Monné 

Thanks, Roger.

[ovmf test] 160757: all pass - PUSHED

[osstest service owner]

flight 160757 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/160757/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf 4ac02962017c77bf38b462f970c884c2dc7931cf
baseline version:
 ovmf f95cdd316c3d56e8f76b5044be54b9645e1dc60f

Last test of basis   160687  2021-04-03 01:55:05 Z3 days
Testing same since   160757  2021-04-06 01:11:18 Z0 days1 attempts


People who touched revisions under test:
  Jiaxin Wu 
  Zhang Hongbin1 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   f95cdd316c..4ac0296201  4ac02962017c77bf38b462f970c884c2dc7931cf -> 
xen-tested-master

Re: [PATCH 00/11] Rid W=1 warnings from Block

[Lee Jones]

On Tue, 30 Mar 2021, Lee Jones wrote:

> On Fri, 12 Mar 2021, Lee Jones wrote:
> 
> > This set is part of a larger effort attempting to clean-up W=1
> > kernel builds, which are currently overwhelmingly riddled with
> > niggly little warnings.
> > 
> > Lee Jones (11):
> >   block: rsxx: core: Remove superfluous const qualifier
> >   block: drbd: drbd_interval: Demote some kernel-doc abuses and fix
> > another header
> >   block: mtip32xx: mtip32xx: Mark debugging variable 'start' as
> > __maybe_unused
> >   block: drbd: drbd_state: Fix some function documentation issues
> >   block: drbd: drbd_receiver: Demote non-conformant kernel-doc headers
> >   block: drbd: drbd_main: Remove duplicate field initialisation
> >   block: drbd: drbd_nl: Make conversion to 'enum drbd_ret_code' explicit
> >   block: drbd: drbd_main: Fix a bunch of function documentation
> > discrepancies
> >   block: drbd: drbd_receiver: Demote less than half complete kernel-doc
> > header
> >   block: xen-blkfront: Demote kernel-doc abuses
> >   block: drbd: drbd_nl: Demote half-complete kernel-doc headers
> 
> Would you like me to resubmit these?

It's been 3.5 weeks since these were submitted.

We are now at -rc6, so these should be merged soon.

Is there anything you'd like me to do to help?

-- 
Lee Jones [李琼斯]
Senior Technical Lead - Developer Services
Linaro.org │ Open source software for Arm SoCs
Follow Linaro: Facebook | Twitter | Blog

[PATCH] rangeset: no need to use snprintf()

[Jan Beulich]

As of the conversion to safe_strcpy() years ago there has been no need
anymore to use snprintf() to prevent storing a not-nul-terminated string.

Signed-off-by: Jan Beulich 

--- a/xen/common/rangeset.c
+++ b/xen/common/rangeset.c
@@ -436,14 +436,7 @@ struct rangeset *rangeset_new(
 BUG_ON(flags & ~RANGESETF_prettyprint_hex);
 r->flags = flags;
 
-if ( name != NULL )
-{
-safe_strcpy(r->name, name);
-}
-else
-{
-snprintf(r->name, sizeof(r->name), "(no name)");
-}
+safe_strcpy(r->name, name ?: "(no name)");
 
 if ( (r->domain = d) != NULL )
 {

[libvirt test] 160761: regressions - FAIL

[osstest service owner]

flight 160761 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/160761/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf-libvirt   6 libvirt-buildfail REGR. vs. 151777
 build-amd64-libvirt   6 libvirt-buildfail REGR. vs. 151777
 build-arm64-libvirt   6 libvirt-buildfail REGR. vs. 151777
 build-i386-libvirt6 libvirt-buildfail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a

version targeted for testing:
 libvirt  2fc3a704e7083afab0537cf0777521afa9cc1db6
baseline version:
 libvirt  2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  270 days
Failing since151818  2020-07-11 04:18:52 Z  269 days  262 attempts
Testing same since   160661  2021-04-02 04:19:00 Z4 days5 attempts


People who touched revisions under test:
  Adolfo Jayme Barrientos 
  Aleksandr Alekseev 
  Aleksei Zakharov 
  Andika Triwidada 
  Andrea Bolognani 
  Balázs Meskó 
  Barrett Schonefeld 
  Bastien Orivel 
  BiaoXiang Ye 
  Bihong Yu 
  Binfeng Wu 
  Boris Fiuczynski 
  Brian Turek 
  Bruno Haible 
  Chris Mayo 
  Christian Ehrhardt 
  Christian Schoenebeck 
  Cole Robinson 
  Collin Walling 
  Cornelia Huck 
  Cédric Bosdonnat 
  Côme Borsoi 
  Daniel Henrique Barboza 
  Daniel Letai 
  Daniel P. Berrange 
  Daniel P. Berrangé 
  Dmytro Linkin 
  Eiichi Tsukata 
  Erik Skultety 
  Fabian Affolter 
  Fabian Freyer 
  Fangge Jin 
  Farhan Ali 
  Fedora Weblate Translation 
  gongwei 
  Guoyi Tu
  Göran Uddeborg 
  Halil Pasic 
  Han Han 
  Hao Wang 
  Hela Basa 
  Helmut Grohne 
  Ian Wienand 
  Jakob Meng 
  Jamie Strandboge 
  Jamie Strandboge 
  Jan Kuparinen 
  Jean-Baptiste Holcroft 
  Jianan Gao 
  Jim Fehlig 
  Jin Yan 
  Jiri Denemark 
  John Ferlan 
  Jonathan Watt 
  Jonathon Jongsma 
  Julio Faracco 
  Ján Tomko 
  Kashyap Chamarthy 
  Kevin Locke 
  Kristina Hanicova 
  Laine Stump 
  Laszlo Ersek 
  Liao Pingfang 
  Lin Ma 
  Lin Ma 
  Lin Ma 
  Marc Hartmayer 
  Marc-André Lureau 
  Marek Marczykowski-Górecki 
  Markus Schade 
  Martin Kletzander 
  Masayoshi Mizuma 
  Matt Coleman 
  Matt Coleman 
  Mauro Matteo Cascella 
  Meina Li 
  Michal Privoznik 
  Michał Smyk 
  Milo Casagrande 
  Moshe Levi 
  Muha Aliss 
  Neal Gompa 
  Nick Shyrokovskiy 
  Nickys Music Group 
  Nico Pache 
  Nikolay Shirokovskiy 
  Olaf Hering 
  Olesya Gerasimenko 
  Orion Poplawski 
  Pany 
  Patrick Magauran 
  Paulo de Rezende Pinatti 
  Pavel Hrdina 
  Peng Liang 
  Peter Krempa 
  Pino Toscano 
  Pino Toscano 
  Piotr Drąg 
  Prathamesh Chavan 
  Ricky Tigg 
  Roman Bogorodskiy 
  Roman Bolshakov 
  Ryan Gahagan 
  Ryan Schmidt 
  Sam Hartman 
  Scott Shambarger 
  Sebastian Mitterle 
  Shalini Chellathurai Saroja 
  Shaojun Yang 
  Shi Lei 
  simmon 
  Simon Gaiser 
  Stefan Bader 
  Stefan Berger 
  Stefan Berger 
  Szymon Scholz 
  Thomas Huth 
  Tim Wiederhake 
  Tomáš Golembiovský 
  Tomáš Janoušek 
  Tuguoyi 
  Ville Skyttä 
  Wang Xin 
  WangJian 
  Weblate 
  Yalei Li <274268...@qq.com>
  Yalei Li 
  Yang Hang 
  Yanqiu Zhang 
  Yaroslav Kargin 
  Yi Li 
  Yi Wang 
  Yuri Chornoivan 
  Zheng Chuan 
  zhenwei pi 
  Zhenyu Zheng 

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-arm64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt

Re: multiboot2 and module2 boot issues via GRUB2

[Jan Beulich]

On 01.04.2021 21:43, Andrew Cooper wrote:
> On 01/04/2021 09:44, Roger Pau Monné wrote:
>> On Thu, Apr 01, 2021 at 09:31:07AM +0200, Jan Beulich wrote:
>>> On 01.04.2021 03:06, Roman Shaposhnik wrote:
 And the obvious next question: is my EVE usecase esoteric enough that
 I should just go ahead and do a custom GRUB patch or is there a more
 general interest in this?
>>> Not sure if it ought to be a grub patch - the issue could as well
>>> be dealt with in Xen, by concatenating modules to form a monolithic
>>> initrd.
>> I would rather have it done in the loader than Xen, mostly because
>> it's a Linux boot specific format, and hence I don't think Xen should
>> have any knowledge about it.
>>
>> If it turns out to be impossible to implement on the loader side we
>> should consider doing it in Xen, but that's not my first option.
> 
> Concatenating random things which may or may not be initrds is
> absolutely not something Xen should do.  We don't have enough context to
> do it safely/sensibly.

Well, I wasn't suggesting anywhere to concatenate random things.
Instead I was envisioning a command line option giving us the
context we need (e.g. "initrd=3+5").

Jan

Re: [PATCH 04/14] xen/char: console: Use const whenever we point to literal strings

[Jan Beulich]

On 05.04.2021 17:57, Julien Grall wrote:
> --- a/xen/drivers/char/console.c
> +++ b/xen/drivers/char/console.c
> @@ -168,7 +168,7 @@ static int parse_guest_loglvl(const char *s);
>  static char xenlog_val[LOGLVL_VAL_SZ];
>  static char xenlog_guest_val[LOGLVL_VAL_SZ];
>  
> -static char *lvl2opt[] = { "none", "error", "warning", "info", "all" };
> +static const char *lvl2opt[] = { "none", "error", "warning", "info", "all" };

If you add any const here, then I think you should go to full way
and also add the second missing one. Then
Reviewed-by: Jan Beulich 

Arguably the array should also be local to xenlog_update_val().

Jan

Re: [PATCH 02/14] xen/sched: Constify name and opt_name in struct scheduler

[Jan Beulich]

On 05.04.2021 17:57, Julien Grall wrote:
> From: Julien Grall 
> 
> Both name and opt_name are pointing to literal string. So mark both of
> the fields as const.
> 
> Signed-off-by: Julien Grall 

Reviewed-by: Jan Beulich 
albeit ...

> --- a/xen/common/sched/private.h
> +++ b/xen/common/sched/private.h
> @@ -272,8 +272,8 @@ static inline spinlock_t *pcpu_schedule_trylock(unsigned 
> int cpu)
>  }
>  
>  struct scheduler {
> -char *name; /* full name for this scheduler  */
> -char *opt_name; /* option name for this scheduler*/
> +const char *name;   /* full name for this scheduler  */
> +const char *opt_name;   /* option name for this scheduler*/

... I'd like to suggest considering at least the latter to become
an array instead of a pointer - there's little point wasting 8
bytes of storage for the pointer when the strings pointed to are
all at most 9 bytes long (right now; I don't expect much longer
ones to appear).

Jan

Re: [PATCH 01/14] xen: Constify the second parameter of rangeset_new()

[Jan Beulich]

On 05.04.2021 17:57, Julien Grall wrote:
> From: Julien Grall 
> 
> The string 'name' will never get modified by the function, so mark it
> as const.
> 
> Signed-off-by: Julien Grall 

Reviewed-by: Jan Beulich 

> --- a/xen/common/rangeset.c
> +++ b/xen/common/rangeset.c
> @@ -421,7 +421,7 @@ bool_t rangeset_is_empty(
>  }
>  
>  struct rangeset *rangeset_new(
> -struct domain *d, char *name, unsigned int flags)
> +struct domain *d, const char *name, unsigned int flags)
>  {
>  struct rangeset *r;

Remotely along these lines the function also has no need anymore to
use snprintf() - safe_strcpy() very well fits both purposes. But
quite likely for another patch.

Jan

Re: [PATCH 00/14] Use const whether we point to literal strings (take 1)

[Jan Beulich]

On 05.04.2021 17:56, Julien Grall wrote:
> From: Julien Grall 
> 
> Hi all,
> 
> By default, both Clang and GCC will happily compile C code where
> non-const char * point to literal strings. This means the following
> code will be accepted:
> 
> char *str = "test";
> 
> str[0] = 'a';
> 
> Literal strings will reside in rodata, so they are not modifiable.
> This will result to an permission fault at runtime if the permissions
> are enforced in the page-tables (this is the case in Xen).
> 
> I am not aware of code trying to modify literal strings in Xen.
> However, there is a frequent use of non-const char * to point to
> literal strings. Given the size of the codebase, there is a risk
> to involuntarily introduce code that will modify literal strings.
> 
> Therefore it would be better to enforce using const when pointing
> to such strings. Both GCC and Clang provide an option to warn
> for such case (see -Wwrite-strings) and therefore could be used
> by Xen.
> 
> This series doesn't yet make use of -Wwrite-strings because
> the tree is not fully converted. Instead, it contains some easy
> and likely non-controversial use const in the code.
> 
> The major blockers to enable -Wwrite-strings are the following:
> - xen/common/efi: union string is used in both const and
> non-const situation. It doesn't feel right to specific one member
> const and the other non-const.

I'd be happy to see a suggestion of how to avoid this in a not overly
intrusive way.

> - libxl: the major block is the flexarray framework as we would use
> it with string (now const char*). I thought it would be possible to
> make the interface const, but it looks like there are a couple of
> places where we need to modify the content (such as in
> libxl_json.c).
> 
> Ideally, I would like to have -Wwrite-strings unconditionally used
> tree-wide. But, some of the area may required some heavy refactoring.
> 
> One solution would be to enable it tree-wide but turned it off at a
> directroy/file level.

At least as a transient approach I think this would make sense. EFI in
particular has other reasons already to specify a custom option
(-fshort-wchar).

Jan

Re: [PATCH 0/2] xen/arm: Couple of bug fixes when decompressing kernels

[Jan Beulich]

On 02.04.2021 17:21, Julien Grall wrote:
> From: Julien Grall 
> 
> Hi all,
> 
> The main goal of this series is to address the bug report [1]. It is not
> possible

...?

> While testing the series, I also noticed that it is not possible to
> re-use the same compressed kernel for multiple domains as the module
> will be overwritten during the first decompression.
> 
> I am still a bit undecided how to fix it. We can either create a new
> module for the uncompressed kernel and link with the compressed kernel.
> Or we could decompress everytime.
> 
> One inconvenience to kernel to only decompress once is we have to keep
> it until all the domains have booted. This may means a lot of memory to be
> wasted just for keeping the uncompressed kernel (one my setup, it takes
> ~3 times more space).
> 
> Any opinions on how to approach it?

Well, it's not "until all the domains have booted", but until all the
domains have had their kernel image placed in the designated piece of
memory. So while for the time being multiple decompression may indeed
be a reasonable approach, longer term one could populate all the
domains' memory in two steps - first just the kernel space for all of
them, then load the kernel(s), then populate the rest of the memory.

Jan

Re: [PATCH 2/2] xen/gunzip: Allow perform_gunzip() to be called multiple times

[Jan Beulich]

On 02.04.2021 17:21, Julien Grall wrote:
> From: Julien Grall 
> 
> Currently perform_gunzip() can only be called once because the
> the internal state (e.g allocate) is not fully re-initialized.
> 
> This works fine if you are only booting dom0. But this will break when
> booting multiple using the dom0less that uses compressed kernel images.
> 
> This can be resolved by re-initializing bytes_out, malloc_ptr,
> malloc_count every time perform_gunzip() is called.
> 
> Note the latter is only re-initialized for hardening purpose as there is
> no guarantee that every malloc() are followed by free() (It should in
> theory!).
> 
> Take the opportunity to check the return of alloc_heap_pages() to return
> an error rather than dereferencing a NULL pointer later on failure.
> 
> Reported-by: Charles Chiou 
> Signed-off-by: Julien Grall 

Reviewed-by: Jan Beulich 

Re: [PATCH 08/14] tools/firmware: hvmloader: Use const in __bug() and __assert_failed()

[Roger Pau Monné]

On Mon, Apr 05, 2021 at 04:57:07PM +0100, Julien Grall wrote:
> From: Julien Grall 
> 
> __bug() and __assert_failed() are not meant to modify the string
> parameters. So mark them as const.
> 
> Signed-off-by: Julien Grall 

Reviewed-by: Roger Pau Monné 

While looking at this I think we should also make the line parameter
unsigned, but again doesn't need to be part of this patch.

Thanks, Roger.

Re: [PATCH 03/14] xen/x86: shadow: The return type of sh_audit_flags() should be const

[Roger Pau Monné]

On Mon, Apr 05, 2021 at 04:57:02PM +0100, Julien Grall wrote:
> From: Julien Grall 
> 
> The function sh_audit_flags() is returning pointer to literal strings.
> They should not be modified, so the return is now const and this is
> propagated to the callers.
> 
> Take the opportunity to fix the coding style in the declaration of
> sh_audit_flags.
> 
> Signed-off-by: Julien Grall 

While doing the cleanup I think you could narrow the scope of the 's'
variables also, but doesn't need to be part of this patch:

Reviewed-by: Roger Pau Monné 

Thanks, Roger.