[libvirt test] 149803: regressions - FAIL

[osstest service owner]

flight 149803 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/149803/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt   6 libvirt-buildfail REGR. vs. 146182
 build-i386-libvirt6 libvirt-buildfail REGR. vs. 146182
 build-arm64-libvirt   6 libvirt-buildfail REGR. vs. 146182
 build-armhf-libvirt   6 libvirt-buildfail REGR. vs. 146182

Tests which did not succeed, but are not blocking:
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a

version targeted for testing:
 libvirt  6ebb00cd789478ef9017e3eb6b3db4260e57d4bb
baseline version:
 libvirt  a1cd25b919509be2645dbe6f952d5263e0d4e4e5

Last test of basis   146182  2020-01-17 06:00:23 Z   99 days
Failing since146211  2020-01-18 04:18:52 Z   98 days   91 attempts
Testing same since   149803  2020-04-25 04:18:54 Z0 days1 attempts


People who touched revisions under test:
  Andrea Bolognani 
  Arnaud Patard 
  Bjoern Walk 
  Boris Fiuczynski 
  Chen Hanxiao 
  Christian Borntraeger 
  Christian Ehrhardt 
  Christian Schoenebeck 
  Collin Walling 
  Cornelia Huck 
  Daniel Henrique Barboza 
  Daniel P. Berrangé 
  Daniel Veillard 
  Dario Faggioli 
  Erik Skultety 
  Gaurav Agrawal 
  Han Han 
  Jamie Strandboge 
  Jim Fehlig 
  Jiri Denemark 
  Jonathon Jongsma 
  Julio Faracco 
  Ján Tomko 
  Laine Stump 
  Leonid Bloch 
  Lin Ma 
  Marc-André Lureau 
  Marek Marczykowski-Górecki 
  Mark Asselstine 
  Mauro S. M. Rodrigues 
  Michal Privoznik 
  Nikolay Shirokovskiy 
  Pavel Hrdina 
  Pavel Mores 
  Peter Krempa 
  Philipp Hahn 
  Pino Toscano 
  Prathamesh Chavan 
  Rafael Fonseca 
  Richard W.M. Jones 
  Rikard Falkeborn 
  Ryan Moeller 
  Sahid Orentino Ferdjaoui 
  Sebastian Mitterle 
  Seeteena Thoufeek 
  Stefan Berger 
  Stefan Berger 
  Stefan Hajnoczi 
  Thomas Huth 
  Wu Qingliang 
  Yi Li 
  Your Name 
  Zhang Bo 
  zhenwei pi 
  Zhimin Feng 

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-arm64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  fail
 build-arm64-libvirt  fail
 build-armhf-libvirt  fail
 build-i386-libvirt   fail
 build-amd64-pvopspass
 build-arm64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmblocked 
 test-amd64-amd64-libvirt-xsm blocked 
 test-arm64-arm64-libvirt-xsm blocked 
 test-amd64-i386-libvirt-xsm  blocked 
 test-amd64-amd64-libvirt blocked 
 test-arm64-arm64-libvirt blocked 
 test-armhf-armhf-libvirt blocked 
 test-amd64-i386-libvirt  blocked 
 test-amd64-amd64-libvirt-pairblocked 
 test-amd64-i386-libvirt-pair blocked 
 test-arm64-arm64-libvi

[linux-linus test] 149786: regressions - FAIL

[osstest service owner]

flight 149786 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/149786/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 12 
guest-start/debianhvm.repeat fail REGR. vs. 149772
 test-armhf-armhf-xl-vhd  11 guest-start  fail REGR. vs. 149772

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds 15 guest-saverestorefail REGR. vs. 149772

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 149772
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 149772
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 149772
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 149772
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 149772
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 149772
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 149772
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 149772
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass

version targeted for testing:
 linux4544db3f848f1d5d0f48d39c22c9636aecf73cf6
baseline version:
 linuxb4f633221f0aeac102e463a4be46a643b2e3b819

Last test of basis   149772  2020-04-24 03:58:26 Z1 days
Testing same since   149786  2020-04-24 19:09:32 Z0 days1 attempts


People who touched revisions under test:
  Akshu Agrawal 
  Alex Deucher 
  Alex Deucher 
  Alexa

[xen-unstable-smoke test] 149797: regressions - FAIL

[osstest service owner]

flight 149797 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/149797/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64   6 xen-buildfail REGR. vs. 149784

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  f093b08c47b39da6019421a2b61d40745b3e573b
baseline version:
 xen  96b5c267e52657e99bd1bbf81dd51925447115e2

Last test of basis   149784  2020-04-24 14:00:40 Z0 days
Failing since149785  2020-04-24 17:01:40 Z0 days4 attempts
Testing same since   149788  2020-04-24 20:00:41 Z0 days3 attempts


People who touched revisions under test:
  Ian Jackson 
  Stefano Stabellini 
  Stefano Stabellini 
  Wei Liu 

jobs:
 build-arm64-xsm  pass
 build-amd64  fail
 build-armhf  pass
 build-amd64-libvirt  blocked 
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64blocked 
 test-amd64-amd64-libvirt blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.


commit f093b08c47b39da6019421a2b61d40745b3e573b
Author: Stefano Stabellini 
Date:   Tue Apr 21 11:29:46 2020 -0700

Introduce a description of the Backport and Fixes tags

Create a new document under docs/process to describe our special tags.
Add a description of the Fixes tag and the new Backport tag. Also
clarify that lines with tags should not be split.

Signed-off-by: Stefano Stabellini 
Acked-by: Wei Liu 
Acked-by: Ian Jackson 
CC: jbeul...@suse.com
CC: george.dun...@citrix.com
CC: jul...@xen.org
CC: lars.ku...@citrix.com
CC: andrew.coop...@citrix.com
CC: konrad.w...@oracle.com

commit 3acfd35b61688ad9a5b843ee923221eb36e0b613
Author: Ian Jackson 
Date:   Fri Apr 24 15:49:23 2020 +0100

Update QEMU_TRADITIONAL_REVISION

Signed-off-by: Ian Jackson 
(qemu changes not included)

[xen-unstable-smoke test] 149791: regressions - FAIL

[osstest service owner]

flight 149791 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/149791/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64   6 xen-buildfail REGR. vs. 149784

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  f093b08c47b39da6019421a2b61d40745b3e573b
baseline version:
 xen  96b5c267e52657e99bd1bbf81dd51925447115e2

Last test of basis   149784  2020-04-24 14:00:40 Z0 days
Failing since149785  2020-04-24 17:01:40 Z0 days3 attempts
Testing same since   149788  2020-04-24 20:00:41 Z0 days2 attempts


People who touched revisions under test:
  Ian Jackson 
  Stefano Stabellini 
  Stefano Stabellini 
  Wei Liu 

jobs:
 build-arm64-xsm  pass
 build-amd64  fail
 build-armhf  pass
 build-amd64-libvirt  blocked 
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64blocked 
 test-amd64-amd64-libvirt blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.


commit f093b08c47b39da6019421a2b61d40745b3e573b
Author: Stefano Stabellini 
Date:   Tue Apr 21 11:29:46 2020 -0700

Introduce a description of the Backport and Fixes tags

Create a new document under docs/process to describe our special tags.
Add a description of the Fixes tag and the new Backport tag. Also
clarify that lines with tags should not be split.

Signed-off-by: Stefano Stabellini 
Acked-by: Wei Liu 
Acked-by: Ian Jackson 
CC: jbeul...@suse.com
CC: george.dun...@citrix.com
CC: jul...@xen.org
CC: lars.ku...@citrix.com
CC: andrew.coop...@citrix.com
CC: konrad.w...@oracle.com

commit 3acfd35b61688ad9a5b843ee923221eb36e0b613
Author: Ian Jackson 
Date:   Fri Apr 24 15:49:23 2020 +0100

Update QEMU_TRADITIONAL_REVISION

Signed-off-by: Ian Jackson 
(qemu changes not included)

[xen-unstable-smoke test] 149788: regressions - FAIL

[osstest service owner]

flight 149788 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/149788/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64   6 xen-buildfail REGR. vs. 149784

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  f093b08c47b39da6019421a2b61d40745b3e573b
baseline version:
 xen  96b5c267e52657e99bd1bbf81dd51925447115e2

Last test of basis   149784  2020-04-24 14:00:40 Z0 days
Failing since149785  2020-04-24 17:01:40 Z0 days2 attempts
Testing same since   149788  2020-04-24 20:00:41 Z0 days1 attempts


People who touched revisions under test:
  Ian Jackson 
  Stefano Stabellini 
  Stefano Stabellini 
  Wei Liu 

jobs:
 build-arm64-xsm  pass
 build-amd64  fail
 build-armhf  pass
 build-amd64-libvirt  blocked 
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64blocked 
 test-amd64-amd64-libvirt blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.


commit f093b08c47b39da6019421a2b61d40745b3e573b
Author: Stefano Stabellini 
Date:   Tue Apr 21 11:29:46 2020 -0700

Introduce a description of the Backport and Fixes tags

Create a new document under docs/process to describe our special tags.
Add a description of the Fixes tag and the new Backport tag. Also
clarify that lines with tags should not be split.

Signed-off-by: Stefano Stabellini 
Acked-by: Wei Liu 
Acked-by: Ian Jackson 
CC: jbeul...@suse.com
CC: george.dun...@citrix.com
CC: jul...@xen.org
CC: lars.ku...@citrix.com
CC: andrew.coop...@citrix.com
CC: konrad.w...@oracle.com

commit 3acfd35b61688ad9a5b843ee923221eb36e0b613
Author: Ian Jackson 
Date:   Fri Apr 24 15:49:23 2020 +0100

Update QEMU_TRADITIONAL_REVISION

Signed-off-by: Ian Jackson 
(qemu changes not included)

[xen-unstable test] 149783: tolerable FAIL

[osstest service owner]

flight 149783 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/149783/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-examine  4 memdisk-try-append fail pass in 149771

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-rtds 18 guest-localmigrate/x10   fail  like 149771
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 149771
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 149771
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 149771
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 149771
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 149771
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 149771
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 149771
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 149771
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop fail like 149771
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-coresched-i386-xl  2 hosts-allocate   starved in 149771 n/a
 test-amd64-coresched-amd64-xl  2 hosts-allocate  starved in 149771 n/a

version targeted for testing:
 xen  aa14feb6723d3bb15a884533ade1cd9732792145
baseline version:
 xen  aa14feb6723d3bb15a884533ade1cd9732792145

Last test of basis   149783  2020-04-24 13:59:56 Z0 days
Testing same since  (not found) 0 attempts

jobs:
 build-amd64-xs

Re: arm: DomU Networking enable leads to Dom0 kernel oops

[Julien Grall]

On Fri, 24 Apr 2020 at 21:29, Andrei Cherechesu
 wrote:
>
> Hello,

Hi,

Please don't post using HTML and instead configure your client to use
plain text.

> Do you have any idea why this occurs? Have I misconfigured anything?

OOPS should never occur because something has been misconfigured. The
log suggest the address used is bogus, did you trace where the address
was coming from?

Also, looking at the log you are using 4.19.59-rt24+g9d2b51c. AFAICT,
this is not the latest version of RT for 4.19. Can you please use the
latest version to check if the bug can still be reproduced?

Cheers,

Re: Golang Xen packages and the golang packaging system

[Nick Rosbrook]

On Fri, Apr 24, 2020 at 7:26 AM George Dunlap  wrote:
>
>
>
> > On Apr 24, 2020, at 5:04 AM, Nick Rosbrook  wrote:
> >
> > On Thu, Apr 23, 2020 at 1:22 PM George Dunlap  
> > wrote:
> >>
> >>
> >>> On Apr 23, 2020, at 12:49 PM, George Dunlap  
> >>> wrote:
> >>>
> >>>
> >>>
>  On Apr 23, 2020, at 12:27 PM, Ian Jackson  wrote:
> 
>  Ian Jackson writes ("Re: Golang Xen packages and the golang packaging 
>  system"):
> > This is quite unpleasant.  In particular, it makes a git tree out of
> > output files.  What will we do when someone sends us patches to the
> > bindings ?
> 
>  Also, anyone who redistributes your proposed golang package is
>  violating our licence unless they ship a copy of xen.git[1] too, since
>  the golang package is not source code.
> 
>  [1] Technically, a copy of the relevant parts will do.
> >>>
> >>> The “relevant parts” would primarily be gengotypes.py, right?  Oh, and I 
> >>> guess libxl_test.idl and friends.  libxl_test.idl isn’t included in the 
> >>> distribution either.
> >>>
> >>> I’m not an expert in the golang build system, but they generally seem to 
> >>> be trying to keep the functionality simple (which of course, means if you 
> >>> want to do anything non-basic, it’s incredibly complicated or completely 
> >>> impossible).
> >>>
> >>> There’s a command, `go generate`, which we could use to run gengotypes.py 
> >>> to generate the appropriate files.  But I’m not sure how to use that in a 
> >>> practical way for this sort of package: it might end up that people 
> >>> wanting to use the package would need to manually clone it, then manually 
> >>> run `go generate` before manually building the package.
> >>>
> >>> Checking in the generated files means that someone can simply add 
> >>> `golang.xenproject.org/xenlight` as a dependency (perhaps with a specific 
> >>> version tag, like v4.14), and everything Just Works.
> >>>
> >>> Nick may have some ideas on how to use the golang build system more 
> >>> effectively.
> >>
> >> So, the following seems to work quite well actually:
> >>
> >> mkdir vendor
> >> ln -s vendor/golang.xenproject.org 
> >> /usr/share/gocode/src/golang.xenproject.org
> >> echo “golang.xenproject.org/xenlight” >> vendor/modules.txt
> >> go build -mod=vendor
> >>
> >> Using the above method, (say) redctl.git would build exactly the same on 
> >> Xen 4.14 as on Xen 4.15 (assuming redctl wasn’t using anything not 
> >> available in 4.14).
> >>
> >> I’m inclined to say we should start with just telling people to do that, 
> >> and look at doing something else if we discover that’s not suitable for 
> >> some reason.
> >
> > If it's not viable to create another repo for the xenlight package, I
> > think we should should just initialize the go module, i.e. go.mod, at
> > xen.git/tools/golang. The downside is that tags cannot be independent
> > from the rest of xen.git, so users need to have `require  > path>/xenlight@RELEASE-4.14.0` in their go.mod, but at least its `go
> > get`-able. And, this does not fetch the entire git tree.
> >
> > This would also mean that we actually track the generated code (which
> > isn't really a big deal IMO, it's expected that people track their
> > generated gRPC code, for example).
>
> Yes, I was playing with this yesterday and it seems to work OK.
>
> The thing I didn’t necessarily like about this was that suppose you had a 
> public project that used the xenlight bindings, and you upgraded to Xen 4.15, 
> but some of your users hadn’t.  If you updated this to RELEASE-4.15.0, then 
> all your downstreams would stop working, even if you weren’t using any 
> functionality specific to Xen 4.15.

The go.mod is really just a way of specifying dependencies. Definig a
module for xenlight does not take away the ability of downstreams to
do `go build -mod=vendor`, where their preferred version of xenlight
is vendored.

> But I suppose what that would really mean is that:
> 1) We should make sure that xenlight@RELEASE-$V works on > $V as well
> 2) Projects depending on the bindings should use the oldest version of the 
> Xen bindings suitable for their use case.
>
> Both of those are probably reasonable.

I agree.

> Another issue that happens with checking in generated code is that the idl 
> changes and nobody re-generates the code.  We’d probably want an osstest 
> check that would refuse to push from staging -> master if re-running the code 
> generator produced a different output.  (But that has its own annoyances: it 
> seems that different versions of python sort things in different orders, so I 
> often have to throw away spurious changes to the generated files because our 
> two versions of python seem to order some things differently.)

Yeah, I have noticed that on my machine as well. I guess we could just
sort the cases by value or alphabetically before writing the switch
statements. Have you seen the re-ordering happen in other places too?

[...]

> Anyway, do you want

Re: [libvirt test] 149773: regressions - FAIL

[Jim Fehlig]

On 4/24/20 3:53 AM, osstest service owner wrote:

flight 149773 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/149773/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
  build-amd64-libvirt   6 libvirt-buildfail REGR. vs. 146182
  build-i386-libvirt6 libvirt-buildfail REGR. vs. 146182
  build-arm64-libvirt   6 libvirt-buildfail REGR. vs. 146182
  build-armhf-libvirt   6 libvirt-buildfail REGR. vs. 146182


Probably best to disable these tests to avoid all the spam.

Regards,
Jim

arm: DomU Networking enable leads to Dom0 kernel oops

[Andrei Cherechesu]

Hello,

Recently I've been trying to enable the networking in a conventional DomU (not 
dom0less).
The approach I used was the one described here: 
https://wiki.xen.org/wiki/Xen_Networking#Bridging.

But when I use xl to create DomU, I get a Kernel OOPS in Dom0. The setup is 
still responsive after this
and DomU boots successfully. However, if I try to enable `eth0` in DomU (using 
ip link dev set eth0 up),
I get a Kernel Panic in Dom0.

Here are the complete steps:

  1.  Boot Dom0 and configure bridge:
 *   brctl addbr xenbr0
 *   brctl addif xenbr0 eth0
 *   ip link set dev xenbr0 up
 *   ip link set dev eth0 up
 *   dhclient xenbr0



  1.  ifconfig xenbr0 (ping 8.8.8.8 works):
xenbr0Link encap:Ethernet  HWaddr B2:62:37:C9:BB:D7
  inet addr:192.168.0.185  Bcast:192.168.0.255  Mask:255.255.255.0
  inet6 addr: fe80::b062:37ff:fec9:bbd7/64 Scope:Link
  UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
  RX packets:703 errors:0 dropped:0 overruns:0 frame:0
  TX packets:27 errors:0 dropped:0 overruns:0 carrier:0
  collisions:0 txqueuelen:1000
  RX bytes:88213 (86.1 KiB)  TX bytes:3088 (3.0 KiB)


  1.  ifconfig eth0:
eth0  Link encap:Ethernet  HWaddr B2:62:37:C9:BB:D7
  inet6 addr: fe80::b062:37ff:fec9:bbd7/64 Scope:Link
  UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
  RX packets:1535 errors:0 dropped:2 overruns:0 frame:0
  TX packets:43 errors:0 dropped:0 overruns:0 carrier:0
  collisions:0 txqueuelen:1000
  RX bytes:450476 (439.9 KiB)  TX bytes:4248 (4.1 KiB)
  Interrupt:59 Base address:0xc000


  1.  Add bridge to DomU config file:
vif = [ 'bridge=xenbr0' ]


  1.  root@s32g274aevb-Dom0:~# xl create /etc/xen/domU1.cfg
After this I get the Kernel OOPS, because of a failed memory
access
[  413.367873] Unable to handle kernel paging request at virtual address 
06c20070
In
   [  413.548189]  xenvif_rx_ring_slots_available+0x40/0xa0 
[xen_netback]

The full log is attached at [0].
Using gdb, I found out that the access corresponds to the following source code 
lines:
   ==
   (gdb)  l *xenvif_rx_ring_slots_available+0x40
0x65b0 is in xenvif_rx_ring_slots_available 
(/usr/src/kernel/include/linux/skbuff.h:4084).
4082  static inline bool skb_is_gso(const struct sk_buff *skb)
4083  {
4084 return skb_shinfo(skb)->gso_size;
4085  }
  ==


  1.  DomU boots, and then: root@s32g274aevb-DomU1:~# ip link set dev eth0 up

After this, I get a kernel panic in Dom0 because of another failed memory
access
   [ 5338.574809] Unable to handle kernel paging request at virtual 
address 00c00028
in
   [ 5338.753128]  xenvif_tx_build_gops+0x528/0xef8 [xen_netback]

   The full log is attached at [1].
   Using gdb again, I found out the corresponding source code:
  ==
(gdb) l *xenvif_tx_build_gops+0x528
0xce8 is in xenvif_tx_build_gops (/usr/src/kernel/include/linux/skbuff.h:1272).
1269  #ifdef NET_SKBUFF_DATA_USES_OFFSET
1270  static inline unsigned char *skb_end_pointer(const struct sk_buff 
*skb)
1271  {
1272 return skb->head + skb->end;
1273  }
  ==

Do you have any idea why this occurs? Have I misconfigured anything?

I've also tried to pass a static IP configuration for DomU in the config file,
and because it automatically enables eth0 at boot time, I no longer get the
oops, but a panic directly.

Thank you very much for your help,
Andrei Cherechesu,
NXP Semiconductors

Re: [[PATCH v3]] xen/guest_access: Harden *copy_to_guest_offset() to prevent const dest operand

[Stefano Stabellini]

On Fri, 17 Apr 2020, Julien Grall wrote:
> On 16/04/2020 13:19, Jan Beulich wrote:
> > On 16.04.2020 13:24, Julien Grall wrote:
> > > From: Julien Grall 
> > > 
> > > At the moment, *copy_to_guest_offset() will allow the hypervisor to copy
> > > data to guest handle marked const.
> > > 
> > > Thankfully, no users of the helper will do that. Rather than hoping this
> > > can be caught during review, harden copy_to_guest_offset() so the build
> > > will fail if such users are introduced.
> > > 
> > > There is no easy way to check whether a const is NULL in C99. The
> > > approach used is to introduce an unused variable that is non-const and
> > > assign the handle. If the handle were const, this would fail at build
> > > because without an explicit cast, it is not possible to assign a const
> > > variable to a non-const variable.
> > > 
> > > Suggested-by: Jan Beulich 
> > > Signed-off-by: Julien Grall 
> > 
> > Reviewed-by: Jan Beulich 
> > with one further remark:
> > 
> > > --- a/xen/include/asm-x86/guest_access.h
> > > +++ b/xen/include/asm-x86/guest_access.h
> > > @@ -87,6 +87,8 @@
> > >   #define copy_to_guest_offset(hnd, off, ptr, nr) ({  \
> > >   const typeof(*(ptr)) *_s = (ptr);   \
> > >   char (*_d)[sizeof(*_s)] = (void *)(hnd).p;  \
> > > +/* Check if the handle is not const */  \
> > > +void *__maybe_unused _t = (hnd).p;  \
> > 
> > Not being a native speaker, to me "if" doesn't look appropriate
> > here. I'd use "that" instead, but you may want to confirm this.
> > Overall then maybe "Check that the handle is not for a const type"?
> 
> I am happy with the new suggestion. I will fixup while comitting it.
> 
> I would also need a review from Stefano here also.

Acked-by: Stefano Stabellini 

[xen-unstable-smoke test] 149785: regressions - FAIL

[osstest service owner]

flight 149785 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/149785/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64   6 xen-buildfail REGR. vs. 149784

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  3acfd35b61688ad9a5b843ee923221eb36e0b613
baseline version:
 xen  96b5c267e52657e99bd1bbf81dd51925447115e2

Last test of basis   149784  2020-04-24 14:00:40 Z0 days
Testing same since   149785  2020-04-24 17:01:40 Z0 days1 attempts


People who touched revisions under test:
  Ian Jackson 

jobs:
 build-arm64-xsm  pass
 build-amd64  fail
 build-armhf  pass
 build-amd64-libvirt  blocked 
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64blocked 
 test-amd64-amd64-libvirt blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.


commit 3acfd35b61688ad9a5b843ee923221eb36e0b613
Author: Ian Jackson 
Date:   Fri Apr 24 15:49:23 2020 +0100

Update QEMU_TRADITIONAL_REVISION

Signed-off-by: Ian Jackson 
(qemu changes not included)

[PATCH 02/11] xen: Fix and improve handling of device_add usb-host errors

[Markus Armbruster]

usbback_portid_add() leaks the error when qdev_device_add() fails.
Fix that.  While there, use the error to improve the error message.

The qemu_opts_from_qdict() similarly leaks on failure.  But any
failure there is a programming error.  Pass &error_abort.

Fixes: 816ac92ef769f9ffc534e49a1bb6177bddce7aa2
Cc: Stefano Stabellini 
Cc: Anthony Perard 
Cc: Paul Durrant 
Cc: Gerd Hoffmann 
Cc: xen-devel@lists.xenproject.org
Signed-off-by: Markus Armbruster 
---
 hw/usb/xen-usb.c | 18 --
 1 file changed, 8 insertions(+), 10 deletions(-)

diff --git a/hw/usb/xen-usb.c b/hw/usb/xen-usb.c
index 961190d0f7..42643c3390 100644
--- a/hw/usb/xen-usb.c
+++ b/hw/usb/xen-usb.c
@@ -30,6 +30,7 @@
 #include "hw/usb.h"
 #include "hw/xen/xen-legacy-backend.h"
 #include "monitor/qdev.h"
+#include "qapi/error.h"
 #include "qapi/qmp/qdict.h"
 #include "qapi/qmp/qstring.h"
 
@@ -755,13 +756,15 @@ static void usbback_portid_add(struct usbback_info 
*usbif, unsigned port,
 qdict_put_int(qdict, "port", port);
 qdict_put_int(qdict, "hostbus", atoi(busid));
 qdict_put_str(qdict, "hostport", portname);
-opts = qemu_opts_from_qdict(qemu_find_opts("device"), qdict, &local_err);
-if (local_err) {
-goto err;
-}
+opts = qemu_opts_from_qdict(qemu_find_opts("device"), qdict,
+&error_abort);
 usbif->ports[port - 1].dev = USB_DEVICE(qdev_device_add(opts, &local_err));
 if (!usbif->ports[port - 1].dev) {
-goto err;
+qobject_unref(qdict);
+xen_pv_printf(&usbif->xendev, 0,
+  "device %s could not be opened: %s\n",
+  busid, error_get_pretty(local_err));
+error_free(local_err);
 }
 qobject_unref(qdict);
 speed = usbif->ports[port - 1].dev->speed;
@@ -793,11 +796,6 @@ static void usbback_portid_add(struct usbback_info *usbif, 
unsigned port,
 usbback_hotplug_enq(usbif, port);
 
 TR_BUS(&usbif->xendev, "port %d attached\n", port);
-return;
-
-err:
-qobject_unref(qdict);
-xen_pv_printf(&usbif->xendev, 0, "device %s could not be opened\n", busid);
 }
 
 static void usbback_process_port(struct usbback_info *usbif, unsigned port)
-- 
2.21.1

[linux-linus test] 149772: tolerable trouble: fail/pass/starved - PUSHED

[osstest service owner]

flight 149772 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/149772/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-rtds 16 guest-localmigrate   fail  like 149764
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 149764
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 149764
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 149764
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 149764
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 149764
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeatfail  like 149764
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 149764
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 149764
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 149764
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-coresched-i386-xl  2 hosts-allocate   starved  n/a
 test-amd64-coresched-amd64-xl  2 hosts-allocate   starved  n/a

version targeted for testing:
 linuxb4f633221f0aeac102e463a4be46a643b2e3b819
baseline version:
 linuxc578ddb39e565139897124e74e5a43e56538cb33

Last test of basis   149764  2020-04-23 14:40:00 Z1 days
Testing same since   149772  2020-04-24 03:58:26 Z0 days1 attempts


People who touched revisio

[xen-unstable-smoke test] 149784: tolerable all pass - PUSHED

[osstest service owner]

flight 149784 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/149784/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  96b5c267e52657e99bd1bbf81dd51925447115e2
baseline version:
 xen  aa14feb6723d3bb15a884533ade1cd9732792145

Last test of basis   149769  2020-04-23 16:00:35 Z1 days
Testing same since   149784  2020-04-24 14:00:40 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Anthony PERARD 
  Jan Beulich 
  Julien Grall 
  Tamas K Lengyel 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   aa14feb672..96b5c267e5  96b5c267e52657e99bd1bbf81dd51925447115e2 -> smoke

Re: [PATCH] docs/designs: re-work the xenstore migration document...

[Jürgen Groß]

On 24.04.20 18:20, Paul Durrant wrote:

-Original Message-
From: Julien Grall 
Sent: 24 April 2020 17:04
To: Jürgen Groß ; Paul Durrant ; 
xen-devel@lists.xenproject.org
Cc: Paul Durrant 
Subject: Re: [PATCH] docs/designs: re-work the xenstore migration document...

Hi,

On 24/04/2020 16:59, Jürgen Groß wrote:

On 24.04.20 17:44, Julien Grall wrote:
If I extend the record and do a downgrade I'm losing the information,
too, as the old version won't read it in any case. BTW, extending the
record is no problem, as the length is stored in the header. Unknown
records (and extensions) will be just ignored when reading.


That's very much up to the implementation. An implementation may decide
to bail out if the record is not an exact size.



It won't know. The record will be whatever size it says it is, and if the 
format doesn't match what the implementation was expecting then it'll probably 
crash.



In your case when reusing the paddings and doing a downgrade you would
crash, as the paddings are no longer zero.

In case a downgrade should not be done due to vital information loss
then you should just not do it.


Of course, however I don't think a user will necessarily know it should
not do it. So how do you protect against misuse?



The stream is versioned. If information is vital then I'd expect the version to 
be bumped, which should prevent a downgrade.


Uh, this is problematic. I agree that a downgrade will be prevented, but
merely via a crash. We won't have both versions active in parallel, but
the version we are updating to will make it or not. There is no way
back.

The general rule should be to be as forgiving to errors in the stream
as possible in order _not_ to have a crashing xenstored due to strict
parameter testing.

And in case there is a potential of problems it needs to be documented
and then its up to the user to follow the documentation.


Juergen

RE: [PATCH] docs/designs: re-work the xenstore migration document...

[Paul Durrant]

> -Original Message-
> From: Julien Grall 
> Sent: 24 April 2020 17:04
> To: Jürgen Groß ; Paul Durrant ; 
> xen-devel@lists.xenproject.org
> Cc: Paul Durrant 
> Subject: Re: [PATCH] docs/designs: re-work the xenstore migration document...
> 
> Hi,
> 
> On 24/04/2020 16:59, Jürgen Groß wrote:
> > On 24.04.20 17:44, Julien Grall wrote:
> > If I extend the record and do a downgrade I'm losing the information,
> > too, as the old version won't read it in any case. BTW, extending the
> > record is no problem, as the length is stored in the header. Unknown
> > records (and extensions) will be just ignored when reading.
> 
> That's very much up to the implementation. An implementation may decide
> to bail out if the record is not an exact size.
> 

It won't know. The record will be whatever size it says it is, and if the 
format doesn't match what the implementation was expecting then it'll probably 
crash.

> >
> > In your case when reusing the paddings and doing a downgrade you would
> > crash, as the paddings are no longer zero.
> >
> > In case a downgrade should not be done due to vital information loss
> > then you should just not do it.
> 
> Of course, however I don't think a user will necessarily know it should
> not do it. So how do you protect against misuse?
> 

The stream is versioned. If information is vital then I'd expect the version to 
be bumped, which should prevent a downgrade.

  Paul

RE: [ANNOUNCE] Xen 4.14 Development Update

[Durrant, Paul]

> -Original Message-
> 
> > *  VM forking (v11)
> >   -  Tamas K Lengyel
> 
> v17 sent recently, hypervisor side is completely merged, only the
> toolstack patch is waiting for review & merge
> 

Thanks for the update.

  Paul

Re: [XEN PATCH] xen/build: silence make warnings about missing auto.conf*

[Jan Beulich]

On 24.04.2020 16:30, Anthony PERARD wrote:
> In a clean tree, both files include/config/auto.conf{,.cmd} are
> missing and older version of GNU Make complain about it:
> Makefile:103: include/config/auto.conf: No such file or directory
> Makefile:106: include/config/auto.conf.cmd: No such file or directory
> 
> Those warnings are harmless, make will create the files and start over. But
> to avoid confusion, we'll use "-include" to silence the warning.
> 
> Those warning started to appear with commit 6c122d3984a5 ("xen/build:
> include include/config/auto.conf in main Makefile").
> 
> Signed-off-by: Anthony PERARD 

Acked-by: Jan Beulich 

Re: [PATCH] docs/designs: re-work the xenstore migration document...

[Julien Grall]

Hi,

On 24/04/2020 16:59, Jürgen Groß wrote:

On 24.04.20 17:44, Julien Grall wrote:
If I extend the record and do a downgrade I'm losing the information,
too, as the old version won't read it in any case. BTW, extending the
record is no problem, as the length is stored in the header. Unknown
records (and extensions) will be just ignored when reading.


That's very much up to the implementation. An implementation may decide 
to bail out if the record is not an exact size.




In your case when reusing the paddings and doing a downgrade you would
crash, as the paddings are no longer zero.

In case a downgrade should not be done due to vital information loss
then you should just not do it.


Of course, however I don't think a user will necessarily know it should 
not do it. So how do you protect against misuse?


Cheers,

--
Julien Grall

Re: [PATCH] docs/designs: re-work the xenstore migration document...

[Jürgen Groß]

On 24.04.20 17:44, Julien Grall wrote:



On 24/04/2020 15:51, Jürgen Groß wrote:

On 24.04.20 16:38, Julien Grall wrote:

Hi,

On 24/04/2020 15:26, Jürgen Groß wrote:

+```
+    0   1   2   3   4   5   6   7
octet

++---+---+---+---+---+---+---+---+
+| type  | len   |
++---+---+
+| body
+...
+|   | padding (0 to 7 octets)   |
++---+---+
+```
+
+NOTE: padding octets here and in all subsequent format 
specifications must be

+  zero, unless stated otherwise.


What about: "... are written as zero and should be ignored on read."


I would rather not say "ignored" because it doesn't allow us to 
extend the record if needed in a safe way. The read side should abort 
if it sees an other value than 0 in the padding.


I'd rather ignore unknown fields. This allows to add optional data
without having to special case it. 


You will need a special case for 0 in any case.


0 will need to have the "no optional data" semantics.




Writing zeroes is the important part
here, of course.

Ignoring those fields would e.g. enable a downgrade more easily, while
aborting could make that impossible.


You are assuming the fields may contain optional data. Now imagine, we 
realize in a few months we missed some important fields. How would you 
describe the required fields?


I can see two ways:
     1) Re-using the padding fields if possible
     2) Extending the record

If you re-use the padding fields, then when you downgrade you may lose 
information. If you extend the size of the record, then you can't 
downgrade.


If I extend the record and do a downgrade I'm losing the information,
too, as the old version won't read it in any case. BTW, extending the
record is no problem, as the length is stored in the header. Unknown
records (and extensions) will be just ignored when reading.

In your case when reusing the paddings and doing a downgrade you would
crash, as the paddings are no longer zero.

In case a downgrade should not be done due to vital information loss
then you should just not do it.


Juergen

Re: [ANNOUNCE] Xen 4.14 Development Update

[Tamas K Lengyel]

> *  VM forking (v11)
>   -  Tamas K Lengyel

v17 sent recently, hypervisor side is completely merged, only the
toolstack patch is waiting for review & merge

Tamas

Re: [PATCH] docs/designs: re-work the xenstore migration document...

[Julien Grall]

On 24/04/2020 15:51, Jürgen Groß wrote:

On 24.04.20 16:38, Julien Grall wrote:

Hi,

On 24/04/2020 15:26, Jürgen Groß wrote:

+```
+    0   1   2   3   4   5   6   7    octet
++---+---+---+---+---+---+---+---+
+| type  | len   |
++---+---+
+| body
+...
+|   | padding (0 to 7 octets)   |
++---+---+
+```
+
+NOTE: padding octets here and in all subsequent format 
specifications must be

+  zero, unless stated otherwise.


What about: "... are written as zero and should be ignored on read."


I would rather not say "ignored" because it doesn't allow us to extend 
the record if needed in a safe way. The read side should abort if it 
sees an other value than 0 in the padding.


I'd rather ignore unknown fields. This allows to add optional data
without having to special case it. 


You will need a special case for 0 in any case.


Writing zeroes is the important part
here, of course.

Ignoring those fields would e.g. enable a downgrade more easily, while
aborting could make that impossible.


You are assuming the fields may contain optional data. Now imagine, we 
realize in a few months we missed some important fields. How would you 
describe the required fields?


I can see two ways:
1) Re-using the padding fields if possible
2) Extending the record

If you re-use the padding fields, then when you downgrade you may lose 
information. If you extend the size of the record, then you can't downgrade.


Cheers,

--
Julien Grall

Re: [PATCH] docs/designs: re-work the xenstore migration document...

[Jürgen Groß]

On 24.04.20 16:38, Julien Grall wrote:

Hi,

On 24/04/2020 15:26, Jürgen Groß wrote:

+```
+    0   1   2   3   4   5   6   7    octet
++---+---+---+---+---+---+---+---+
+| type  | len   |
++---+---+
+| body
+...
+|   | padding (0 to 7 octets)   |
++---+---+
+```
+
+NOTE: padding octets here and in all subsequent format 
specifications must be

+  zero, unless stated otherwise.


What about: "... are written as zero and should be ignored on read."


I would rather not say "ignored" because it doesn't allow us to extend 
the record if needed in a safe way. The read side should abort if it 
sees an other value than 0 in the padding.


I'd rather ignore unknown fields. This allows to add optional data
without having to special case it. Writing zeroes is the important part
here, of course.

Ignoring those fields would e.g. enable a downgrade more easily, while
aborting could make that impossible.

And in case a version would write non-zero bytes (e.g. due to a bug)
this version could never be live-updated.

Juergen

Re: [PATCH 0/3] Cleanup IOREQ server on exit

[Ian Jackson]

Maximilian Heyne writes ("Re: [PATCH 0/3] Cleanup IOREQ server on exit"):
> Could someone please have a look at this patch? It solves an actual issue:
> Try soft-reset with qemu-xen-traditional and it will fail.

Thanks.  I reviewed this.

qemu is in deep freeze but the changes looked correct and are indeed
solving a regression.  I convinced myself that they were appropriately
low risk, so

Acked-by: Ian Jackson 

for all three and I have pushed them.

In theory a backport might be appropriate since this is a bugfix but
my inclination is to leave existing releases where they are, since
anyone using qemu-trad probably wants things super-stable.  Contrary
opinions welcome.

It has been a very long time since I did an update of qemu trad so it
is possible that I have mangled the process somehow.  We will see I
guess...

Thanks also to Paul for chasing me about this.

Regards,
Ian.

Re: [PATCH] docs/designs: re-work the xenstore migration document...

[Julien Grall]

Hi,

On 24/04/2020 15:26, Jürgen Groß wrote:

+```
+    0   1   2   3   4   5   6   7    octet
++---+---+---+---+---+---+---+---+
+| type  | len   |
++---+---+
+| body
+...
+|   | padding (0 to 7 octets)   |
++---+---+
+```
+
+NOTE: padding octets here and in all subsequent format specifications 
must be

+  zero, unless stated otherwise.


What about: "... are written as zero and should be ignored on read."


I would rather not say "ignored" because it doesn't allow us to extend 
the record if needed in a safe way. The read side should abort if it 
sees an other value than 0 in the padding.


Cheers,

--
Julien Grall

[XEN PATCH] xen/build: silence make warnings about missing auto.conf*

[Anthony PERARD]

In a clean tree, both files include/config/auto.conf{,.cmd} are
missing and older version of GNU Make complain about it:
Makefile:103: include/config/auto.conf: No such file or directory
Makefile:106: include/config/auto.conf.cmd: No such file or directory

Those warnings are harmless, make will create the files and start over. But
to avoid confusion, we'll use "-include" to silence the warning.

Those warning started to appear with commit 6c122d3984a5 ("xen/build:
include include/config/auto.conf in main Makefile").

Signed-off-by: Anthony PERARD 
---
 xen/Makefile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/Makefile b/xen/Makefile
index fc8eef6a2817..eedfef26b245 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -154,10 +154,10 @@ config: FORCE
 else # !config-build
 
 ifeq ($(need-config),y)
-include include/config/auto.conf
+-include include/config/auto.conf
 # Read in dependencies to all Kconfig* files, make sure to run syncconfig if
 # changes are detected.
-include include/config/auto.conf.cmd
+-include include/config/auto.conf.cmd
 
 # Allow people to just run `make` as before and not force them to configure
 $(KCONFIG_CONFIG):
-- 
Anthony PERARD

Re: [PATCH] docs/designs: re-work the xenstore migration document...

[Jürgen Groß]

On 24.04.20 15:37, Paul Durrant wrote:

From: Paul Durrant 

... to specify a separate migration stream that will also be suitable for
live update.

The original scope of the document was to support non-cooperative migration
of guests [1] but, since then, live update of xenstored has been brought into
scope. Thus it makes more sense to define a separate image format for
serializing xenstore state that is suitable for both purposes.

The document has been limited to specifying a new image format. The mechanism
for acquiring the image for live update or migration is not covered as that
is more appropriately dealt with by a patch to docs/misc/xenstore.txt. It is
also expected that, when the first implementation of live update or migration
making use of this specification is committed, that the document is moved from
docs/designs into docs/specs.

[1] See 
https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=docs/designs/non-cooperative-migration.md

Signed-off-by: Paul Durrant 
---
Juergen Gross 
Andrew Cooper 
George Dunlap 
Ian Jackson 
Jan Beulich 
Julien Grall 
Stefano Stabellini 
Wei Liu 
---
  docs/designs/xenstore-migration.md | 472 +++--
  1 file changed, 309 insertions(+), 163 deletions(-)

diff --git a/docs/designs/xenstore-migration.md 
b/docs/designs/xenstore-migration.md
index 6ab351e8fe..c96bad48eb 100644
--- a/docs/designs/xenstore-migration.md
+++ b/docs/designs/xenstore-migration.md
@@ -3,254 +3,400 @@
  ## Background
  
  The design for *Non-Cooperative Migration of Guests*[1] explains that extra

-save records are required in the migrations stream to allow a guest running
-PV drivers to be migrated without its co-operation. Moreover the save
-records must include details of registered xenstore watches as well as
-content; information that cannot currently be recovered from `xenstored`,
-and hence some extension to the xenstore protocol[2] will also be required.
-
-The *libxenlight Domain Image Format* specification[3] already defines a
-record type `EMULATOR_XENSTORE_DATA` but this is not suitable for
-transferring xenstore data pertaining to the domain directly as it is
-specified such that keys are relative to the path
-`/local/domain/$dm_domid/device-model/$domid`. Thus it is necessary to
-define at least one new save record type.
+save records are required in the migrations stream to allow a guest running PV
+drivers to be migrated without its co-operation. Moreover the save records must
+include details of registered xenstore watches as well ascontent; information
+that cannot currently be recovered from `xenstored`, and hence some extension
+to the xenstored implementations will also be required.
+
+As a similar set of data is needed for transferring xenstore data from one
+instance to another when live updating xenstored this document proposes an
+image format for a 'migration stream' suitable for both purposes.
  
  ## Proposal
  
-### New Save Record

+The image format consists of a _header_ followed by 1 or more _records_. Each
+record consists of a type and length field, followed by any data mandated by
+the record type. At minimum there will be a single record of type `END`
+(defined below).
  
-A new mandatory record type should be defined within the libxenlight Domain

-Image Format:
+### Header
  
-`0x0007: DOMAIN_XENSTORE_DATA`

+The header identifies the stream as a `xenstore` stream, including the version
+of the specification that it complies with.
  
-An arbitrary number of these records may be present in the migration

-stream and may appear in any order. The format of each record should be as
-follows:
+All fields in this header must be in _big-endian_ byte order, regardless of
+the setting of the endianness bit.
  
  
  ```

  0   1   2   3   4   5   6   7octet
  +---+---+---+---+---+---+---+---+
-| type  | record specific data  |
-+---+   |
-...
-+---+
+| ident |
++---+---|
+| version   | flags |
++---+---+
  ```
  
-where type is one of the following values
  
+| Field | Description   |

+|---|---|
+| `ident`   | 0x78656e73746f7265 ('xenstore' in ASCII)  |
+|   |   |
+| `version` | 0x0001 (the version of the specification) |
+|   |   |
+| `flags`   | 0 (LSB): Endianness: 0 = little, 1 = big  |
+|   |   |
+|   | 1-31: Reserved (mu

[PATCH v6 15/15] x86/mm: drop _new suffix for page table APIs

[Hongyan Xia]

From: Wei Liu 

No functional change.

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
---
 xen/arch/x86/mm.c| 48 
 xen/arch/x86/smpboot.c   | 12 +-
 xen/arch/x86/x86_64/mm.c | 10 -
 xen/common/efi/boot.c| 10 -
 xen/include/asm-x86/mm.h |  4 ++--
 5 files changed, 42 insertions(+), 42 deletions(-)

diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 7e212cc3e0..a17ae0004a 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -356,7 +356,7 @@ void __init arch_init_memory(void)
 ASSERT(root_pgt_pv_xen_slots < ROOT_PAGETABLE_PV_XEN_SLOTS);
 if ( l4_table_offset(split_va) == l4_table_offset(split_va - 1) )
 {
-mfn_t l3mfn = alloc_xen_pagetable_new();
+mfn_t l3mfn = alloc_xen_pagetable();
 
 if ( !mfn_eq(l3mfn, INVALID_MFN) )
 {
@@ -4919,7 +4919,7 @@ int mmcfg_intercept_write(
  * them. The caller must check whether the allocation has succeeded, and only
  * pass valid MFNs to map_domain_page().
  */
-mfn_t alloc_xen_pagetable_new(void)
+mfn_t alloc_xen_pagetable(void)
 {
 if ( system_state != SYS_STATE_early_boot )
 {
@@ -4934,7 +4934,7 @@ mfn_t alloc_xen_pagetable_new(void)
 }
 
 /* mfn can be INVALID_MFN */
-void free_xen_pagetable_new(mfn_t mfn)
+void free_xen_pagetable(mfn_t mfn)
 {
 if ( system_state != SYS_STATE_early_boot && !mfn_eq(mfn, INVALID_MFN) )
 free_domheap_page(mfn_to_page(mfn));
@@ -4955,7 +4955,7 @@ static l3_pgentry_t *virt_to_xen_l3e(unsigned long v)
 {
 bool locking = system_state > SYS_STATE_boot;
 l3_pgentry_t *l3t;
-mfn_t l3mfn = alloc_xen_pagetable_new();
+mfn_t l3mfn = alloc_xen_pagetable();
 
 if ( mfn_eq(l3mfn, INVALID_MFN) )
 return NULL;
@@ -4974,7 +4974,7 @@ static l3_pgentry_t *virt_to_xen_l3e(unsigned long v)
 }
 if ( locking )
 spin_unlock(&map_pgdir_lock);
-free_xen_pagetable_new(l3mfn);
+free_xen_pagetable(l3mfn);
 }
 
 return map_l3t_from_l4e(*pl4e) + l3_table_offset(v);
@@ -4993,7 +4993,7 @@ static l2_pgentry_t *virt_to_xen_l2e(unsigned long v)
 {
 bool locking = system_state > SYS_STATE_boot;
 l2_pgentry_t *l2t;
-mfn_t l2mfn = alloc_xen_pagetable_new();
+mfn_t l2mfn = alloc_xen_pagetable();
 
 if ( mfn_eq(l2mfn, INVALID_MFN) )
 {
@@ -5012,7 +5012,7 @@ static l2_pgentry_t *virt_to_xen_l2e(unsigned long v)
 }
 if ( locking )
 spin_unlock(&map_pgdir_lock);
-free_xen_pagetable_new(l2mfn);
+free_xen_pagetable(l2mfn);
 }
 
 BUG_ON(l3e_get_flags(*pl3e) & _PAGE_PSE);
@@ -5035,7 +5035,7 @@ l1_pgentry_t *virt_to_xen_l1e(unsigned long v)
 {
 bool locking = system_state > SYS_STATE_boot;
 l1_pgentry_t *l1t;
-mfn_t l1mfn = alloc_xen_pagetable_new();
+mfn_t l1mfn = alloc_xen_pagetable();
 
 if ( mfn_eq(l1mfn, INVALID_MFN) )
 {
@@ -5054,7 +5054,7 @@ l1_pgentry_t *virt_to_xen_l1e(unsigned long v)
 }
 if ( locking )
 spin_unlock(&map_pgdir_lock);
-free_xen_pagetable_new(l1mfn);
+free_xen_pagetable(l1mfn);
 }
 
 BUG_ON(l2e_get_flags(*pl2e) & _PAGE_PSE);
@@ -5163,10 +5163,10 @@ int map_pages_to_xen(
 ol2e = l2t[i];
 if ( (l2e_get_flags(ol2e) & _PAGE_PRESENT) &&
  !(l2e_get_flags(ol2e) & _PAGE_PSE) )
-free_xen_pagetable_new(l2e_get_mfn(ol2e));
+free_xen_pagetable(l2e_get_mfn(ol2e));
 }
 unmap_domain_page(l2t);
-free_xen_pagetable_new(l3e_get_mfn(ol3e));
+free_xen_pagetable(l3e_get_mfn(ol3e));
 }
 }
 
@@ -5205,7 +5205,7 @@ int map_pages_to_xen(
 continue;
 }
 
-l2mfn = alloc_xen_pagetable_new();
+l2mfn = alloc_xen_pagetable();
 if ( mfn_eq(l2mfn, INVALID_MFN) )
 goto out;
 
@@ -5233,7 +5233,7 @@ int map_pages_to_xen(
 spin_unlock(&map_pgdir_lock);
 flush_area(virt, flush_flags);
 
-free_xen_pagetable_new(l2mfn);
+free_xen_pagetable(l2mfn);
 }
 
 pl2e = virt_to_xen_l2e(virt);
@@ -5267,7 +5267,7 @@ int map_pages_to_xen(
 flush_flags(l1e_get_flags(l1t[i]));
 flush_area(virt, flush_flags);
 unmap_domain_page(l1t);
-free_xen_pagetable_new(l2e_get_mfn(ol2e));
+free_xen_pagetable(l2e_get_mfn(ol2e));
 }
 }
 
@@ -5313,7 +5313,7 @@ int map_pages_to_xen(
 goto check_l3;
 }
 
-l1mfn = alloc_xen_pagetable_new();
+   

[PATCH v6 10/15] efi: switch to new APIs in EFI code

[Hongyan Xia]

From: Wei Liu 

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
---
 xen/arch/x86/efi/runtime.h | 10 +++--
 xen/common/efi/boot.c  | 46 ++
 xen/common/efi/efi.h   |  3 ++-
 xen/common/efi/runtime.c   |  8 +++
 4 files changed, 46 insertions(+), 21 deletions(-)

diff --git a/xen/arch/x86/efi/runtime.h b/xen/arch/x86/efi/runtime.h
index d9eb8f5c27..d2483645c6 100644
--- a/xen/arch/x86/efi/runtime.h
+++ b/xen/arch/x86/efi/runtime.h
@@ -1,12 +1,18 @@
+#include 
+#include 
 #include 
 #include 
 
 #ifndef COMPAT
-l4_pgentry_t *__read_mostly efi_l4_pgtable;
+mfn_t __read_mostly efi_l4_mfn = INVALID_MFN_INITIALIZER;
 
 void efi_update_l4_pgtable(unsigned int l4idx, l4_pgentry_t l4e)
 {
-if ( efi_l4_pgtable )
+if ( !mfn_eq(efi_l4_mfn, INVALID_MFN) )
+{
+l4_pgentry_t *efi_l4_pgtable = map_domain_page(efi_l4_mfn);
 l4e_write(efi_l4_pgtable + l4idx, l4e);
+unmap_domain_page(efi_l4_pgtable);
+}
 }
 #endif
diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
index 8e96ef8de2..715217d2a9 100644
--- a/xen/common/efi/boot.c
+++ b/xen/common/efi/boot.c
@@ -6,6 +6,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -1442,6 +1443,7 @@ static __init void copy_mapping(unsigned long mfn, 
unsigned long end,
  unsigned long emfn))
 {
 unsigned long next;
+l4_pgentry_t *efi_l4_pgtable = map_domain_page(efi_l4_mfn);
 
 for ( ; mfn < end; mfn = next )
 {
@@ -1469,6 +1471,8 @@ static __init void copy_mapping(unsigned long mfn, 
unsigned long end,
 unmap_domain_page(l3src);
 unmap_domain_page(l3dst);
 }
+
+unmap_domain_page(efi_l4_pgtable);
 }
 
 static bool __init ram_range_valid(unsigned long smfn, unsigned long emfn)
@@ -1489,6 +1493,7 @@ static bool __init rt_range_valid(unsigned long smfn, 
unsigned long emfn)
 void __init efi_init_memory(void)
 {
 unsigned int i;
+l4_pgentry_t *efi_l4_pgtable;
 struct rt_extra {
 struct rt_extra *next;
 unsigned long smfn, emfn;
@@ -1603,8 +1608,9 @@ void __init efi_init_memory(void)
  * Set up 1:1 page tables for runtime calls. See SetVirtualAddressMap() in
  * efi_exit_boot().
  */
-efi_l4_pgtable = alloc_xen_pagetable();
-BUG_ON(!efi_l4_pgtable);
+efi_l4_mfn = alloc_xen_pagetable_new();
+BUG_ON(mfn_eq(efi_l4_mfn, INVALID_MFN));
+efi_l4_pgtable = map_domain_page(efi_l4_mfn);
 clear_page(efi_l4_pgtable);
 
 copy_mapping(0, max_page, ram_range_valid);
@@ -1637,39 +1643,45 @@ void __init efi_init_memory(void)
 
 if ( !(l4e_get_flags(l4e) & _PAGE_PRESENT) )
 {
-pl3e = alloc_xen_pagetable();
-BUG_ON(!pl3e);
+mfn_t l3mfn = alloc_xen_pagetable_new();
+
+BUG_ON(mfn_eq(l3mfn, INVALID_MFN));
+pl3e = map_domain_page(l3mfn);
 clear_page(pl3e);
 efi_l4_pgtable[l4_table_offset(addr)] =
-l4e_from_paddr(virt_to_maddr(pl3e), __PAGE_HYPERVISOR);
+l4e_from_mfn(l3mfn, __PAGE_HYPERVISOR);
 }
 else
-pl3e = l4e_to_l3e(l4e);
+pl3e = map_l3t_from_l4e(l4e);
 pl3e += l3_table_offset(addr);
 if ( !(l3e_get_flags(*pl3e) & _PAGE_PRESENT) )
 {
-pl2e = alloc_xen_pagetable();
-BUG_ON(!pl2e);
+mfn_t l2mfn = alloc_xen_pagetable_new();
+
+BUG_ON(mfn_eq(l2mfn, INVALID_MFN));
+pl2e = map_domain_page(l2mfn);
 clear_page(pl2e);
-*pl3e = l3e_from_paddr(virt_to_maddr(pl2e), __PAGE_HYPERVISOR);
+*pl3e = l3e_from_mfn(l2mfn, __PAGE_HYPERVISOR);
 }
 else
 {
 BUG_ON(l3e_get_flags(*pl3e) & _PAGE_PSE);
-pl2e = l3e_to_l2e(*pl3e);
+pl2e = map_l2t_from_l3e(*pl3e);
 }
 pl2e += l2_table_offset(addr);
 if ( !(l2e_get_flags(*pl2e) & _PAGE_PRESENT) )
 {
-l1t = alloc_xen_pagetable();
-BUG_ON(!l1t);
+mfn_t l1mfn = alloc_xen_pagetable_new();
+
+BUG_ON(mfn_eq(l1mfn, INVALID_MFN));
+l1t = map_domain_page(l1mfn);
 clear_page(l1t);
-*pl2e = l2e_from_paddr(virt_to_maddr(l1t), __PAGE_HYPERVISOR);
+*pl2e = l2e_from_mfn(l1mfn, __PAGE_HYPERVISOR);
 }
 else
 {
 BUG_ON(l2e_get_flags(*pl2e) & _PAGE_PSE);
-l1t = l2e_to_l1e(*pl2e);
+l1t = map_l1t_from_l2e(*pl2e);
 }
 for ( i = l1_table_offset(addr);
   i < L1_PAGETABLE_ENTRIES && extra->smfn < extra->emfn;
@@ -1681,11 +1693,17 @@ void __init efi_init_memory(void)
 extra_head = extra->next;
 xfree(extra);
 }
+
+unmap_domain_page(l1t);
+unmap_domain_page(pl2e);
+unmap_domain_page(pl3e);
 }
 
 /* Insert Xen mappi

[PATCH v6 13/15] x86/mm: drop old page table APIs

[Hongyan Xia]

From: Hongyan Xia 

Two sets of old APIs, alloc/free_xen_pagetable() and lXe_to_lYe(), are
now dropped to avoid the dependency on direct map.

There are two special cases which still have not been re-written into
the new APIs, thus need special treatment:

rpt in smpboot.c cannot use ephemeral mappings yet. The problem is that
rpt is read and written in context switch code, but the mapping
infrastructure is NOT context-switch-safe, meaning we cannot map rpt in
one domain and unmap in another. Before the mapping infrastructure
supports context switches, rpt has to be globally mapped.

Also, lXe_to_lYe() during Xen image relocation cannot be converted into
map/unmap pairs. We cannot hold on to mappings while the mapping
infrastructure is being relocated! It is enough to remove the direct map
in the second e820 pass, so we still use the direct map (<4GiB) in Xen
relocation (which is during the first e820 pass).

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
---
 xen/arch/x86/mm.c  | 14 --
 xen/arch/x86/setup.c   |  4 ++--
 xen/arch/x86/smpboot.c |  4 ++--
 xen/include/asm-x86/mm.h   |  2 --
 xen/include/asm-x86/page.h |  5 -
 5 files changed, 4 insertions(+), 25 deletions(-)

diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index e8c16027d8..749b6f23e5 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -4913,20 +4913,6 @@ int mmcfg_intercept_write(
 return X86EMUL_OKAY;
 }
 
-void *alloc_xen_pagetable(void)
-{
-mfn_t mfn = alloc_xen_pagetable_new();
-
-return mfn_eq(mfn, INVALID_MFN) ? NULL : mfn_to_virt(mfn_x(mfn));
-}
-
-void free_xen_pagetable(void *v)
-{
-mfn_t mfn = v ? virt_to_mfn(v) : INVALID_MFN;
-
-free_xen_pagetable_new(mfn);
-}
-
 /*
  * For these PTE APIs, the caller must follow the alloc-map-unmap-free
  * lifecycle, which means explicitly mapping the PTE pages before accessing
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 885919d5c3..c76fbf80dc 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1114,7 +1114,7 @@ void __init noreturn __start_xen(unsigned long mbi_p)
 continue;
 *pl4e = l4e_from_intpte(l4e_get_intpte(*pl4e) +
 xen_phys_start);
-pl3e = l4e_to_l3e(*pl4e);
+pl3e = __va(l4e_get_paddr(*pl4e));
 for ( j = 0; j < L3_PAGETABLE_ENTRIES; j++, pl3e++ )
 {
 /* Not present, 1GB mapping, or already relocated? */
@@ -1124,7 +1124,7 @@ void __init noreturn __start_xen(unsigned long mbi_p)
 continue;
 *pl3e = l3e_from_intpte(l3e_get_intpte(*pl3e) +
 xen_phys_start);
-pl2e = l3e_to_l2e(*pl3e);
+pl2e = __va(l3e_get_paddr(*pl3e));
 for ( k = 0; k < L2_PAGETABLE_ENTRIES; k++, pl2e++ )
 {
 /* Not present, PSE, or already relocated? */
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index 0e0ae56c76..71d61794ec 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -815,7 +815,7 @@ static int setup_cpu_root_pgt(unsigned int cpu)
 if ( !opt_xpti_hwdom && !opt_xpti_domu )
 return 0;
 
-rpt = alloc_xen_pagetable();
+rpt = alloc_xenheap_page();
 if ( !rpt )
 return -ENOMEM;
 
@@ -919,7 +919,7 @@ static void cleanup_cpu_root_pgt(unsigned int cpu)
 free_xen_pagetable_new(l3mfn);
 }
 
-free_xen_pagetable(rpt);
+free_xenheap_page(rpt);
 
 /* Also zap the stub mapping for this CPU. */
 if ( stub_linear )
diff --git a/xen/include/asm-x86/mm.h b/xen/include/asm-x86/mm.h
index 3d3f9d49ac..cf855b48fd 100644
--- a/xen/include/asm-x86/mm.h
+++ b/xen/include/asm-x86/mm.h
@@ -583,8 +583,6 @@ int vcpu_destroy_pagetables(struct vcpu *);
 void *do_page_walk(struct vcpu *v, unsigned long addr);
 
 /* Allocator functions for Xen pagetables. */
-void *alloc_xen_pagetable(void);
-void free_xen_pagetable(void *v);
 mfn_t alloc_xen_pagetable_new(void);
 void free_xen_pagetable_new(mfn_t mfn);
 
diff --git a/xen/include/asm-x86/page.h b/xen/include/asm-x86/page.h
index 8f711f4992..2e3056c08e 100644
--- a/xen/include/asm-x86/page.h
+++ b/xen/include/asm-x86/page.h
@@ -188,11 +188,6 @@ static inline l4_pgentry_t l4e_from_paddr(paddr_t pa, 
unsigned int flags)
 #define l4e_has_changed(x,y,flags) \
 ( !!(((x).l4 ^ (y).l4) & ((PADDR_MASK&PAGE_MASK)|put_pte_flags(flags))) )
 
-/* Pagetable walking. */
-#define l2e_to_l1e(x)  ((l1_pgentry_t *)__va(l2e_get_paddr(x)))
-#define l3e_to_l2e(x)  ((l2_pgentry_t *)__va(l3e_get_paddr(x)))
-#define l4e_to_l3e(x)  ((l3_pgentry_t *)__va(l4e_get_paddr(x)))
-
 #define map_l1t_from_l2e(x)(l1_pgentry_t 
*)map_domain_page(l2e_get_mfn(x))
 #define map_l2t_from_l3e(x)(l2_pgentry_t 
*)map_domain_page(l3e_get_mfn(x))
 #define 

[PATCH v6 14/15] x86: switch to use domheap page for page tables

[Hongyan Xia]

From: Hongyan Xia 

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
---
 xen/arch/x86/mm.c | 9 +
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 749b6f23e5..7e212cc3e0 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -4923,10 +4923,11 @@ mfn_t alloc_xen_pagetable_new(void)
 {
 if ( system_state != SYS_STATE_early_boot )
 {
-void *ptr = alloc_xenheap_page();
 
-BUG_ON(!hardware_domain && !ptr);
-return ptr ? virt_to_mfn(ptr) : INVALID_MFN;
+struct page_info *pg = alloc_domheap_page(NULL, 0);
+
+BUG_ON(!hardware_domain && !pg);
+return pg ? page_to_mfn(pg) : INVALID_MFN;
 }
 
 return alloc_boot_pages(1, 1);
@@ -4936,7 +4937,7 @@ mfn_t alloc_xen_pagetable_new(void)
 void free_xen_pagetable_new(mfn_t mfn)
 {
 if ( system_state != SYS_STATE_early_boot && !mfn_eq(mfn, INVALID_MFN) )
-free_xenheap_page(mfn_to_virt(mfn_x(mfn)));
+free_domheap_page(mfn_to_page(mfn));
 }
 
 static DEFINE_SPINLOCK(map_pgdir_lock);
-- 
2.24.1.AMZN

[PATCH v6 11/15] x86/smpboot: clone_mapping should have one exit path

[Hongyan Xia]

From: Wei Liu 

We will soon need to clean up page table mappings in the exit path.

No functional change.

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
---
 xen/arch/x86/smpboot.c | 16 +++-
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index 275ce7661d..5b0e24f925 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -675,6 +675,7 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
 l3_pgentry_t *pl3e;
 l2_pgentry_t *pl2e;
 l1_pgentry_t *pl1e;
+int rc = -ENOMEM;
 
 /*
  * Sanity check 'linear'.  We only allow cloning from the Xen virtual
@@ -715,7 +716,10 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
 pl1e = l2e_to_l1e(*pl2e) + l1_table_offset(linear);
 flags = l1e_get_flags(*pl1e);
 if ( !(flags & _PAGE_PRESENT) )
-return 0;
+{
+rc = 0;
+goto out;
+}
 pfn = l1e_get_pfn(*pl1e);
 }
 }
@@ -724,7 +728,7 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
 {
 pl3e = alloc_xen_pagetable();
 if ( !pl3e )
-return -ENOMEM;
+goto out;
 clear_page(pl3e);
 l4e_write(&rpt[root_table_offset(linear)],
   l4e_from_paddr(__pa(pl3e), __PAGE_HYPERVISOR));
@@ -738,7 +742,7 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
 {
 pl2e = alloc_xen_pagetable();
 if ( !pl2e )
-return -ENOMEM;
+goto out;
 clear_page(pl2e);
 l3e_write(pl3e, l3e_from_paddr(__pa(pl2e), __PAGE_HYPERVISOR));
 }
@@ -754,7 +758,7 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
 {
 pl1e = alloc_xen_pagetable();
 if ( !pl1e )
-return -ENOMEM;
+goto out;
 clear_page(pl1e);
 l2e_write(pl2e, l2e_from_paddr(__pa(pl1e), __PAGE_HYPERVISOR));
 }
@@ -775,7 +779,9 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
 else
 l1e_write(pl1e, l1e_from_pfn(pfn, flags));
 
-return 0;
+rc = 0;
+ out:
+return rc;
 }
 
 DEFINE_PER_CPU(root_pgentry_t *, root_pgt);
-- 
2.24.1.AMZN

[PATCH v6 12/15] x86/smpboot: switch pl*e to use new APIs in clone_mapping

[Hongyan Xia]

From: Wei Liu 

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
---
 xen/arch/x86/smpboot.c | 54 +++---
 1 file changed, 35 insertions(+), 19 deletions(-)

diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index 5b0e24f925..0e0ae56c76 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -672,9 +672,9 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
 {
 unsigned long linear = (unsigned long)ptr, pfn;
 unsigned int flags;
-l3_pgentry_t *pl3e;
-l2_pgentry_t *pl2e;
-l1_pgentry_t *pl1e;
+l3_pgentry_t *pl3e = NULL;
+l2_pgentry_t *pl2e = NULL;
+l1_pgentry_t *pl1e = NULL;
 int rc = -ENOMEM;
 
 /*
@@ -689,8 +689,8 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
  (linear >= XEN_VIRT_END && linear < DIRECTMAP_VIRT_START) )
 return -EINVAL;
 
-pl3e = l4e_to_l3e(idle_pg_table[root_table_offset(linear)]) +
-l3_table_offset(linear);
+pl3e = map_l3t_from_l4e(idle_pg_table[root_table_offset(linear)]);
+pl3e += l3_table_offset(linear);
 
 flags = l3e_get_flags(*pl3e);
 ASSERT(flags & _PAGE_PRESENT);
@@ -702,7 +702,7 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
 }
 else
 {
-pl2e = l3e_to_l2e(*pl3e) + l2_table_offset(linear);
+pl2e = map_l2t_from_l3e(*pl3e) + l2_table_offset(linear);
 flags = l2e_get_flags(*pl2e);
 ASSERT(flags & _PAGE_PRESENT);
 if ( flags & _PAGE_PSE )
@@ -713,7 +713,7 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
 }
 else
 {
-pl1e = l2e_to_l1e(*pl2e) + l1_table_offset(linear);
+pl1e = map_l1t_from_l2e(*pl2e) + l1_table_offset(linear);
 flags = l1e_get_flags(*pl1e);
 if ( !(flags & _PAGE_PRESENT) )
 {
@@ -724,48 +724,61 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
 }
 }
 
+UNMAP_DOMAIN_PAGE(pl1e);
+UNMAP_DOMAIN_PAGE(pl2e);
+UNMAP_DOMAIN_PAGE(pl3e);
+
 if ( !(root_get_flags(rpt[root_table_offset(linear)]) & _PAGE_PRESENT) )
 {
-pl3e = alloc_xen_pagetable();
-if ( !pl3e )
+mfn_t l3mfn = alloc_xen_pagetable_new();
+
+if ( mfn_eq(l3mfn, INVALID_MFN) )
 goto out;
+
+pl3e = map_domain_page(l3mfn);
 clear_page(pl3e);
 l4e_write(&rpt[root_table_offset(linear)],
-  l4e_from_paddr(__pa(pl3e), __PAGE_HYPERVISOR));
+  l4e_from_mfn(l3mfn, __PAGE_HYPERVISOR));
 }
 else
-pl3e = l4e_to_l3e(rpt[root_table_offset(linear)]);
+pl3e = map_l3t_from_l4e(rpt[root_table_offset(linear)]);
 
 pl3e += l3_table_offset(linear);
 
 if ( !(l3e_get_flags(*pl3e) & _PAGE_PRESENT) )
 {
-pl2e = alloc_xen_pagetable();
-if ( !pl2e )
+mfn_t l2mfn = alloc_xen_pagetable_new();
+
+if ( mfn_eq(l2mfn, INVALID_MFN) )
 goto out;
+
+pl2e = map_domain_page(l2mfn);
 clear_page(pl2e);
-l3e_write(pl3e, l3e_from_paddr(__pa(pl2e), __PAGE_HYPERVISOR));
+l3e_write(pl3e, l3e_from_mfn(l2mfn, __PAGE_HYPERVISOR));
 }
 else
 {
 ASSERT(!(l3e_get_flags(*pl3e) & _PAGE_PSE));
-pl2e = l3e_to_l2e(*pl3e);
+pl2e = map_l2t_from_l3e(*pl3e);
 }
 
 pl2e += l2_table_offset(linear);
 
 if ( !(l2e_get_flags(*pl2e) & _PAGE_PRESENT) )
 {
-pl1e = alloc_xen_pagetable();
-if ( !pl1e )
+mfn_t l1mfn = alloc_xen_pagetable_new();
+
+if ( mfn_eq(l1mfn, INVALID_MFN) )
 goto out;
+
+pl1e = map_domain_page(l1mfn);
 clear_page(pl1e);
-l2e_write(pl2e, l2e_from_paddr(__pa(pl1e), __PAGE_HYPERVISOR));
+l2e_write(pl2e, l2e_from_mfn(l1mfn, __PAGE_HYPERVISOR));
 }
 else
 {
 ASSERT(!(l2e_get_flags(*pl2e) & _PAGE_PSE));
-pl1e = l2e_to_l1e(*pl2e);
+pl1e = map_l1t_from_l2e(*pl2e);
 }
 
 pl1e += l1_table_offset(linear);
@@ -781,6 +794,9 @@ static int clone_mapping(const void *ptr, root_pgentry_t 
*rpt)
 
 rc = 0;
  out:
+UNMAP_DOMAIN_PAGE(pl1e);
+UNMAP_DOMAIN_PAGE(pl2e);
+UNMAP_DOMAIN_PAGE(pl3e);
 return rc;
 }
 
-- 
2.24.1.AMZN

[PATCH v6 02/15] x86/mm: make sure there is one exit path for modify_xen_mappings

[Hongyan Xia]

From: Wei Liu 

We will soon need to handle dynamically mapping / unmapping page
tables in the said function. Since dynamic mappings may map and unmap
pl3e in different iterations, lift pl3e out of the loop.

No functional change.

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 

---
Changed since v4:
- drop the end_of_loop goto label.

Changed since v3:
- remove asserts on rc since it never gets changed to anything else.
---
 xen/arch/x86/mm.c | 15 +++
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 0d7f1f1265..13a34a8d57 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -5462,10 +5462,12 @@ int populate_pt_range(unsigned long virt, unsigned long 
nr_mfns)
 int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf)
 {
 bool locking = system_state > SYS_STATE_boot;
+l3_pgentry_t *pl3e;
 l2_pgentry_t *pl2e;
 l1_pgentry_t *pl1e;
 unsigned int  i;
 unsigned long v = s;
+int rc = -ENOMEM;
 
 /* Set of valid PTE bits which may be altered. */
 #define FLAGS_MASK (_PAGE_NX|_PAGE_RW|_PAGE_PRESENT)
@@ -5476,7 +5478,7 @@ int modify_xen_mappings(unsigned long s, unsigned long e, 
unsigned int nf)
 
 while ( v < e )
 {
-l3_pgentry_t *pl3e = virt_to_xen_l3e(v);
+pl3e = virt_to_xen_l3e(v);
 
 if ( !pl3e || !(l3e_get_flags(*pl3e) & _PAGE_PRESENT) )
 {
@@ -5509,7 +5511,8 @@ int modify_xen_mappings(unsigned long s, unsigned long e, 
unsigned int nf)
 /* PAGE1GB: shatter the superpage and fall through. */
 l2t = alloc_xen_pagetable();
 if ( !l2t )
-return -ENOMEM;
+goto out;
+
 for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++ )
 l2e_write(l2t + i,
   l2e_from_pfn(l3e_get_pfn(*pl3e) +
@@ -5566,7 +5569,8 @@ int modify_xen_mappings(unsigned long s, unsigned long e, 
unsigned int nf)
 /* PSE: shatter the superpage and try again. */
 l1t = alloc_xen_pagetable();
 if ( !l1t )
-return -ENOMEM;
+goto out;
+
 for ( i = 0; i < L1_PAGETABLE_ENTRIES; i++ )
 l1e_write(&l1t[i],
   l1e_from_pfn(l2e_get_pfn(*pl2e) + i,
@@ -5699,7 +5703,10 @@ int modify_xen_mappings(unsigned long s, unsigned long 
e, unsigned int nf)
 flush_area(NULL, FLUSH_TLB_GLOBAL);
 
 #undef FLAGS_MASK
-return 0;
+rc = 0;
+
+ out:
+return rc;
 }
 
 #undef flush_area
-- 
2.24.1.AMZN

[PATCH v6 09/15] efi: use new page table APIs in copy_mapping

[Hongyan Xia]

From: Wei Liu 

After inspection ARM doesn't have alloc_xen_pagetable so this function
is x86 only, which means it is safe for us to change.

Signed-off-by: Wei Liu 
---
 xen/common/efi/boot.c | 14 +-
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
index a6f84c945a..8e96ef8de2 100644
--- a/xen/common/efi/boot.c
+++ b/xen/common/efi/boot.c
@@ -1454,16 +1454,20 @@ static __init void copy_mapping(unsigned long mfn, 
unsigned long end,
 continue;
 if ( !(l4e_get_flags(l4e) & _PAGE_PRESENT) )
 {
-l3dst = alloc_xen_pagetable();
-BUG_ON(!l3dst);
+mfn_t l3mfn = alloc_xen_pagetable_new();
+
+BUG_ON(mfn_eq(l3mfn, INVALID_MFN));
+l3dst = map_domain_page(l3mfn);
 clear_page(l3dst);
 efi_l4_pgtable[l4_table_offset(mfn << PAGE_SHIFT)] =
-l4e_from_paddr(virt_to_maddr(l3dst), __PAGE_HYPERVISOR);
+l4e_from_mfn(l3mfn, __PAGE_HYPERVISOR);
 }
 else
-l3dst = l4e_to_l3e(l4e);
-l3src = l4e_to_l3e(idle_pg_table[l4_table_offset(va)]);
+l3dst = map_l3t_from_l4e(l4e);
+l3src = map_l3t_from_l4e(idle_pg_table[l4_table_offset(va)]);
 l3dst[l3_table_offset(mfn << PAGE_SHIFT)] = l3src[l3_table_offset(va)];
+unmap_domain_page(l3src);
+unmap_domain_page(l3dst);
 }
 }
 
-- 
2.24.1.AMZN

[PATCH v6 05/15] x86/mm: switch to new APIs in modify_xen_mappings

[Hongyan Xia]

From: Wei Liu 

Page tables allocated in that function should be mapped and unmapped
now.

Note that pl2e now maybe mapped and unmapped in different iterations, so
we need to add clean-ups for that.

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
---
 xen/arch/x86/mm.c | 57 ++-
 1 file changed, 36 insertions(+), 21 deletions(-)

diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 2c14cdd83a..e8c16027d8 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -5508,7 +5508,7 @@ int modify_xen_mappings(unsigned long s, unsigned long e, 
unsigned int nf)
 {
 bool locking = system_state > SYS_STATE_boot;
 l3_pgentry_t *pl3e = NULL;
-l2_pgentry_t *pl2e;
+l2_pgentry_t *pl2e = NULL;
 l1_pgentry_t *pl1e;
 unsigned int  i;
 unsigned long v = s;
@@ -5524,6 +5524,7 @@ int modify_xen_mappings(unsigned long s, unsigned long e, 
unsigned int nf)
 while ( v < e )
 {
 /* Clean up mappings mapped in the previous iteration. */
+UNMAP_DOMAIN_PAGE(pl2e);
 UNMAP_DOMAIN_PAGE(pl3e);
 
 pl3e = virt_to_xen_l3e(v);
@@ -5541,6 +5542,7 @@ int modify_xen_mappings(unsigned long s, unsigned long e, 
unsigned int nf)
 if ( l3e_get_flags(*pl3e) & _PAGE_PSE )
 {
 l2_pgentry_t *l2t;
+mfn_t l2mfn;
 
 if ( l2_table_offset(v) == 0 &&
  l1_table_offset(v) == 0 &&
@@ -5557,35 +5559,38 @@ int modify_xen_mappings(unsigned long s, unsigned long 
e, unsigned int nf)
 }
 
 /* PAGE1GB: shatter the superpage and fall through. */
-l2t = alloc_xen_pagetable();
-if ( !l2t )
+l2mfn = alloc_xen_pagetable_new();
+if ( mfn_eq(l2mfn, INVALID_MFN) )
 goto out;
 
+l2t = map_domain_page(l2mfn);
 for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++ )
 l2e_write(l2t + i,
   l2e_from_pfn(l3e_get_pfn(*pl3e) +
(i << PAGETABLE_ORDER),
l3e_get_flags(*pl3e)));
+UNMAP_DOMAIN_PAGE(l2t);
+
 if ( locking )
 spin_lock(&map_pgdir_lock);
 if ( (l3e_get_flags(*pl3e) & _PAGE_PRESENT) &&
  (l3e_get_flags(*pl3e) & _PAGE_PSE) )
 {
-l3e_write_atomic(pl3e, l3e_from_mfn(virt_to_mfn(l2t),
-__PAGE_HYPERVISOR));
-l2t = NULL;
+l3e_write_atomic(pl3e,
+ l3e_from_mfn(l2mfn, __PAGE_HYPERVISOR));
+l2mfn = INVALID_MFN;
 }
 if ( locking )
 spin_unlock(&map_pgdir_lock);
-if ( l2t )
-free_xen_pagetable(l2t);
+
+free_xen_pagetable_new(l2mfn);
 }
 
 /*
  * The L3 entry has been verified to be present, and we've dealt with
  * 1G pages as well, so the L2 table cannot require allocation.
  */
-pl2e = l3e_to_l2e(*pl3e) + l2_table_offset(v);
+pl2e = map_l2t_from_l3e(*pl3e) + l2_table_offset(v);
 
 if ( !(l2e_get_flags(*pl2e) & _PAGE_PRESENT) )
 {
@@ -5613,41 +5618,45 @@ int modify_xen_mappings(unsigned long s, unsigned long 
e, unsigned int nf)
 else
 {
 l1_pgentry_t *l1t;
-
 /* PSE: shatter the superpage and try again. */
-l1t = alloc_xen_pagetable();
-if ( !l1t )
+mfn_t l1mfn = alloc_xen_pagetable_new();
+
+if ( mfn_eq(l1mfn, INVALID_MFN) )
 goto out;
 
+l1t = map_domain_page(l1mfn);
 for ( i = 0; i < L1_PAGETABLE_ENTRIES; i++ )
 l1e_write(&l1t[i],
   l1e_from_pfn(l2e_get_pfn(*pl2e) + i,
l2e_get_flags(*pl2e) & ~_PAGE_PSE));
+UNMAP_DOMAIN_PAGE(l1t);
+
 if ( locking )
 spin_lock(&map_pgdir_lock);
 if ( (l2e_get_flags(*pl2e) & _PAGE_PRESENT) &&
  (l2e_get_flags(*pl2e) & _PAGE_PSE) )
 {
-l2e_write_atomic(pl2e, l2e_from_mfn(virt_to_mfn(l1t),
+l2e_write_atomic(pl2e, l2e_from_mfn(l1mfn,
 __PAGE_HYPERVISOR));
-l1t = NULL;
+l1mfn = INVALID_MFN;
 }
 if ( locking )
 spin_unlock(&map_pgdir_lock);
-if ( l1t )
-free_xen_pagetable(l1t);
+
+free_xen_pagetable_new(l1mfn);
 }
 }
 else
 {
 l1_pgentry_t nl1e, *l1t;
+mfn_t l1mfn;
 
 /*
  * Ordinary 4kB mapping: The L

[PATCH v6 08/15] x86_64/mm: switch to new APIs in setup_m2p_table

[Hongyan Xia]

From: Wei Liu 

Also reduce the scope of l2_ro_mpt as it is only used inside the loop,
and remove two lines of l2_ro_mpt check which serve no purpose.

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
---
 xen/arch/x86/x86_64/mm.c | 24 +---
 1 file changed, 13 insertions(+), 11 deletions(-)

diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c
index 3cf699d27b..12e9dc6eb2 100644
--- a/xen/arch/x86/x86_64/mm.c
+++ b/xen/arch/x86/x86_64/mm.c
@@ -379,13 +379,13 @@ static int setup_m2p_table(struct mem_hotadd_info *info)
 {
 unsigned long i, va, smap, emap;
 unsigned int n;
-l2_pgentry_t *l2_ro_mpt = NULL;
 l3_pgentry_t *l3_ro_mpt = NULL;
 int ret = 0;
 
 ASSERT(l4e_get_flags(idle_pg_table[l4_table_offset(RO_MPT_VIRT_START)])
 & _PAGE_PRESENT);
-l3_ro_mpt = l4e_to_l3e(idle_pg_table[l4_table_offset(RO_MPT_VIRT_START)]);
+l3_ro_mpt = map_l3t_from_l4e(
+idle_pg_table[l4_table_offset(RO_MPT_VIRT_START)]);
 
 smap = (info->spfn & (~((1UL << (L2_PAGETABLE_SHIFT - 3)) -1)));
 emap = ((info->epfn + ((1UL << (L2_PAGETABLE_SHIFT - 3)) - 1 )) &
@@ -424,6 +424,7 @@ static int setup_m2p_table(struct mem_hotadd_info *info)
 break;
 if ( n < CNT )
 {
+l2_pgentry_t *l2_ro_mpt;
 mfn_t mfn = alloc_hotadd_mfn(info);
 
 ret = map_pages_to_xen(
@@ -440,30 +441,30 @@ static int setup_m2p_table(struct mem_hotadd_info *info)
   _PAGE_PSE));
 if ( l3e_get_flags(l3_ro_mpt[l3_table_offset(va)]) &
   _PAGE_PRESENT )
-l2_ro_mpt = l3e_to_l2e(l3_ro_mpt[l3_table_offset(va)]) +
-  l2_table_offset(va);
+l2_ro_mpt = map_l2t_from_l3e(l3_ro_mpt[l3_table_offset(va)]);
 else
 {
-l2_ro_mpt = alloc_xen_pagetable();
-if ( !l2_ro_mpt )
+mfn_t l2_ro_mpt_mfn = alloc_xen_pagetable_new();
+
+if ( mfn_eq(l2_ro_mpt_mfn, INVALID_MFN) )
 {
 ret = -ENOMEM;
 goto error;
 }
 
+l2_ro_mpt = map_domain_page(l2_ro_mpt_mfn);
 clear_page(l2_ro_mpt);
 l3e_write(&l3_ro_mpt[l3_table_offset(va)],
-  l3e_from_paddr(__pa(l2_ro_mpt),
- __PAGE_HYPERVISOR_RO | _PAGE_USER));
-l2_ro_mpt += l2_table_offset(va);
+  l3e_from_mfn(l2_ro_mpt_mfn,
+   __PAGE_HYPERVISOR_RO | _PAGE_USER));
 }
+l2_ro_mpt += l2_table_offset(va);
 
 /* NB. Cannot be GLOBAL: guest user mode should not see it. */
 l2e_write(l2_ro_mpt, l2e_from_mfn(mfn,
/*_PAGE_GLOBAL|*/_PAGE_PSE|_PAGE_USER|_PAGE_PRESENT));
+unmap_domain_page(l2_ro_mpt);
 }
-if ( !((unsigned long)l2_ro_mpt & ~PAGE_MASK) )
-l2_ro_mpt = NULL;
 i += ( 1UL << (L2_PAGETABLE_SHIFT - 3));
 }
 #undef CNT
@@ -471,6 +472,7 @@ static int setup_m2p_table(struct mem_hotadd_info *info)
 
 ret = setup_compat_m2p_table(info);
 error:
+UNMAP_DOMAIN_PAGE(l3_ro_mpt);
 return ret;
 }
 
-- 
2.24.1.AMZN

[PATCH v6 04/15] x86/mm: switch to new APIs in map_pages_to_xen

[Hongyan Xia]

From: Wei Liu 

Page tables allocated in that function should be mapped and unmapped
now.

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
---
 xen/arch/x86/mm.c | 60 ---
 1 file changed, 36 insertions(+), 24 deletions(-)

diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 3328d887c4..2c14cdd83a 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -5151,7 +5151,7 @@ int map_pages_to_xen(
 }
 else
 {
-l2_pgentry_t *l2t = l3e_to_l2e(ol3e);
+l2_pgentry_t *l2t = map_l2t_from_l3e(ol3e);
 
 for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++ )
 {
@@ -5163,10 +5163,11 @@ int map_pages_to_xen(
 else
 {
 unsigned int j;
-const l1_pgentry_t *l1t = l2e_to_l1e(ol2e);
+const l1_pgentry_t *l1t = map_l1t_from_l2e(ol2e);
 
 for ( j = 0; j < L1_PAGETABLE_ENTRIES; j++ )
 flush_flags(l1e_get_flags(l1t[j]));
+unmap_domain_page(l1t);
 }
 }
 flush_area(virt, flush_flags);
@@ -5175,9 +5176,10 @@ int map_pages_to_xen(
 ol2e = l2t[i];
 if ( (l2e_get_flags(ol2e) & _PAGE_PRESENT) &&
  !(l2e_get_flags(ol2e) & _PAGE_PSE) )
-free_xen_pagetable(l2e_to_l1e(ol2e));
+free_xen_pagetable_new(l2e_get_mfn(ol2e));
 }
-free_xen_pagetable(l2t);
+unmap_domain_page(l2t);
+free_xen_pagetable_new(l3e_get_mfn(ol3e));
 }
 }
 
@@ -5194,6 +5196,7 @@ int map_pages_to_xen(
 unsigned int flush_flags =
 FLUSH_TLB | FLUSH_ORDER(2 * PAGETABLE_ORDER);
 l2_pgentry_t *l2t;
+mfn_t l2mfn;
 
 /* Skip this PTE if there is no change. */
 if ( ((l3e_get_pfn(ol3e) & ~(L2_PAGETABLE_ENTRIES *
@@ -5215,15 +5218,17 @@ int map_pages_to_xen(
 continue;
 }
 
-l2t = alloc_xen_pagetable();
-if ( l2t == NULL )
+l2mfn = alloc_xen_pagetable_new();
+if ( mfn_eq(l2mfn, INVALID_MFN) )
 goto out;
 
+l2t = map_domain_page(l2mfn);
 for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++ )
 l2e_write(l2t + i,
   l2e_from_pfn(l3e_get_pfn(ol3e) +
(i << PAGETABLE_ORDER),
l3e_get_flags(ol3e)));
+UNMAP_DOMAIN_PAGE(l2t);
 
 if ( l3e_get_flags(ol3e) & _PAGE_GLOBAL )
 flush_flags |= FLUSH_TLB_GLOBAL;
@@ -5233,15 +5238,15 @@ int map_pages_to_xen(
 if ( (l3e_get_flags(*pl3e) & _PAGE_PRESENT) &&
  (l3e_get_flags(*pl3e) & _PAGE_PSE) )
 {
-l3e_write_atomic(pl3e, l3e_from_mfn(virt_to_mfn(l2t),
-__PAGE_HYPERVISOR));
-l2t = NULL;
+l3e_write_atomic(pl3e,
+ l3e_from_mfn(l2mfn, __PAGE_HYPERVISOR));
+l2mfn = INVALID_MFN;
 }
 if ( locking )
 spin_unlock(&map_pgdir_lock);
 flush_area(virt, flush_flags);
-if ( l2t )
-free_xen_pagetable(l2t);
+
+free_xen_pagetable_new(l2mfn);
 }
 
 pl2e = virt_to_xen_l2e(virt);
@@ -5269,12 +5274,13 @@ int map_pages_to_xen(
 }
 else
 {
-l1_pgentry_t *l1t = l2e_to_l1e(ol2e);
+l1_pgentry_t *l1t = map_l1t_from_l2e(ol2e);
 
 for ( i = 0; i < L1_PAGETABLE_ENTRIES; i++ )
 flush_flags(l1e_get_flags(l1t[i]));
 flush_area(virt, flush_flags);
-free_xen_pagetable(l1t);
+unmap_domain_page(l1t);
+free_xen_pagetable_new(l2e_get_mfn(ol2e));
 }
 }
 
@@ -5300,6 +5306,7 @@ int map_pages_to_xen(
 unsigned int flush_flags =
 FLUSH_TLB | FLUSH_ORDER(PAGETABLE_ORDER);
 l1_pgentry_t *l1t;
+mfn_t l1mfn;
 
 /* Skip this PTE if there is no change. */
 if ( (((l2e_get_pfn(*pl2e) & ~(L1_PAGETABLE_ENTRIES - 1)) +
@@ -5319,14 +5326,16 @@ int map_pages_to_xen(
 goto check_l3;
 }
 
-l1t = alloc_xen_pagetable();
-if ( l1t == NULL )
+l1mfn = alloc_xen_pagetable_new();
+i

[PATCH v6 00/15] switch to domheap for Xen page tables

[Hongyan Xia]

From: Hongyan Xia 

This series rewrites all the remaining functions and finally makes the
switch from xenheap to domheap for Xen page tables, so that they no
longer need to rely on the direct map, which is a big step towards
removing the direct map.

This series depends on the following two mini-series:
https://lists.xenproject.org/archives/html/xen-devel/2020-04/msg00730.html
https://lists.xenproject.org/archives/html/xen-devel/2020-04/msg00940.html

Since v1, 9 patches have already been merged. Apart from the first 2
patches in this series, the rest have not seen comments since v1 so they
only contain modifications from my side compared with v1.

---
Changed in v6:
- drop the patches that have already been merged.
- rebase and cleanup.
- rewrite map_pages_to_xen() and modify_xen_mappings() in a way that
  does not require an end_of_loop goto label.

Hongyan Xia (2):
  x86/mm: drop old page table APIs
  x86: switch to use domheap page for page tables

Wei Liu (13):
  x86/mm: map_pages_to_xen would better have one exit path
  x86/mm: make sure there is one exit path for modify_xen_mappings
  x86/mm: rewrite virt_to_xen_l*e
  x86/mm: switch to new APIs in map_pages_to_xen
  x86/mm: switch to new APIs in modify_xen_mappings
  x86_64/mm: introduce pl2e in paging_init
  x86_64/mm: switch to new APIs in paging_init
  x86_64/mm: switch to new APIs in setup_m2p_table
  efi: use new page table APIs in copy_mapping
  efi: switch to new APIs in EFI code
  x86/smpboot: clone_mapping should have one exit path
  x86/smpboot: switch pl*e to use new APIs in clone_mapping
  x86/mm: drop _new suffix for page table APIs

 xen/arch/x86/domain_page.c |  11 +-
 xen/arch/x86/efi/runtime.h |  10 +-
 xen/arch/x86/mm.c  | 262 +++--
 xen/arch/x86/setup.c   |   4 +-
 xen/arch/x86/smpboot.c |  80 +++
 xen/arch/x86/x86_64/mm.c   |  81 +++-
 xen/common/efi/boot.c  |  60 ++---
 xen/common/efi/efi.h   |   3 +-
 xen/common/efi/runtime.c   |   8 +-
 xen/common/vmap.c  |   1 +
 xen/include/asm-x86/mm.h   |   6 +-
 xen/include/asm-x86/page.h |  13 +-
 12 files changed, 339 insertions(+), 200 deletions(-)

-- 
2.24.1.AMZN

[PATCH v6 06/15] x86_64/mm: introduce pl2e in paging_init

[Hongyan Xia]

From: Wei Liu 

Introduce pl2e so that we can use l2_ro_mpt to point to the page table
itself.

No functional change.

Signed-off-by: Wei Liu 
---
 xen/arch/x86/x86_64/mm.c | 16 +---
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c
index 6557255494..35f9de4ad4 100644
--- a/xen/arch/x86/x86_64/mm.c
+++ b/xen/arch/x86/x86_64/mm.c
@@ -479,7 +479,7 @@ void __init paging_init(void)
 unsigned long i, mpt_size, va;
 unsigned int n, memflags;
 l3_pgentry_t *l3_ro_mpt;
-l2_pgentry_t *l2_ro_mpt = NULL;
+l2_pgentry_t *pl2e = NULL, *l2_ro_mpt = NULL;
 struct page_info *l1_pg;
 
 /*
@@ -529,7 +529,7 @@ void __init paging_init(void)
 (L2_PAGETABLE_SHIFT - 3 + PAGE_SHIFT)));
 
 if ( cpu_has_page1gb &&
- !((unsigned long)l2_ro_mpt & ~PAGE_MASK) &&
+ !((unsigned long)pl2e & ~PAGE_MASK) &&
  (mpt_size >> L3_PAGETABLE_SHIFT) > (i >> PAGETABLE_ORDER) )
 {
 unsigned int k, holes;
@@ -589,7 +589,7 @@ void __init paging_init(void)
 memset((void *)(RDWR_MPT_VIRT_START + (i << L2_PAGETABLE_SHIFT)),
0xFF, 1UL << L2_PAGETABLE_SHIFT);
 }
-if ( !((unsigned long)l2_ro_mpt & ~PAGE_MASK) )
+if ( !((unsigned long)pl2e & ~PAGE_MASK) )
 {
 if ( (l2_ro_mpt = alloc_xen_pagetable()) == NULL )
 goto nomem;
@@ -597,13 +597,14 @@ void __init paging_init(void)
 l3e_write(&l3_ro_mpt[l3_table_offset(va)],
   l3e_from_paddr(__pa(l2_ro_mpt),
  __PAGE_HYPERVISOR_RO | _PAGE_USER));
+pl2e = l2_ro_mpt;
 ASSERT(!l2_table_offset(va));
 }
 /* NB. Cannot be GLOBAL: guest user mode should not see it. */
 if ( l1_pg )
-l2e_write(l2_ro_mpt, l2e_from_page(
+l2e_write(pl2e, l2e_from_page(
 l1_pg, /*_PAGE_GLOBAL|*/_PAGE_PSE|_PAGE_USER|_PAGE_PRESENT));
-l2_ro_mpt++;
+pl2e++;
 }
 #undef CNT
 #undef MFN
@@ -613,6 +614,7 @@ void __init paging_init(void)
 goto nomem;
 compat_idle_pg_table_l2 = l2_ro_mpt;
 clear_page(l2_ro_mpt);
+pl2e = l2_ro_mpt;
 /* Allocate and map the compatibility mode machine-to-phys table. */
 mpt_size = (mpt_size >> 1) + (1UL << (L2_PAGETABLE_SHIFT - 1));
 if ( mpt_size > RDWR_COMPAT_MPT_VIRT_END - RDWR_COMPAT_MPT_VIRT_START )
@@ -625,7 +627,7 @@ void __init paging_init(void)
  sizeof(*compat_machine_to_phys_mapping))
 BUILD_BUG_ON((sizeof(*frame_table) & ~sizeof(*frame_table)) % \
  sizeof(*compat_machine_to_phys_mapping));
-for ( i = 0; i < (mpt_size >> L2_PAGETABLE_SHIFT); i++, l2_ro_mpt++ )
+for ( i = 0; i < (mpt_size >> L2_PAGETABLE_SHIFT); i++, pl2e++ )
 {
 memflags = MEMF_node(phys_to_nid(i <<
 (L2_PAGETABLE_SHIFT - 2 + PAGE_SHIFT)));
@@ -647,7 +649,7 @@ void __init paging_init(void)
 (i << L2_PAGETABLE_SHIFT)),
0xFF, 1UL << L2_PAGETABLE_SHIFT);
 /* NB. Cannot be GLOBAL as the ptes get copied into per-VM space. */
-l2e_write(l2_ro_mpt, l2e_from_page(l1_pg, _PAGE_PSE|_PAGE_PRESENT));
+l2e_write(pl2e, l2e_from_page(l1_pg, _PAGE_PSE|_PAGE_PRESENT));
 }
 #undef CNT
 #undef MFN
-- 
2.24.1.AMZN

[PATCH v6 03/15] x86/mm: rewrite virt_to_xen_l*e

[Hongyan Xia]

From: Wei Liu 

Rewrite those functions to use the new APIs. Modify its callers to unmap
the pointer returned.

Note that the change of virt_to_xen_l1e() also requires vmap_to_mfn() to
unmap the page, which requires domain_page.h header in vmap.

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
---
 xen/arch/x86/domain_page.c | 11 +++--
 xen/arch/x86/mm.c  | 85 +++---
 xen/common/vmap.c  |  1 +
 xen/include/asm-x86/page.h |  8 +++-
 4 files changed, 76 insertions(+), 29 deletions(-)

diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c
index dd32712d2f..3a244bb500 100644
--- a/xen/arch/x86/domain_page.c
+++ b/xen/arch/x86/domain_page.c
@@ -333,21 +333,24 @@ void unmap_domain_page_global(const void *ptr)
 mfn_t domain_page_map_to_mfn(const void *ptr)
 {
 unsigned long va = (unsigned long)ptr;
-const l1_pgentry_t *pl1e;
+l1_pgentry_t l1e;
 
 if ( va >= DIRECTMAP_VIRT_START )
 return _mfn(virt_to_mfn(ptr));
 
 if ( va >= VMAP_VIRT_START && va < VMAP_VIRT_END )
 {
-pl1e = virt_to_xen_l1e(va);
+const l1_pgentry_t *pl1e = virt_to_xen_l1e(va);
+
 BUG_ON(!pl1e);
+l1e = *pl1e;
+unmap_domain_page(pl1e);
 }
 else
 {
 ASSERT(va >= MAPCACHE_VIRT_START && va < MAPCACHE_VIRT_END);
-pl1e = &__linear_l1_table[l1_linear_offset(va)];
+l1e = __linear_l1_table[l1_linear_offset(va)];
 }
 
-return l1e_get_mfn(*pl1e);
+return l1e_get_mfn(l1e);
 }
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 13a34a8d57..3328d887c4 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -4955,6 +4955,10 @@ void free_xen_pagetable_new(mfn_t mfn)
 
 static DEFINE_SPINLOCK(map_pgdir_lock);
 
+/*
+ * For virt_to_xen_lXe() functions, they take a virtual address and return a
+ * pointer to Xen's LX entry. Caller needs to unmap the pointer.
+ */
 static l3_pgentry_t *virt_to_xen_l3e(unsigned long v)
 {
 l4_pgentry_t *pl4e;
@@ -4963,33 +4967,36 @@ static l3_pgentry_t *virt_to_xen_l3e(unsigned long v)
 if ( !(l4e_get_flags(*pl4e) & _PAGE_PRESENT) )
 {
 bool locking = system_state > SYS_STATE_boot;
-l3_pgentry_t *l3t = alloc_xen_pagetable();
+l3_pgentry_t *l3t;
+mfn_t l3mfn = alloc_xen_pagetable_new();
 
-if ( !l3t )
+if ( mfn_eq(l3mfn, INVALID_MFN) )
 return NULL;
+l3t = map_domain_page(l3mfn);
 clear_page(l3t);
+UNMAP_DOMAIN_PAGE(l3t);
 if ( locking )
 spin_lock(&map_pgdir_lock);
 if ( !(l4e_get_flags(*pl4e) & _PAGE_PRESENT) )
 {
-l4_pgentry_t l4e = l4e_from_paddr(__pa(l3t), __PAGE_HYPERVISOR);
+l4_pgentry_t l4e = l4e_from_mfn(l3mfn, __PAGE_HYPERVISOR);
 
 l4e_write(pl4e, l4e);
 efi_update_l4_pgtable(l4_table_offset(v), l4e);
-l3t = NULL;
+l3mfn = INVALID_MFN;
 }
 if ( locking )
 spin_unlock(&map_pgdir_lock);
-if ( l3t )
-free_xen_pagetable(l3t);
+free_xen_pagetable_new(l3mfn);
 }
 
-return l4e_to_l3e(*pl4e) + l3_table_offset(v);
+return map_l3t_from_l4e(*pl4e) + l3_table_offset(v);
 }
 
 static l2_pgentry_t *virt_to_xen_l2e(unsigned long v)
 {
 l3_pgentry_t *pl3e;
+l2_pgentry_t *pl2e;
 
 pl3e = virt_to_xen_l3e(v);
 if ( !pl3e )
@@ -4998,31 +5005,40 @@ static l2_pgentry_t *virt_to_xen_l2e(unsigned long v)
 if ( !(l3e_get_flags(*pl3e) & _PAGE_PRESENT) )
 {
 bool locking = system_state > SYS_STATE_boot;
-l2_pgentry_t *l2t = alloc_xen_pagetable();
+l2_pgentry_t *l2t;
+mfn_t l2mfn = alloc_xen_pagetable_new();
 
-if ( !l2t )
+if ( mfn_eq(l2mfn, INVALID_MFN) )
+{
+UNMAP_DOMAIN_PAGE(pl3e);
 return NULL;
+}
+l2t = map_domain_page(l2mfn);
 clear_page(l2t);
+UNMAP_DOMAIN_PAGE(l2t);
 if ( locking )
 spin_lock(&map_pgdir_lock);
 if ( !(l3e_get_flags(*pl3e) & _PAGE_PRESENT) )
 {
-l3e_write(pl3e, l3e_from_paddr(__pa(l2t), __PAGE_HYPERVISOR));
-l2t = NULL;
+l3e_write(pl3e, l3e_from_mfn(l2mfn, __PAGE_HYPERVISOR));
+l2mfn = INVALID_MFN;
 }
 if ( locking )
 spin_unlock(&map_pgdir_lock);
-if ( l2t )
-free_xen_pagetable(l2t);
+free_xen_pagetable_new(l2mfn);
 }
 
 BUG_ON(l3e_get_flags(*pl3e) & _PAGE_PSE);
-return l3e_to_l2e(*pl3e) + l2_table_offset(v);
+pl2e = map_l2t_from_l3e(*pl3e) + l2_table_offset(v);
+unmap_domain_page(pl3e);
+
+return pl2e;
 }
 
 l1_pgentry_t *virt_to_xen_l1e(unsigned long v)
 {
 l2_pgentry_t *pl2e;
+l1_pgentry_t *pl1e;
 
 pl2e = virt_to_xen_l2e(v);
 if ( !pl2e )
@@ -5031,26 +5047,34 @@ l1_pgentry_t *virt_to_xen_l1e(unsigned long v)
 if ( !(l2e_get_flags(*pl2e) & _PAGE_PRESENT

[PATCH v6 07/15] x86_64/mm: switch to new APIs in paging_init

[Hongyan Xia]

From: Wei Liu 

Map and unmap pages instead of relying on the direct map.

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
---
 xen/arch/x86/x86_64/mm.c | 43 
 1 file changed, 30 insertions(+), 13 deletions(-)

diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c
index 35f9de4ad4..3cf699d27b 100644
--- a/xen/arch/x86/x86_64/mm.c
+++ b/xen/arch/x86/x86_64/mm.c
@@ -478,9 +478,10 @@ void __init paging_init(void)
 {
 unsigned long i, mpt_size, va;
 unsigned int n, memflags;
-l3_pgentry_t *l3_ro_mpt;
+l3_pgentry_t *l3_ro_mpt = NULL;
 l2_pgentry_t *pl2e = NULL, *l2_ro_mpt = NULL;
 struct page_info *l1_pg;
+mfn_t l3_ro_mpt_mfn, l2_ro_mpt_mfn;
 
 /*
  * We setup the L3s for 1:1 mapping if host support memory hotplug
@@ -493,22 +494,28 @@ void __init paging_init(void)
 if ( !(l4e_get_flags(idle_pg_table[l4_table_offset(va)]) &
   _PAGE_PRESENT) )
 {
-l3_pgentry_t *pl3t = alloc_xen_pagetable();
+l3_pgentry_t *pl3t;
+mfn_t l3mfn = alloc_xen_pagetable_new();
 
-if ( !pl3t )
+if ( mfn_eq(l3mfn, INVALID_MFN) )
 goto nomem;
+
+pl3t = map_domain_page(l3mfn);
 clear_page(pl3t);
 l4e_write(&idle_pg_table[l4_table_offset(va)],
-  l4e_from_paddr(__pa(pl3t), __PAGE_HYPERVISOR_RW));
+  l4e_from_mfn(l3mfn, __PAGE_HYPERVISOR_RW));
+unmap_domain_page(pl3t);
 }
 }
 
 /* Create user-accessible L2 directory to map the MPT for guests. */
-if ( (l3_ro_mpt = alloc_xen_pagetable()) == NULL )
+l3_ro_mpt_mfn = alloc_xen_pagetable_new();
+if ( mfn_eq(l3_ro_mpt_mfn, INVALID_MFN) )
 goto nomem;
+l3_ro_mpt = map_domain_page(l3_ro_mpt_mfn);
 clear_page(l3_ro_mpt);
 l4e_write(&idle_pg_table[l4_table_offset(RO_MPT_VIRT_START)],
-  l4e_from_paddr(__pa(l3_ro_mpt), __PAGE_HYPERVISOR_RO | 
_PAGE_USER));
+  l4e_from_mfn(l3_ro_mpt_mfn, __PAGE_HYPERVISOR_RO | _PAGE_USER));
 
 /*
  * Allocate and map the machine-to-phys table.
@@ -591,12 +598,17 @@ void __init paging_init(void)
 }
 if ( !((unsigned long)pl2e & ~PAGE_MASK) )
 {
-if ( (l2_ro_mpt = alloc_xen_pagetable()) == NULL )
+UNMAP_DOMAIN_PAGE(l2_ro_mpt);
+
+l2_ro_mpt_mfn = alloc_xen_pagetable_new();
+if ( mfn_eq(l2_ro_mpt_mfn, INVALID_MFN) )
 goto nomem;
+
+l2_ro_mpt = map_domain_page(l2_ro_mpt_mfn);
 clear_page(l2_ro_mpt);
 l3e_write(&l3_ro_mpt[l3_table_offset(va)],
-  l3e_from_paddr(__pa(l2_ro_mpt),
- __PAGE_HYPERVISOR_RO | _PAGE_USER));
+  l3e_from_mfn(l2_ro_mpt_mfn,
+   __PAGE_HYPERVISOR_RO | _PAGE_USER));
 pl2e = l2_ro_mpt;
 ASSERT(!l2_table_offset(va));
 }
@@ -608,13 +620,16 @@ void __init paging_init(void)
 }
 #undef CNT
 #undef MFN
+UNMAP_DOMAIN_PAGE(l2_ro_mpt);
+UNMAP_DOMAIN_PAGE(l3_ro_mpt);
 
 /* Create user-accessible L2 directory to map the MPT for compat guests. */
-if ( (l2_ro_mpt = alloc_xen_pagetable()) == NULL )
+l2_ro_mpt_mfn = alloc_xen_pagetable_new();
+if ( mfn_eq(l2_ro_mpt_mfn, INVALID_MFN) )
 goto nomem;
-compat_idle_pg_table_l2 = l2_ro_mpt;
-clear_page(l2_ro_mpt);
-pl2e = l2_ro_mpt;
+compat_idle_pg_table_l2 = map_domain_page_global(l2_ro_mpt_mfn);
+clear_page(compat_idle_pg_table_l2);
+pl2e = compat_idle_pg_table_l2;
 /* Allocate and map the compatibility mode machine-to-phys table. */
 mpt_size = (mpt_size >> 1) + (1UL << (L2_PAGETABLE_SHIFT - 1));
 if ( mpt_size > RDWR_COMPAT_MPT_VIRT_END - RDWR_COMPAT_MPT_VIRT_START )
@@ -662,6 +677,8 @@ void __init paging_init(void)
 return;
 
  nomem:
+UNMAP_DOMAIN_PAGE(l2_ro_mpt);
+UNMAP_DOMAIN_PAGE(l3_ro_mpt);
 panic("Not enough memory for m2p table\n");
 }
 
-- 
2.24.1.AMZN

[PATCH v6 01/15] x86/mm: map_pages_to_xen would better have one exit path

[Hongyan Xia]

From: Wei Liu 

We will soon rewrite the function to handle dynamically mapping and
unmapping of page tables. Since dynamic mappings may map and unmap pages
in different iterations of the while loop, we need to lift pl3e out of
the loop.

No functional change.

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 

---
Changed since v4:
- drop the end_of_loop goto label.

Changed since v3:
- remove asserts on rc since rc never gets changed to anything else.
- reword commit message.
---
 xen/arch/x86/mm.c | 20 +---
 1 file changed, 13 insertions(+), 7 deletions(-)

diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 355c50ff91..0d7f1f1265 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -5073,9 +5073,11 @@ int map_pages_to_xen(
 unsigned int flags)
 {
 bool locking = system_state > SYS_STATE_boot;
+l3_pgentry_t *pl3e, ol3e;
 l2_pgentry_t *pl2e, ol2e;
 l1_pgentry_t *pl1e, ol1e;
 unsigned int  i;
+int rc = -ENOMEM;
 
 #define flush_flags(oldf) do { \
 unsigned int o_ = (oldf);  \
@@ -5093,10 +5095,11 @@ int map_pages_to_xen(
 
 while ( nr_mfns != 0 )
 {
-l3_pgentry_t ol3e, *pl3e = virt_to_xen_l3e(virt);
+pl3e = virt_to_xen_l3e(virt);
 
 if ( !pl3e )
-return -ENOMEM;
+goto out;
+
 ol3e = *pl3e;
 
 if ( cpu_has_page1gb &&
@@ -5186,7 +5189,7 @@ int map_pages_to_xen(
 
 l2t = alloc_xen_pagetable();
 if ( l2t == NULL )
-return -ENOMEM;
+goto out;
 
 for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++ )
 l2e_write(l2t + i,
@@ -5215,7 +5218,7 @@ int map_pages_to_xen(
 
 pl2e = virt_to_xen_l2e(virt);
 if ( !pl2e )
-return -ENOMEM;
+goto out;
 
 if ( virt >> PAGE_SHIFT) | mfn_x(mfn)) &
((1u << PAGETABLE_ORDER) - 1)) == 0) &&
@@ -5259,7 +5262,7 @@ int map_pages_to_xen(
 {
 pl1e = virt_to_xen_l1e(virt);
 if ( pl1e == NULL )
-return -ENOMEM;
+goto out;
 }
 else if ( l2e_get_flags(*pl2e) & _PAGE_PSE )
 {
@@ -5287,7 +5290,7 @@ int map_pages_to_xen(
 
 l1t = alloc_xen_pagetable();
 if ( l1t == NULL )
-return -ENOMEM;
+goto out;
 
 for ( i = 0; i < L1_PAGETABLE_ENTRIES; i++ )
 l1e_write(&l1t[i],
@@ -5433,7 +5436,10 @@ int map_pages_to_xen(
 
 #undef flush_flags
 
-return 0;
+rc = 0;
+
+ out:
+return rc;
 }
 
 int populate_pt_range(unsigned long virt, unsigned long nr_mfns)
-- 
2.24.1.AMZN

Re: [XEN PATCH v5 04/16] xen/build: have the root Makefile generates the CFLAGS

[Anthony PERARD]

On Fri, Apr 24, 2020 at 02:30:56PM +0100, Anthony PERARD wrote:
> On Fri, Apr 24, 2020 at 03:01:32PM +0200, Jan Beulich wrote:
> > While committing this, in my pre-push build test I noticed that
> > presumably an earlier change of yours has caused
> > 
> > Makefile:103: include/config/auto.conf: No such file or directory
> > Makefile:106: include/config/auto.conf.cmd: No such file or directory
> > 
> > for a build in a completely fresh tree.
> 
> Are those presumably "warning" an issue? Are the files still missing
> after make has run? Didn't the build managed to build xen.gz?
> There is maybe a line saying that make will re-execute.
> 
> I've seen those error before, on older version of make. But it's just
> make complaining before even trying to update/create those files. But it
> just create those files and start over.
> 
> Also, that would be patch "build: include include/config/auto.conf in
> main Makefile" which introduce this behavior.

I'll prepare a patch to use "-include" instead of simply "include". That
will silent the warnings and nothing else should change. Linux's Kbuild
doesn't have this issue because we have to run `make menuconfig` or
equivalent which also generates the two auto.conf* files, which I didn't
notice until now.

-- 
Anthony PERARD

[xen-unstable test] 149771: tolerable trouble: fail/pass/starved - PUSHED

[osstest service owner]

flight 149771 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/149771/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-rtds 18 guest-localmigrate/x10   fail  like 149741
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 149741
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 149741
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop fail like 149741
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 149752
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 149752
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 149752
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 149752
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 149752
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 149752
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-coresched-i386-xl  2 hosts-allocate   starved  n/a
 test-amd64-coresched-amd64-xl  2 hosts-allocate   starved  n/a

version targeted for testing:
 xen  aa14feb6723d3bb15a884533ade1cd9732792145
baseline version:
 xen  a62c6fe05c4ae905b7d4cb0ca946508b7f96d522

Last test of basis   149752  2020-04-23 10:22:34 Z1 days
Testing same since   149771  2020-04-23 22:06:56 Z0 days1 attempts


People who touched revisions under test:
  Anthony PERARD 
  George Dunlap 
  Jan Beulich 
  Nick Rosbroo

[examine test] 149782: tolerable trouble: pass/starved

[osstest service owner]

flight 149782 examine real [real]
http://logs.test-lab.xenproject.org/osstest/logs/149782/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 examine-albana1   4 memdisk-try-append  fail blocked in 149768
 examine-albana0   4 memdisk-try-append  fail blocked in 149768
 examine-elbling0  2 hosts-allocate   starved  n/a
 examine-rochester02 hosts-allocate   starved  n/a
 examine-debina0   2 hosts-allocate   starved  n/a
 examine-godello0  2 hosts-allocate   starved  n/a

baseline version:
 flight   149768

jobs:
 examine-albana0  pass
 examine-albana1  pass
 examine-arndale-bluewaterpass
 examine-cubietruck-braquepass
 examine-chardonnay0  pass
 examine-chardonnay1  pass
 examine-debina0  starved 
 examine-debina1  pass
 examine-elbling0 starved 
 examine-elbling1 pass
 examine-fiano0   pass
 examine-fiano1   pass
 examine-cubietruck-gleizes   pass
 examine-godello0 starved 
 examine-godello1 pass
 examine-huxelrebe0   pass
 examine-huxelrebe1   pass
 examine-italia0  pass
 examine-arndale-lakeside pass
 examine-laxton0  pass
 examine-laxton1  pass
 examine-arndale-metrocentre  pass
 examine-cubietruck-metzinger pass
 examine-cubietruck-picasso   pass
 examine-pinot0   pass
 examine-pinot1   pass
 examine-rimava1  pass
 examine-rochester0   starved 
 examine-rochester1   pass
 examine-arndale-westfieldpass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Push not applicable.

[PATCH] docs/designs: re-work the xenstore migration document...

[Paul Durrant]

From: Paul Durrant 

... to specify a separate migration stream that will also be suitable for
live update.

The original scope of the document was to support non-cooperative migration
of guests [1] but, since then, live update of xenstored has been brought into
scope. Thus it makes more sense to define a separate image format for
serializing xenstore state that is suitable for both purposes.

The document has been limited to specifying a new image format. The mechanism
for acquiring the image for live update or migration is not covered as that
is more appropriately dealt with by a patch to docs/misc/xenstore.txt. It is
also expected that, when the first implementation of live update or migration
making use of this specification is committed, that the document is moved from
docs/designs into docs/specs.

[1] See 
https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=docs/designs/non-cooperative-migration.md

Signed-off-by: Paul Durrant 
---
Juergen Gross 
Andrew Cooper 
George Dunlap 
Ian Jackson 
Jan Beulich 
Julien Grall 
Stefano Stabellini 
Wei Liu 
---
 docs/designs/xenstore-migration.md | 472 +++--
 1 file changed, 309 insertions(+), 163 deletions(-)

diff --git a/docs/designs/xenstore-migration.md 
b/docs/designs/xenstore-migration.md
index 6ab351e8fe..c96bad48eb 100644
--- a/docs/designs/xenstore-migration.md
+++ b/docs/designs/xenstore-migration.md
@@ -3,254 +3,400 @@
 ## Background
 
 The design for *Non-Cooperative Migration of Guests*[1] explains that extra
-save records are required in the migrations stream to allow a guest running
-PV drivers to be migrated without its co-operation. Moreover the save
-records must include details of registered xenstore watches as well as
-content; information that cannot currently be recovered from `xenstored`,
-and hence some extension to the xenstore protocol[2] will also be required.
-
-The *libxenlight Domain Image Format* specification[3] already defines a
-record type `EMULATOR_XENSTORE_DATA` but this is not suitable for
-transferring xenstore data pertaining to the domain directly as it is
-specified such that keys are relative to the path
-`/local/domain/$dm_domid/device-model/$domid`. Thus it is necessary to
-define at least one new save record type.
+save records are required in the migrations stream to allow a guest running PV
+drivers to be migrated without its co-operation. Moreover the save records must
+include details of registered xenstore watches as well ascontent; information
+that cannot currently be recovered from `xenstored`, and hence some extension
+to the xenstored implementations will also be required.
+
+As a similar set of data is needed for transferring xenstore data from one
+instance to another when live updating xenstored this document proposes an
+image format for a 'migration stream' suitable for both purposes.
 
 ## Proposal
 
-### New Save Record
+The image format consists of a _header_ followed by 1 or more _records_. Each
+record consists of a type and length field, followed by any data mandated by
+the record type. At minimum there will be a single record of type `END`
+(defined below).
 
-A new mandatory record type should be defined within the libxenlight Domain
-Image Format:
+### Header
 
-`0x0007: DOMAIN_XENSTORE_DATA`
+The header identifies the stream as a `xenstore` stream, including the version
+of the specification that it complies with.
 
-An arbitrary number of these records may be present in the migration
-stream and may appear in any order. The format of each record should be as
-follows:
+All fields in this header must be in _big-endian_ byte order, regardless of
+the setting of the endianness bit.
 
 
 ```
 0   1   2   3   4   5   6   7octet
 +---+---+---+---+---+---+---+---+
-| type  | record specific data  |
-+---+   |
-...
-+---+
+| ident |
++---+---|
+| version   | flags |
++---+---+
 ```
 
-where type is one of the following values
 
+| Field | Description   |
+|---|---|
+| `ident`   | 0x78656e73746f7265 ('xenstore' in ASCII)  |
+|   |   |
+| `version` | 0x0001 (the version of the specification) |
+|   |   |
+| `flags`   | 0 (LSB): Endianness: 0 = little, 1 = big  |
+|   |   |
+|   | 1-31: Reserved (must be zero) |
 
-| Field  | Description

Re: [XEN PATCH v5 04/16] xen/build: have the root Makefile generates the CFLAGS

[Anthony PERARD]

On Fri, Apr 24, 2020 at 03:01:32PM +0200, Jan Beulich wrote:
> While committing this, in my pre-push build test I noticed that
> presumably an earlier change of yours has caused
> 
> Makefile:103: include/config/auto.conf: No such file or directory
> Makefile:106: include/config/auto.conf.cmd: No such file or directory
> 
> for a build in a completely fresh tree.

Are those presumably "warning" an issue? Are the files still missing
after make has run? Didn't the build managed to build xen.gz?
There is maybe a line saying that make will re-execute.

I've seen those error before, on older version of make. But it's just
make complaining before even trying to update/create those files. But it
just create those files and start over.

Also, that would be patch "build: include include/config/auto.conf in
main Makefile" which introduce this behavior.

Thanks,

-- 
Anthony PERARD

Re: [PATCH v17 2/2] xen/tools: VM forking toolstack side

[Jan Beulich]

On 23.04.2020 17:30, Tamas K Lengyel wrote:
> Add necessary bits to implement "xl fork-vm" commands. The command allows the
> user to specify how to launch the device model allowing for a late-launch 
> model
> in which the user can execute the fork without the device model and decide to
> only later launch it.
> 
> Signed-off-by: Tamas K Lengyel 

As before this will be left for the tool stack maintainers to
ack and commit.

Jan

Re: [XEN PATCH v5 04/16] xen/build: have the root Makefile generates the CFLAGS

[Jan Beulich]

On 21.04.2020 18:11, Anthony PERARD wrote:
> Instead of generating the CFLAGS in Rules.mk everytime we enter a new
> subdirectory, we are going to generate most of them a single time, and
> export the result in the environment so that Rules.mk can use it.  The
> only flags left to be generated are the ones that depend on the
> targets, but the variable $(c_flags) takes care of that.
> 
> Arch specific CFLAGS are generated by a new file "arch/*/arch.mk"
> which is included by the root Makefile.
> 
> We export the *FLAGS via the environment variables XEN_*FLAGS because
> Rules.mk still includes Config.mk and would add duplicated flags to
> CFLAGS.
> 
> When running Rules.mk in the root directory (xen/), the variable
> `root-make-done' is set, so `need-config' will remain undef and so the
> root Makefile will not generate the cflags again.
> 
> We can't use CFLAGS in subdirectories to add flags to particular
> targets, instead start to use CFLAGS-y. Idem for AFLAGS.
> So there are two different CFLAGS-y, the one in xen/Makefile (and
> arch.mk), and the one in subdirs that Rules.mk is going to use.
> We can't add to XEN_CFLAGS because it is exported, so making change to
> it might be propagated to subdirectory which isn't intended.
> 
> Some style change are introduced in this patch:
> when LDFLAGS_DIRECT is included in LDFLAGS
> use of CFLAGS-$(CONFIG_INDIRECT_THUNK) instead of ifeq().
> 
> The LTO change hasn't been tested properly, as LTO is marked as
> broken.
> 
> Signed-off-by: Anthony PERARD 

While committing this, in my pre-push build test I noticed that
presumably an earlier change of yours has caused

Makefile:103: include/config/auto.conf: No such file or directory
Makefile:106: include/config/auto.conf.cmd: No such file or directory

for a build in a completely fresh tree.

Jan

Re: [PATCH v17 1/2] mem_sharing: fix sharability check during fork reset

[Roger Pau Monné]

On Fri, Apr 24, 2020 at 06:18:24AM -0600, Tamas K Lengyel wrote:
> On Fri, Apr 24, 2020 at 3:44 AM Roger Pau Monné  wrote:
> >
> > On Thu, Apr 23, 2020 at 08:30:06AM -0700, Tamas K Lengyel wrote:
> > > When resetting a VM fork we ought to only remove pages that were 
> > > allocated for
> > > the fork during it's execution and the contents copied over from the 
> > > parent.
> > > This can be determined if the page is sharable as special pages used by 
> > > the
> > > fork for other purposes will not pass this test. Unfortunately during the 
> > > fork
> > > reset loop we only partially check whether that's the case. A page's type 
> > > may
> > > indicate it is sharable (pass p2m_is_sharable) but that's not a sufficient
> > > check by itself. All checks that are normally performed before a page is
> > > converted to the sharable type need to be performed to avoid removing 
> > > pages
> > > from the p2m that may be used for other purposes. For example, currently 
> > > the
> > > reset loop also removes the vcpu info pages from the p2m, potentially 
> > > putting
> > > the guest into infinite page-fault loops.
> > >
> > > Signed-off-by: Tamas K Lengyel 
> >
> > Reviewed-by: Roger Pau Monné 
> 
> Thanks!
> 
> >
> > I've been looking however and I'm not able to spot where you copy the
> > shared_info data, which I think it's also quite critical for the
> > domain, as it contains the info about event channels (when using L2).
> > Also for HVM forks the shared info should be mapped at the same gfn as
> > the parent, or else the child trying to access it will trigger an EPT
> > fault that won't be able to populate such gfn, because the shared_info
> > on the parent is already shared between Xen and the parent.
> 
> Pages that cause an EPT fault but can't be made shared get a new page
> allocated for them and copied in mem_sharing_fork_page. There are many
> pages like that, mostly due to QEMU mapping them and thus holding an
> extra reference. That said, shared_info is manually copied during
> forking in copy_special_pages, at the end of the function you will
> find:
> 
> old_mfn = _mfn(virt_to_mfn(d->shared_info));
> new_mfn = _mfn(virt_to_mfn(cd->shared_info));
> 
> copy_domain_page(new_mfn, old_mfn);

Oh, sorry for the noise, I somehow missed it.

Roger.

[PATCH AUTOSEL 4.4 5/8] xen/xenbus: ensure xenbus_map_ring_valloc() returns proper grant status

[Sasha Levin]

From: Juergen Gross 

[ Upstream commit 6b51fd3f65a22e3d1471b18a1d56247e246edd46 ]

xenbus_map_ring_valloc() maps a ring page and returns the status of the
used grant (0 meaning success).

There are Xen hypervisors which might return the value 1 for the status
of a failed grant mapping due to a bug. Some callers of
xenbus_map_ring_valloc() test for errors by testing the returned status
to be less than zero, resulting in no error detected and crashing later
due to a not available ring page.

Set the return value of xenbus_map_ring_valloc() to GNTST_general_error
in case the grant status reported by Xen is greater than zero.

This is part of XSA-316.

Signed-off-by: Juergen Gross 
Reviewed-by: Wei Liu 
Link: https://lore.kernel.org/r/20200326080358.1018-1-jgr...@suse.com
Signed-off-by: Juergen Gross 
Signed-off-by: Sasha Levin 
---
 drivers/xen/xenbus/xenbus_client.c | 9 -
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/xen/xenbus/xenbus_client.c 
b/drivers/xen/xenbus/xenbus_client.c
index 056da6ee1a357..df27cefb2fa35 100644
--- a/drivers/xen/xenbus/xenbus_client.c
+++ b/drivers/xen/xenbus/xenbus_client.c
@@ -469,7 +469,14 @@ EXPORT_SYMBOL_GPL(xenbus_free_evtchn);
 int xenbus_map_ring_valloc(struct xenbus_device *dev, grant_ref_t *gnt_refs,
   unsigned int nr_grefs, void **vaddr)
 {
-   return ring_ops->map(dev, gnt_refs, nr_grefs, vaddr);
+   int err;
+
+   err = ring_ops->map(dev, gnt_refs, nr_grefs, vaddr);
+   /* Some hypervisors are buggy and can return 1. */
+   if (err > 0)
+   err = GNTST_general_error;
+
+   return err;
 }
 EXPORT_SYMBOL_GPL(xenbus_map_ring_valloc);
 
-- 
2.20.1

[PATCH AUTOSEL 4.19 11/18] xen/xenbus: ensure xenbus_map_ring_valloc() returns proper grant status

[Sasha Levin]

From: Juergen Gross 

[ Upstream commit 6b51fd3f65a22e3d1471b18a1d56247e246edd46 ]

xenbus_map_ring_valloc() maps a ring page and returns the status of the
used grant (0 meaning success).

There are Xen hypervisors which might return the value 1 for the status
of a failed grant mapping due to a bug. Some callers of
xenbus_map_ring_valloc() test for errors by testing the returned status
to be less than zero, resulting in no error detected and crashing later
due to a not available ring page.

Set the return value of xenbus_map_ring_valloc() to GNTST_general_error
in case the grant status reported by Xen is greater than zero.

This is part of XSA-316.

Signed-off-by: Juergen Gross 
Reviewed-by: Wei Liu 
Link: https://lore.kernel.org/r/20200326080358.1018-1-jgr...@suse.com
Signed-off-by: Juergen Gross 
Signed-off-by: Sasha Levin 
---
 drivers/xen/xenbus/xenbus_client.c | 9 -
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/xen/xenbus/xenbus_client.c 
b/drivers/xen/xenbus/xenbus_client.c
index a1c17000129ba..e94a61eaeceb0 100644
--- a/drivers/xen/xenbus/xenbus_client.c
+++ b/drivers/xen/xenbus/xenbus_client.c
@@ -450,7 +450,14 @@ EXPORT_SYMBOL_GPL(xenbus_free_evtchn);
 int xenbus_map_ring_valloc(struct xenbus_device *dev, grant_ref_t *gnt_refs,
   unsigned int nr_grefs, void **vaddr)
 {
-   return ring_ops->map(dev, gnt_refs, nr_grefs, vaddr);
+   int err;
+
+   err = ring_ops->map(dev, gnt_refs, nr_grefs, vaddr);
+   /* Some hypervisors are buggy and can return 1. */
+   if (err > 0)
+   err = GNTST_general_error;
+
+   return err;
 }
 EXPORT_SYMBOL_GPL(xenbus_map_ring_valloc);
 
-- 
2.20.1

[PATCH AUTOSEL 4.14 10/21] xen/xenbus: ensure xenbus_map_ring_valloc() returns proper grant status

[Sasha Levin]

From: Juergen Gross 

[ Upstream commit 6b51fd3f65a22e3d1471b18a1d56247e246edd46 ]

xenbus_map_ring_valloc() maps a ring page and returns the status of the
used grant (0 meaning success).

There are Xen hypervisors which might return the value 1 for the status
of a failed grant mapping due to a bug. Some callers of
xenbus_map_ring_valloc() test for errors by testing the returned status
to be less than zero, resulting in no error detected and crashing later
due to a not available ring page.

Set the return value of xenbus_map_ring_valloc() to GNTST_general_error
in case the grant status reported by Xen is greater than zero.

This is part of XSA-316.

Signed-off-by: Juergen Gross 
Reviewed-by: Wei Liu 
Link: https://lore.kernel.org/r/20200326080358.1018-1-jgr...@suse.com
Signed-off-by: Juergen Gross 
Signed-off-by: Sasha Levin 
---
 drivers/xen/xenbus/xenbus_client.c | 9 -
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/xen/xenbus/xenbus_client.c 
b/drivers/xen/xenbus/xenbus_client.c
index a1c17000129ba..e94a61eaeceb0 100644
--- a/drivers/xen/xenbus/xenbus_client.c
+++ b/drivers/xen/xenbus/xenbus_client.c
@@ -450,7 +450,14 @@ EXPORT_SYMBOL_GPL(xenbus_free_evtchn);
 int xenbus_map_ring_valloc(struct xenbus_device *dev, grant_ref_t *gnt_refs,
   unsigned int nr_grefs, void **vaddr)
 {
-   return ring_ops->map(dev, gnt_refs, nr_grefs, vaddr);
+   int err;
+
+   err = ring_ops->map(dev, gnt_refs, nr_grefs, vaddr);
+   /* Some hypervisors are buggy and can return 1. */
+   if (err > 0)
+   err = GNTST_general_error;
+
+   return err;
 }
 EXPORT_SYMBOL_GPL(xenbus_map_ring_valloc);
 
-- 
2.20.1

[PATCH AUTOSEL 4.9 09/13] xen/xenbus: ensure xenbus_map_ring_valloc() returns proper grant status

[Sasha Levin]

From: Juergen Gross 

[ Upstream commit 6b51fd3f65a22e3d1471b18a1d56247e246edd46 ]

xenbus_map_ring_valloc() maps a ring page and returns the status of the
used grant (0 meaning success).

There are Xen hypervisors which might return the value 1 for the status
of a failed grant mapping due to a bug. Some callers of
xenbus_map_ring_valloc() test for errors by testing the returned status
to be less than zero, resulting in no error detected and crashing later
due to a not available ring page.

Set the return value of xenbus_map_ring_valloc() to GNTST_general_error
in case the grant status reported by Xen is greater than zero.

This is part of XSA-316.

Signed-off-by: Juergen Gross 
Reviewed-by: Wei Liu 
Link: https://lore.kernel.org/r/20200326080358.1018-1-jgr...@suse.com
Signed-off-by: Juergen Gross 
Signed-off-by: Sasha Levin 
---
 drivers/xen/xenbus/xenbus_client.c | 9 -
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/xen/xenbus/xenbus_client.c 
b/drivers/xen/xenbus/xenbus_client.c
index 056da6ee1a357..df27cefb2fa35 100644
--- a/drivers/xen/xenbus/xenbus_client.c
+++ b/drivers/xen/xenbus/xenbus_client.c
@@ -469,7 +469,14 @@ EXPORT_SYMBOL_GPL(xenbus_free_evtchn);
 int xenbus_map_ring_valloc(struct xenbus_device *dev, grant_ref_t *gnt_refs,
   unsigned int nr_grefs, void **vaddr)
 {
-   return ring_ops->map(dev, gnt_refs, nr_grefs, vaddr);
+   int err;
+
+   err = ring_ops->map(dev, gnt_refs, nr_grefs, vaddr);
+   /* Some hypervisors are buggy and can return 1. */
+   if (err > 0)
+   err = GNTST_general_error;
+
+   return err;
 }
 EXPORT_SYMBOL_GPL(xenbus_map_ring_valloc);
 
-- 
2.20.1

[PATCH AUTOSEL 5.6 21/38] xen/xenbus: ensure xenbus_map_ring_valloc() returns proper grant status

[Sasha Levin]

From: Juergen Gross 

[ Upstream commit 6b51fd3f65a22e3d1471b18a1d56247e246edd46 ]

xenbus_map_ring_valloc() maps a ring page and returns the status of the
used grant (0 meaning success).

There are Xen hypervisors which might return the value 1 for the status
of a failed grant mapping due to a bug. Some callers of
xenbus_map_ring_valloc() test for errors by testing the returned status
to be less than zero, resulting in no error detected and crashing later
due to a not available ring page.

Set the return value of xenbus_map_ring_valloc() to GNTST_general_error
in case the grant status reported by Xen is greater than zero.

This is part of XSA-316.

Signed-off-by: Juergen Gross 
Reviewed-by: Wei Liu 
Link: https://lore.kernel.org/r/20200326080358.1018-1-jgr...@suse.com
Signed-off-by: Juergen Gross 
Signed-off-by: Sasha Levin 
---
 drivers/xen/xenbus/xenbus_client.c | 9 -
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/xen/xenbus/xenbus_client.c 
b/drivers/xen/xenbus/xenbus_client.c
index e17ca81561713..a38292ef79f6d 100644
--- a/drivers/xen/xenbus/xenbus_client.c
+++ b/drivers/xen/xenbus/xenbus_client.c
@@ -448,7 +448,14 @@ EXPORT_SYMBOL_GPL(xenbus_free_evtchn);
 int xenbus_map_ring_valloc(struct xenbus_device *dev, grant_ref_t *gnt_refs,
   unsigned int nr_grefs, void **vaddr)
 {
-   return ring_ops->map(dev, gnt_refs, nr_grefs, vaddr);
+   int err;
+
+   err = ring_ops->map(dev, gnt_refs, nr_grefs, vaddr);
+   /* Some hypervisors are buggy and can return 1. */
+   if (err > 0)
+   err = GNTST_general_error;
+
+   return err;
 }
 EXPORT_SYMBOL_GPL(xenbus_map_ring_valloc);
 
-- 
2.20.1

[PATCH AUTOSEL 5.4 16/26] xen/xenbus: ensure xenbus_map_ring_valloc() returns proper grant status

[Sasha Levin]

From: Juergen Gross 

[ Upstream commit 6b51fd3f65a22e3d1471b18a1d56247e246edd46 ]

xenbus_map_ring_valloc() maps a ring page and returns the status of the
used grant (0 meaning success).

There are Xen hypervisors which might return the value 1 for the status
of a failed grant mapping due to a bug. Some callers of
xenbus_map_ring_valloc() test for errors by testing the returned status
to be less than zero, resulting in no error detected and crashing later
due to a not available ring page.

Set the return value of xenbus_map_ring_valloc() to GNTST_general_error
in case the grant status reported by Xen is greater than zero.

This is part of XSA-316.

Signed-off-by: Juergen Gross 
Reviewed-by: Wei Liu 
Link: https://lore.kernel.org/r/20200326080358.1018-1-jgr...@suse.com
Signed-off-by: Juergen Gross 
Signed-off-by: Sasha Levin 
---
 drivers/xen/xenbus/xenbus_client.c | 9 -
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/xen/xenbus/xenbus_client.c 
b/drivers/xen/xenbus/xenbus_client.c
index e17ca81561713..a38292ef79f6d 100644
--- a/drivers/xen/xenbus/xenbus_client.c
+++ b/drivers/xen/xenbus/xenbus_client.c
@@ -448,7 +448,14 @@ EXPORT_SYMBOL_GPL(xenbus_free_evtchn);
 int xenbus_map_ring_valloc(struct xenbus_device *dev, grant_ref_t *gnt_refs,
   unsigned int nr_grefs, void **vaddr)
 {
-   return ring_ops->map(dev, gnt_refs, nr_grefs, vaddr);
+   int err;
+
+   err = ring_ops->map(dev, gnt_refs, nr_grefs, vaddr);
+   /* Some hypervisors are buggy and can return 1. */
+   if (err > 0)
+   err = GNTST_general_error;
+
+   return err;
 }
 EXPORT_SYMBOL_GPL(xenbus_map_ring_valloc);
 
-- 
2.20.1

Re: [PATCH v17 1/2] mem_sharing: fix sharability check during fork reset

[Tamas K Lengyel]

On Fri, Apr 24, 2020 at 3:44 AM Roger Pau Monné  wrote:
>
> On Thu, Apr 23, 2020 at 08:30:06AM -0700, Tamas K Lengyel wrote:
> > When resetting a VM fork we ought to only remove pages that were allocated 
> > for
> > the fork during it's execution and the contents copied over from the parent.
> > This can be determined if the page is sharable as special pages used by the
> > fork for other purposes will not pass this test. Unfortunately during the 
> > fork
> > reset loop we only partially check whether that's the case. A page's type 
> > may
> > indicate it is sharable (pass p2m_is_sharable) but that's not a sufficient
> > check by itself. All checks that are normally performed before a page is
> > converted to the sharable type need to be performed to avoid removing pages
> > from the p2m that may be used for other purposes. For example, currently the
> > reset loop also removes the vcpu info pages from the p2m, potentially 
> > putting
> > the guest into infinite page-fault loops.
> >
> > Signed-off-by: Tamas K Lengyel 
>
> Reviewed-by: Roger Pau Monné 

Thanks!

>
> I've been looking however and I'm not able to spot where you copy the
> shared_info data, which I think it's also quite critical for the
> domain, as it contains the info about event channels (when using L2).
> Also for HVM forks the shared info should be mapped at the same gfn as
> the parent, or else the child trying to access it will trigger an EPT
> fault that won't be able to populate such gfn, because the shared_info
> on the parent is already shared between Xen and the parent.

Pages that cause an EPT fault but can't be made shared get a new page
allocated for them and copied in mem_sharing_fork_page. There are many
pages like that, mostly due to QEMU mapping them and thus holding an
extra reference. That said, shared_info is manually copied during
forking in copy_special_pages, at the end of the function you will
find:

old_mfn = _mfn(virt_to_mfn(d->shared_info));
new_mfn = _mfn(virt_to_mfn(cd->shared_info));

copy_domain_page(new_mfn, old_mfn);

Cheers,
Tamas

Re: Golang Xen packages and the golang packaging system

[George Dunlap]

> On Apr 24, 2020, at 5:04 AM, Nick Rosbrook  wrote:
> 
> On Thu, Apr 23, 2020 at 1:22 PM George Dunlap  
> wrote:
>> 
>> 
>>> On Apr 23, 2020, at 12:49 PM, George Dunlap  
>>> wrote:
>>> 
>>> 
>>> 
 On Apr 23, 2020, at 12:27 PM, Ian Jackson  wrote:
 
 Ian Jackson writes ("Re: Golang Xen packages and the golang packaging 
 system"):
> This is quite unpleasant.  In particular, it makes a git tree out of
> output files.  What will we do when someone sends us patches to the
> bindings ?
 
 Also, anyone who redistributes your proposed golang package is
 violating our licence unless they ship a copy of xen.git[1] too, since
 the golang package is not source code.
 
 [1] Technically, a copy of the relevant parts will do.
>>> 
>>> The “relevant parts” would primarily be gengotypes.py, right?  Oh, and I 
>>> guess libxl_test.idl and friends.  libxl_test.idl isn’t included in the 
>>> distribution either.
>>> 
>>> I’m not an expert in the golang build system, but they generally seem to be 
>>> trying to keep the functionality simple (which of course, means if you want 
>>> to do anything non-basic, it’s incredibly complicated or completely 
>>> impossible).
>>> 
>>> There’s a command, `go generate`, which we could use to run gengotypes.py 
>>> to generate the appropriate files.  But I’m not sure how to use that in a 
>>> practical way for this sort of package: it might end up that people wanting 
>>> to use the package would need to manually clone it, then manually run `go 
>>> generate` before manually building the package.
>>> 
>>> Checking in the generated files means that someone can simply add 
>>> `golang.xenproject.org/xenlight` as a dependency (perhaps with a specific 
>>> version tag, like v4.14), and everything Just Works.
>>> 
>>> Nick may have some ideas on how to use the golang build system more 
>>> effectively.
>> 
>> So, the following seems to work quite well actually:
>> 
>> mkdir vendor
>> ln -s vendor/golang.xenproject.org 
>> /usr/share/gocode/src/golang.xenproject.org
>> echo “golang.xenproject.org/xenlight” >> vendor/modules.txt
>> go build -mod=vendor
>> 
>> Using the above method, (say) redctl.git would build exactly the same on Xen 
>> 4.14 as on Xen 4.15 (assuming redctl wasn’t using anything not available in 
>> 4.14).
>> 
>> I’m inclined to say we should start with just telling people to do that, and 
>> look at doing something else if we discover that’s not suitable for some 
>> reason.
> 
> If it's not viable to create another repo for the xenlight package, I
> think we should should just initialize the go module, i.e. go.mod, at
> xen.git/tools/golang. The downside is that tags cannot be independent
> from the rest of xen.git, so users need to have `require  path>/xenlight@RELEASE-4.14.0` in their go.mod, but at least its `go
> get`-able. And, this does not fetch the entire git tree.
> 
> This would also mean that we actually track the generated code (which
> isn't really a big deal IMO, it's expected that people track their
> generated gRPC code, for example).

Yes, I was playing with this yesterday and it seems to work OK.

The thing I didn’t necessarily like about this was that suppose you had a 
public project that used the xenlight bindings, and you upgraded to Xen 4.15, 
but some of your users hadn’t.  If you updated this to RELEASE-4.15.0, then all 
your downstreams would stop working, even if you weren’t using any 
functionality specific to Xen 4.15.

But I suppose what that would really mean is that:
1) We should make sure that xenlight@RELEASE-$V works on > $V as well
2) Projects depending on the bindings should use the oldest version of the Xen 
bindings suitable for their use case.

Both of those are probably reasonable.

Another issue that happens with checking in generated code is that the idl 
changes and nobody re-generates the code.  We’d probably want an osstest check 
that would refuse to push from staging -> master if re-running the code 
generator produced a different output.  (But that has its own annoyances: it 
seems that different versions of python sort things in different orders, so I 
often have to throw away spurious changes to the generated files because our 
two versions of python seem to order some things differently.)

BTW the separate repo isn’t off the table.  But there were some things other 
Ian pointed out:

1. The GPL requires that you provide the “preferred form for modification” to 
all the code.  I’m not sure this has been adjudicated in court, but there’s a 
strong argument that *generated* code doesn’t match that criteria: that to 
satisfy the GPL you’d need to include libxl_types.idl, idl.py, gengotypes.py, 
and a Makefile suitable for tying them all together.  (Not that the generation 
needs to be run with `go build`, but that ideally the infrastructure would be 
there so that it *could* be run.)

2. Ian was concerned with how someone using the bindings would submit a patch 
upstream.  

Re: [XEN PATCH v5 04/16] xen/build: have the root Makefile generates the CFLAGS

[Jan Beulich]

On 24.04.2020 11:45, Anthony PERARD wrote:
> On Thu, Apr 23, 2020 at 06:40:33PM +0200, Jan Beulich wrote:
>> On 21.04.2020 18:11, Anthony PERARD wrote:
>>> --- a/xen/arch/x86/Rules.mk
>>> +++ b/xen/arch/x86/Rules.mk
> [..]
>>> +c_flags += $(object_label_flags) $(CFLAGS-stack-boundary)
>>> +a_flags += $(object_label_flags) $(CFLAGS-stack-boundary)
>>
>> Why are you also adding these to a_flags? It probably doesn't hurt,
>> but I'd prefer if it was removed (could be done while committing if
>> all acks arrive and other other need for av6 arises) if there's no
>> clear need.
> 
> Those flags are already in a_flags (or previously AFLAGS). I've tried to
> be careful to avoid changing the list of *flags in an already
> complicated enough patch. I would like to keep this patch that way.
> 
> If the -D__OBJECT_LABEL__ and the stack-bondary flags aren't needed in
> a_flags, it would be better to remove them in a separated patch, with
> some explanation on why they are removed.

Ah, fair point - previously we had

# Most CFLAGS are safe for assembly files:
#  -std=gnu{89,99} gets confused by #-prefixed end-of-line comments
#  -flto makes no sense and annoys clang
AFLAGS += $(filter-out -std=gnu% -flto,$(CFLAGS))

and hence implicitly inherited all kinds of inapplicable flags.
I'm fine with this (and probably other things) getting taken care
of later then.

> What is av6?

I was missing a blank: "a v6". Also s/other other/no other".

Jan

Re: [PATCH 1/6] x86_64/mm: map and unmap page tables in cleanup_frame_table

[Jan Beulich]

On 24.04.2020 11:02, Julien Grall wrote:
> On 17/04/2020 10:52, Hongyan Xia wrote:
>> @@ -763,10 +763,10 @@ static void cleanup_frame_table(struct mem_hotadd_info 
>> *info)
>>   continue;
>>   }
>>   -    ASSERT(l1e_get_flags(l2e_to_l1e(l2e)[l1_table_offset(sva)]) &
>> -    _PAGE_PRESENT);
>> - sva = (sva & ~((1UL << PAGE_SHIFT) - 1)) +
>> -    (1UL << PAGE_SHIFT);
>> +    ASSERT(l1e_get_flags(l1e_from_l2e(l2e, l1_table_offset(sva))) &
>> +   _PAGE_PRESENT);
>> +
>> +    sva = (sva & ~((1UL << PAGE_SHIFT) - 1)) + (1UL << PAGE_SHIFT);
> 
> NIT: While you are modifying the indentation. Couldn't we use PAGE_MASK and 
> PAGE_SIZE here?

Oh, yes, this would be nice.

Jan

RE: [PATCH 0/3] xenoprof: XSA-313 follow-up

[Paul Durrant]

> -Original Message-
> From: Julien Grall 
> Sent: 24 April 2020 11:35
> To: p...@xen.org; 'Jan Beulich' ; 
> xen-devel@lists.xenproject.org
> Cc: 'Andrew Cooper' ; 'George Dunlap' 
> ; 'Ian
> Jackson' ; 'Stefano Stabellini' 
> ; 'Wei Liu'
> 
> Subject: Re: [PATCH 0/3] xenoprof: XSA-313 follow-up
> 
> Hi Paul,
> 
> On 20/04/2020 16:01, Paul Durrant wrote:
> >> -Original Message-
> >> From: Julien Grall 
> >> Sent: 20 April 2020 15:15
> >> To: p...@xen.org; 'Jan Beulich' ; 
> >> xen-devel@lists.xenproject.org
> >> Cc: 'Andrew Cooper' ; 'George Dunlap' 
> >> ; 'Ian
> >> Jackson' ; 'Stefano Stabellini' 
> >> ; 'Wei Liu'
> >> 
> >> Subject: Re: [PATCH 0/3] xenoprof: XSA-313 follow-up
> >>
> >> Hi Paul,
> >>
> >> On 15/04/2020 09:50, Paul Durrant wrote:
>  -Original Message-
>  From: Jan Beulich 
>  Sent: 15 April 2020 09:45
>  To: xen-devel@lists.xenproject.org
>  Cc: Andrew Cooper ; George Dunlap 
>  ; Ian
> >> Jackson
>  ; Julien Grall ; Stefano 
>  Stabellini
>  ; Wei Liu ; Paul Durrant 
>  
>  Subject: [PATCH 0/3] xenoprof: XSA-313 follow-up
> 
>  Patch 1 was considered to become part of the XSA, but it was then
>  decided against. The other two are a little bit of cleanup, albeit
>  there's certainly far more room for tidying. Yet then again Paul,
>  in his mail from Mar 13, was asking whether we shouldn't drop
>  xenoprof altogether, at which point cleaning up the code would be
>  wasted effort.
> 
> >>>
> >>> That's still my opinion. This is a large chunk of (only passively 
> >>> maintained) code which I think
> is
> >> of very limited value (since it relates to an old tool, and it only works 
> >> for PV domains).
> >>
> >> While there are no active user we are aware of, this is an example on
> >> how to implement a profiler backend with Xen. So I would agree with
> >> Andrew here.
> >>
> >> IIRC, the reason behind your request is it makes difficult for your
> >> xenheap work. Am I correct? If so, do you have a thread explaining the
> >> issues?
> >
> > After shared info and grant table, it is the only other occurrence of a 
> > xenheap page shared with a
> non-system domain. Also, it cannot be trivially replaced with an 'extra' 
> domheap page because its
> assignment changes. Thus a whole bunch of cleanup work that I was hoping to 
> do (largely in
> domain_relinquish_resources and free_domheap_pages) is either ruled out, or 
> would have to special-case
> this type of page.
> 
> My knowledge of xenoprof is very limited, so I might be wrong.
> 
>  From my understanding, a buffer can only be shared between two domains:
> - When in passive mode, the buffer will be shared with the primary
> profiler (always the hwdom per the check in xenoprof_op_init()).
> - When in active mode, the buffer will be shared with the domain
> requesting to be profiled.
> 
> Would it be possible to consider to have a separate buffer for the
> passive mode and active mode?

I think that may well work, yes.

> 
> > Also, I am unconvinced that PV guests are sufficiently common these days 
> > (apart from dom0) that
> profiling them is of any real use.
> 
> Even an HVM guest can't profile itself, I think it would be useful to
> have dom0 to profile an HVM guest. Are you suggesting this doesn't work?
> 

No. That may work for a PV dom0; I'll have to try it. So, yes, passive 
profiling may indeed still have value.

  Paul

Re: [PATCH 0/3] xenoprof: XSA-313 follow-up

[Julien Grall]

Hi Paul,

On 20/04/2020 16:01, Paul Durrant wrote:

-Original Message-
From: Julien Grall 
Sent: 20 April 2020 15:15
To: p...@xen.org; 'Jan Beulich' ; 
xen-devel@lists.xenproject.org
Cc: 'Andrew Cooper' ; 'George Dunlap' 
; 'Ian
Jackson' ; 'Stefano Stabellini' 
; 'Wei Liu'

Subject: Re: [PATCH 0/3] xenoprof: XSA-313 follow-up

Hi Paul,

On 15/04/2020 09:50, Paul Durrant wrote:

-Original Message-
From: Jan Beulich 
Sent: 15 April 2020 09:45
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper ; George Dunlap 
; Ian

Jackson

; Julien Grall ; Stefano Stabellini
; Wei Liu ; Paul Durrant 
Subject: [PATCH 0/3] xenoprof: XSA-313 follow-up

Patch 1 was considered to become part of the XSA, but it was then
decided against. The other two are a little bit of cleanup, albeit
there's certainly far more room for tidying. Yet then again Paul,
in his mail from Mar 13, was asking whether we shouldn't drop
xenoprof altogether, at which point cleaning up the code would be
wasted effort.



That's still my opinion. This is a large chunk of (only passively maintained) 
code which I think is

of very limited value (since it relates to an old tool, and it only works for 
PV domains).

While there are no active user we are aware of, this is an example on
how to implement a profiler backend with Xen. So I would agree with
Andrew here.

IIRC, the reason behind your request is it makes difficult for your
xenheap work. Am I correct? If so, do you have a thread explaining the
issues?


After shared info and grant table, it is the only other occurrence of a xenheap 
page shared with a non-system domain. Also, it cannot be trivially replaced 
with an 'extra' domheap page because its assignment changes. Thus a whole bunch 
of cleanup work that I was hoping to do (largely in domain_relinquish_resources 
and free_domheap_pages) is either ruled out, or would have to special-case this 
type of page.


My knowledge of xenoprof is very limited, so I might be wrong.

From my understanding, a buffer can only be shared between two domains:
   - When in passive mode, the buffer will be shared with the primary 
profiler (always the hwdom per the check in xenoprof_op_init()).
   - When in active mode, the buffer will be shared with the domain 
requesting to be profiled.


Would it be possible to consider to have a separate buffer for the 
passive mode and active mode?



Also, I am unconvinced that PV guests are sufficiently common these days (apart 
from dom0) that profiling them is of any real use.


Even an HVM guest can't profile itself, I think it would be useful to 
have dom0 to profile an HVM guest. Are you suggesting this doesn't work?


Cheers,

--
Julien Grall

Re: [PATCH] x86: drop cpu_has_ffxsr

[Andrew Cooper]

On 24/04/2020 11:24, Jan Beulich wrote:
> It's definition is bogus when it comes to Hygon CPUs, but since we don't
> use it anywhere drop it rather than correcting it.
>
> Signed-off-by: Jan Beulich 

I had wondered about that too.

Acked-by: Andrew Cooper 

[PATCH] x86: drop cpu_has_ffxsr

[Jan Beulich]

It's definition is bogus when it comes to Hygon CPUs, but since we don't
use it anywhere drop it rather than correcting it.

Signed-off-by: Jan Beulich 

--- a/xen/include/asm-x86/cpufeature.h
+++ b/xen/include/asm-x86/cpufeature.h
@@ -66,8 +66,6 @@
 
 /* CPUID level 0x8001.edx */
 #define cpu_has_nx  boot_cpu_has(X86_FEATURE_NX)
-#define cpu_has_ffxsr   ((boot_cpu_data.x86_vendor == X86_VENDOR_AMD) \
- && boot_cpu_has(X86_FEATURE_FFXSR))
 #define cpu_has_page1gb boot_cpu_has(X86_FEATURE_PAGE1GB)
 #define cpu_has_rdtscp  boot_cpu_has(X86_FEATURE_RDTSCP)
 #define cpu_has_3dnow_ext   boot_cpu_has(X86_FEATURE_3DNOWEXT)

[libvirt test] 149773: regressions - FAIL

[osstest service owner]

flight 149773 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/149773/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt   6 libvirt-buildfail REGR. vs. 146182
 build-i386-libvirt6 libvirt-buildfail REGR. vs. 146182
 build-arm64-libvirt   6 libvirt-buildfail REGR. vs. 146182
 build-armhf-libvirt   6 libvirt-buildfail REGR. vs. 146182

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a

version targeted for testing:
 libvirt  e2f1e2687fe5c976d1aecbae9350e761f1d1ddc0
baseline version:
 libvirt  a1cd25b919509be2645dbe6f952d5263e0d4e4e5

Last test of basis   146182  2020-01-17 06:00:23 Z   98 days
Failing since146211  2020-01-18 04:18:52 Z   97 days   90 attempts
Testing same since   149773  2020-04-24 04:18:45 Z0 days1 attempts


People who touched revisions under test:
  Andrea Bolognani 
  Arnaud Patard 
  Bjoern Walk 
  Boris Fiuczynski 
  Chen Hanxiao 
  Christian Borntraeger 
  Christian Ehrhardt 
  Christian Schoenebeck 
  Collin Walling 
  Cornelia Huck 
  Daniel Henrique Barboza 
  Daniel P. Berrangé 
  Daniel Veillard 
  Dario Faggioli 
  Erik Skultety 
  Gaurav Agrawal 
  Han Han 
  Jamie Strandboge 
  Jim Fehlig 
  Jiri Denemark 
  Jonathon Jongsma 
  Julio Faracco 
  Ján Tomko 
  Laine Stump 
  Leonid Bloch 
  Lin Ma 
  Marc-André Lureau 
  Marek Marczykowski-Górecki 
  Mark Asselstine 
  Mauro S. M. Rodrigues 
  Michal Privoznik 
  Nikolay Shirokovskiy 
  Pavel Hrdina 
  Pavel Mores 
  Peter Krempa 
  Philipp Hahn 
  Pino Toscano 
  Prathamesh Chavan 
  Rafael Fonseca 
  Richard W.M. Jones 
  Rikard Falkeborn 
  Ryan Moeller 
  Sahid Orentino Ferdjaoui 
  Sebastian Mitterle 
  Seeteena Thoufeek 
  Stefan Berger 
  Stefan Berger 
  Stefan Hajnoczi 
  Thomas Huth 
  Wu Qingliang 
  Yi Li 
  Your Name 
  Zhang Bo 
  zhenwei pi 
  Zhimin Feng 

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-arm64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  fail
 build-arm64-libvirt  fail
 build-armhf-libvirt  fail
 build-i386-libvirt   fail
 build-amd64-pvopspass
 build-arm64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmblocked 
 test-amd64-amd64-libvirt-xsm blocked 
 test-arm64-arm64-libvirt-xsm blocked 
 test-amd64-i386-libvirt-xsm  blocked 
 test-amd64-amd64-libvirt blocked 
 test-arm64-arm64-libvirt blocked 
 test-armhf-armhf-libvirt blocked 
 test-amd64-i386-libvirt  blocked 
 test-amd64-amd64-libvirt-pairblocked 
 test-amd64-i386-libvirt-pair blocked 
 test-arm64-arm64-libvi

Re: [XEN PATCH v5 04/16] xen/build: have the root Makefile generates the CFLAGS

[Anthony PERARD]

On Thu, Apr 23, 2020 at 06:40:33PM +0200, Jan Beulich wrote:
> On 21.04.2020 18:11, Anthony PERARD wrote:
> > Instead of generating the CFLAGS in Rules.mk everytime we enter a new
> > subdirectory, we are going to generate most of them a single time, and
> > export the result in the environment so that Rules.mk can use it.  The
> > only flags left to be generated are the ones that depend on the
> > targets, but the variable $(c_flags) takes care of that.
> > 
> > Arch specific CFLAGS are generated by a new file "arch/*/arch.mk"
> > which is included by the root Makefile.
> > 
> > We export the *FLAGS via the environment variables XEN_*FLAGS because
> > Rules.mk still includes Config.mk and would add duplicated flags to
> > CFLAGS.
> > 
> > When running Rules.mk in the root directory (xen/), the variable
> > `root-make-done' is set, so `need-config' will remain undef and so the
> > root Makefile will not generate the cflags again.
> > 
> > We can't use CFLAGS in subdirectories to add flags to particular
> > targets, instead start to use CFLAGS-y. Idem for AFLAGS.
> > So there are two different CFLAGS-y, the one in xen/Makefile (and
> > arch.mk), and the one in subdirs that Rules.mk is going to use.
> > We can't add to XEN_CFLAGS because it is exported, so making change to
> > it might be propagated to subdirectory which isn't intended.
> > 
> > Some style change are introduced in this patch:
> > when LDFLAGS_DIRECT is included in LDFLAGS
> > use of CFLAGS-$(CONFIG_INDIRECT_THUNK) instead of ifeq().
> > 
> > The LTO change hasn't been tested properly, as LTO is marked as
> > broken.
> > 
> > Signed-off-by: Anthony PERARD 
> 
> Reviewed-by: Jan Beulich 
> with one further question:
> 
> > --- a/xen/arch/x86/Rules.mk
> > +++ b/xen/arch/x86/Rules.mk
[..]
> > +c_flags += $(object_label_flags) $(CFLAGS-stack-boundary)
> > +a_flags += $(object_label_flags) $(CFLAGS-stack-boundary)
> 
> Why are you also adding these to a_flags? It probably doesn't hurt,
> but I'd prefer if it was removed (could be done while committing if
> all acks arrive and other other need for av6 arises) if there's no
> clear need.

Those flags are already in a_flags (or previously AFLAGS). I've tried to
be careful to avoid changing the list of *flags in an already
complicated enough patch. I would like to keep this patch that way.

If the -D__OBJECT_LABEL__ and the stack-bondary flags aren't needed in
a_flags, it would be better to remove them in a separated patch, with
some explanation on why they are removed.

What is av6?

Thanks,

-- 
Anthony PERARD

Re: [PATCH v17 1/2] mem_sharing: fix sharability check during fork reset

[Roger Pau Monné]

On Thu, Apr 23, 2020 at 08:30:06AM -0700, Tamas K Lengyel wrote:
> When resetting a VM fork we ought to only remove pages that were allocated for
> the fork during it's execution and the contents copied over from the parent.
> This can be determined if the page is sharable as special pages used by the
> fork for other purposes will not pass this test. Unfortunately during the fork
> reset loop we only partially check whether that's the case. A page's type may
> indicate it is sharable (pass p2m_is_sharable) but that's not a sufficient
> check by itself. All checks that are normally performed before a page is
> converted to the sharable type need to be performed to avoid removing pages
> from the p2m that may be used for other purposes. For example, currently the
> reset loop also removes the vcpu info pages from the p2m, potentially putting
> the guest into infinite page-fault loops.
> 
> Signed-off-by: Tamas K Lengyel 

Reviewed-by: Roger Pau Monné 

I've been looking however and I'm not able to spot where you copy the
shared_info data, which I think it's also quite critical for the
domain, as it contains the info about event channels (when using L2).
Also for HVM forks the shared info should be mapped at the same gfn as
the parent, or else the child trying to access it will trigger an EPT
fault that won't be able to populate such gfn, because the shared_info
on the parent is already shared between Xen and the parent.

Thanks, Roger.

Re: [XEN PATCH v5 11/16] xen/build: factorise generation of the linker scripts

[Julien Grall]

On 21/04/2020 17:12, Anthony PERARD wrote:

In Arm and X86 makefile, generating the linker script is the same, so
we can simply have both call the same macro.

We need to add *.lds files into extra-y so that Rules.mk can find the
.*.cmd dependency file and load it.

Change made to the command line:
- Use of $(CPP) instead of $(CC) -E
- Remove -Ui386.
   We don't compile Xen for i386 anymore, so that macro is never
   defined. Also it doesn't make sense on Arm.
- Use $(cpp_flags) which simply filter -Wa,% options from $(a_flags).

Signed-off-by: Anthony PERARD 


For the Arm bits:

Acked-by: Julien Grall 

Cheers,

--
Julien Grall

Re: [XEN PATCH v5 07/16] xen/build: Start using if_changed

[Julien Grall]

Hi,

On 21/04/2020 17:11, Anthony PERARD wrote:

This patch start to use if_changed introduced in a previous commit.

Whenever if_changed is called, the target must have FORCE as
dependency so that if_changed can check if the command line to be
run has changed, so the macro $(real-prereqs) must be used to
discover the dependencies without "FORCE".

Whenever a target isn't in obj-y, it should be added to extra-y so the
.*.cmd dependency file associated with the target can be loaded. This
is done for xsm/flask/ and both common/lib{elf,fdt}/ and
arch/x86/Makefile.

For the targets that generate .*.d dependency files, there's going to
be two dependency files (.*.d and .*.cmd) until we can merge them
together in a later patch via fixdep from Linux.

One cleanup, libelf-relocate.o doesn't exist anymore.

We import cmd_ld and cmd_objcopy from Linux v5.4.

Signed-off-by: Anthony PERARD 
Reviewed-by: Roger Pau Monné 
Acked-by: Jan Beulich 


For the Arm bits:

Acked-by: Julien Grall 

Cheers,


---

Notes:
 v4:
 - typos
 - fix missing FORCE in libfdt/Makefile
 - typo flask vs flash in xsm
 - in xsm, use *_H_DEPEND in command lines of mkaccess and mkflask
   instead of $(real-prereq) to avoid including the script as argument of
   itself.

  xen/Rules.mk   | 68 +++---
  xen/arch/arm/Makefile  |  4 +--
  xen/arch/x86/Makefile  |  1 +
  xen/arch/x86/efi/Makefile  |  7 ++--
  xen/common/libelf/Makefile | 12 ---
  xen/common/libfdt/Makefile | 11 +++---
  xen/xsm/flask/Makefile | 17 +++---
  7 files changed, 85 insertions(+), 35 deletions(-)

diff --git a/xen/Rules.mk b/xen/Rules.mk
index 21cac7f5f51a..2e28c572305a 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -56,6 +56,18 @@ SPECIAL_DATA_SECTIONS := rodata $(foreach a,1 2 4 8 16, \
  
  include Makefile
  
+# Linking

+# ---
+
+quiet_cmd_ld = LD  $@
+cmd_ld = $(LD) $(XEN_LDFLAGS) -r -o $@ $(real-prereqs)
+
+# Objcopy
+# ---
+
+quiet_cmd_objcopy = OBJCOPY $@
+cmd_objcopy = $(OBJCOPY) $(OBJCOPYFLAGS) $< $@
+
  define gendep
  ifneq ($(1),$(subst /,:,$(1)))
  DEPS += $(dir $(1)).$(notdir $(1)).d
@@ -164,29 +176,47 @@ else
$(CC) $(c_flags) -c $< -o $@
  endif
  
-%.o: %.S Makefile

-   $(CC) $(a_flags) -c $< -o $@
+quiet_cmd_cc_o_S = CC  $@
+cmd_cc_o_S = $(CC) $(a_flags) -c $< -o $@
+
+%.o: %.S FORCE
+   $(call if_changed,cc_o_S)
+
+
+quiet_cmd_obj_init_o = INIT_O  $@
+define cmd_obj_init_o
+$(OBJDUMP) -h $< | sed -n '/[0-9]/{s,00*,0,g;p;}' | while read idx name sz 
rest; do \
+case "$$name" in \
+.*.local) ;; \
+.text|.text.*|.data|.data.*|.bss) \
+test $$sz != 0 || continue; \
+echo "Error: size of $<:$$name is 0x$$sz" >&2; \
+exit $$(expr $$idx + 1);; \
+esac; \
+done; \
+$(OBJCOPY) $(foreach s,$(SPECIAL_DATA_SECTIONS),--rename-section 
.$(s)=.init.$(s)) $< $@
+endef
+
+$(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): %.init.o: %.o FORCE
+   $(call if_changed,obj_init_o)
+
+quiet_cmd_cpp_i_c = CPP $@
+cmd_cpp_i_c = $(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) $< -o $@
+
+quiet_cmd_cc_s_c = CC  $@
+cmd_cc_s_c = $(CC) $(filter-out -Wa$(comma)%,$(c_flags)) -S $< -o $@
  
-$(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): %.init.o: %.o Makefile

-   $(OBJDUMP) -h $< | sed -n '/[0-9]/{s,00*,0,g;p;}' | while read idx name 
sz rest; do \
-   case "$$name" in \
-   .*.local) ;; \
-   .text|.text.*|.data|.data.*|.bss) \
-   test $$sz != 0 || continue; \
-   echo "Error: size of $<:$$name is 0x$$sz" >&2; \
-   exit $$(expr $$idx + 1);; \
-   esac; \
-   done
-   $(OBJCOPY) $(foreach s,$(SPECIAL_DATA_SECTIONS),--rename-section 
.$(s)=.init.$(s)) $< $@
+quiet_cmd_s_S = CPP $@
+cmd_s_S = $(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@
  
-%.i: %.c Makefile

-   $(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) $< -o $@
+%.i: %.c FORCE
+   $(call if_changed,cpp_i_c)
  
-%.s: %.c Makefile

-   $(CC) $(filter-out -Wa$(comma)%,$(c_flags)) -S $< -o $@
+%.s: %.c FORCE
+   $(call if_changed,cc_s_c)
  
-%.s: %.S Makefile

-   $(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@
+%.s: %.S FORCE
+   $(call if_changed,cpp_s_S)
  
  # Add intermediate targets:

  # When building objects with specific suffix patterns, add intermediate
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 9f1ab2335756..b79ad55646bd 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -97,8 +97,8 @@ prelink_lto.o: $(ALL_OBJS)
  prelink.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS)) prelink_lto.o
$(LD) $(XEN_LDFLAGS) -r -o $@ $^
  else
-prelink.o: $(ALL_OBJS)
-

Re: [XEN PATCH v5 04/16] xen/build: have the root Makefile generates the CFLAGS

[Julien Grall]

Hi Anthony,

On 21/04/2020 17:11, Anthony PERARD wrote:

Instead of generating the CFLAGS in Rules.mk everytime we enter a new
subdirectory, we are going to generate most of them a single time, and
export the result in the environment so that Rules.mk can use it.  The
only flags left to be generated are the ones that depend on the
targets, but the variable $(c_flags) takes care of that.

Arch specific CFLAGS are generated by a new file "arch/*/arch.mk"
which is included by the root Makefile.

We export the *FLAGS via the environment variables XEN_*FLAGS because
Rules.mk still includes Config.mk and would add duplicated flags to
CFLAGS.

When running Rules.mk in the root directory (xen/), the variable
`root-make-done' is set, so `need-config' will remain undef and so the
root Makefile will not generate the cflags again.

We can't use CFLAGS in subdirectories to add flags to particular
targets, instead start to use CFLAGS-y. Idem for AFLAGS.
So there are two different CFLAGS-y, the one in xen/Makefile (and
arch.mk), and the one in subdirs that Rules.mk is going to use.
We can't add to XEN_CFLAGS because it is exported, so making change to
it might be propagated to subdirectory which isn't intended.

Some style change are introduced in this patch:
 when LDFLAGS_DIRECT is included in LDFLAGS
 use of CFLAGS-$(CONFIG_INDIRECT_THUNK) instead of ifeq().

The LTO change hasn't been tested properly, as LTO is marked as
broken.

Signed-off-by: Anthony PERARD 


For the Arm bits:

Acked-by: Julien Grall 

Cheers,

--
Julien Grall

Re: [PATCH 1/6] x86_64/mm: map and unmap page tables in cleanup_frame_table

[Julien Grall]

On 24/04/2020 10:21, Hongyan Xia wrote:

Hi Julien,

On Fri, 2020-04-24 at 09:59 +0100, Julien Grall wrote:

(resending)

On 17/04/2020 10:52, Hongyan Xia wrote:

From: Wei Liu 

Also fix a weird indentation.

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
---
   xen/arch/x86/x86_64/mm.c | 14 +++---
   1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c
index e85ef449f3..18210405f4 100644
--- a/xen/arch/x86/x86_64/mm.c
+++ b/xen/arch/x86/x86_64/mm.c
@@ -737,8 +737,8 @@ static void cleanup_frame_table(struct
mem_hotadd_info *info)
   
   while (sva < eva)

   {
-l3e = l4e_to_l3e(idle_pg_table[l4_table_offset(sva)])[
-  l3_table_offset(sva)];
+l3e = l3e_from_l4e(idle_pg_table[l4_table_offset(sva)],
+   l3_table_offset(sva));


This macro doesn't exist yet in the tree. It would be good to spell
out
the dependencies in the cover letter so this doesn't get merged
before
the dependency is merged.


I believe the introduction of the new macros has been merged in staging
as 6c8afe5aadb33761431b24157d99b25eac15fc7e.


H you are right. I must have been blind. Sorry for the noise.

Cheers,

--
Julien Grall

Re: [PATCH 1/6] x86_64/mm: map and unmap page tables in cleanup_frame_table

[Hongyan Xia]

Hi Julien,

On Fri, 2020-04-24 at 09:59 +0100, Julien Grall wrote:
> (resending)
> 
> On 17/04/2020 10:52, Hongyan Xia wrote:
> > From: Wei Liu 
> > 
> > Also fix a weird indentation.
> > 
> > Signed-off-by: Wei Liu 
> > Signed-off-by: Hongyan Xia 
> > ---
> >   xen/arch/x86/x86_64/mm.c | 14 +++---
> >   1 file changed, 7 insertions(+), 7 deletions(-)
> > 
> > diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c
> > index e85ef449f3..18210405f4 100644
> > --- a/xen/arch/x86/x86_64/mm.c
> > +++ b/xen/arch/x86/x86_64/mm.c
> > @@ -737,8 +737,8 @@ static void cleanup_frame_table(struct
> > mem_hotadd_info *info)
> >   
> >   while (sva < eva)
> >   {
> > -l3e = l4e_to_l3e(idle_pg_table[l4_table_offset(sva)])[
> > -  l3_table_offset(sva)];
> > +l3e = l3e_from_l4e(idle_pg_table[l4_table_offset(sva)],
> > +   l3_table_offset(sva));
> 
> This macro doesn't exist yet in the tree. It would be good to spell
> out 
> the dependencies in the cover letter so this doesn't get merged
> before 
> the dependency is merged.

I believe the introduction of the new macros has been merged in staging
as 6c8afe5aadb33761431b24157d99b25eac15fc7e.

> Reviewed-by: Julien Grall 

Thanks!

Hongyan

Re: [XEN PATCH v5 03/16] xen/build: use new $(c_flags) and $(a_flags) instead of $(CFLAGS)

[Julien Grall]

Hi,

On 21/04/2020 17:11, Anthony PERARD wrote:

In a later patch ("xen/build: have the root Makefile generates the
CFLAGS), we want to generate the CFLAGS in xen/Makefile, then export
it and have Rules.mk use a CFLAGS from the environment variables. That
changes the flavor of the CFLAGS and flags intended for one target
(like -D__OBJECT_FILE__ and -M%) gets propagated and duplicated. So we
start by moving such flags out of $(CFLAGS) and into $(c_flags) which
is to be modified by only Rules.mk.

__OBJECT_FILE__ is only used by arch/x86/mm/*.c files, so having it in
$(c_flags) is enough, we don't need it in $(a_flags).

For include/Makefile and as-insn we can keep using CFLAGS, but since
it doesn't have -M* flags anymore there is no need to filter them out.

The XEN_BUILD_EFI tests in arch/x86/Makefile was filtering out
CFLAGS-y, but according to dd40177c1bc8 ("x86-64/EFI: add CFLAGS to
check compile"), it was done to filter out -MF. CFLAGS doesn't
have those flags anymore, so no filtering is needed.

This is inspired by the way Kbuild generates CFLAGS for each targets.

Signed-off-by: Anthony PERARD 
Reviewed-by: Roger Pau Monné 
Reviewed-by: Jan Beulich 


For the Arm bits:

Acked-by: Julien Grall 

Cheers,

--
Julien Grall

Re: [PATCH 6/6] x86/pv: map and unmap page table in dom0_construct_pv

[Julien Grall]

Hi,

On 17/04/2020 10:52, Hongyan Xia wrote:

From: Wei Liu 

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 


Reviewed-by: Julien Grall 

Cheers,

--
Julien Grall

Re: [PATCH 5/6] x86/pv: map and unmap page tables in mark_pv_pt_pages_rdonly

[Julien Grall]

Hi,

On 17/04/2020 10:52, Hongyan Xia wrote:

From: Wei Liu 

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 


Reviewed-by: Julien Grall 

Cheers,

--
Julien Grall

Re: [PATCH 4/6] x86/smpboot: map and unmap page tables in cleanup_cpu_root_pgt

[Julien Grall]

Hi,

On 17/04/2020 10:52, Hongyan Xia wrote:

From: Wei Liu 

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 


Reviewed-by: Julien Grall 


---
  xen/arch/x86/smpboot.c | 25 +
  1 file changed, 17 insertions(+), 8 deletions(-)

diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index 09264b02d1..275ce7661d 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -858,23 +858,27 @@ static void cleanup_cpu_root_pgt(unsigned int cpu)
r < root_table_offset(HYPERVISOR_VIRT_END); ++r )
  {
  l3_pgentry_t *l3t;
+mfn_t l3mfn;
  unsigned int i3;
  
  if ( !(root_get_flags(rpt[r]) & _PAGE_PRESENT) )

  continue;
  
-l3t = l4e_to_l3e(rpt[r]);

+l3mfn = l4e_get_mfn(rpt[r]);
+l3t = map_domain_page(l3mfn);
  
  for ( i3 = 0; i3 < L3_PAGETABLE_ENTRIES; ++i3 )

  {
  l2_pgentry_t *l2t;
+mfn_t l2mfn;
  unsigned int i2;
  
  if ( !(l3e_get_flags(l3t[i3]) & _PAGE_PRESENT) )

  continue;
  
  ASSERT(!(l3e_get_flags(l3t[i3]) & _PAGE_PSE));

-l2t = l3e_to_l2e(l3t[i3]);
+l2mfn = l3e_get_mfn(l3t[i3]);
+l2t = map_domain_page(l2mfn);
  
  for ( i2 = 0; i2 < L2_PAGETABLE_ENTRIES; ++i2 )

  {
@@ -882,13 +886,15 @@ static void cleanup_cpu_root_pgt(unsigned int cpu)
  continue;
  
  ASSERT(!(l2e_get_flags(l2t[i2]) & _PAGE_PSE));

-free_xen_pagetable(l2e_to_l1e(l2t[i2]));
+free_xen_pagetable_new(l2e_get_mfn(l2t[i2]));
  }
  
-free_xen_pagetable(l2t);

+unmap_domain_page(l2t);
+free_xen_pagetable_new(l2mfn);
  }
  
-free_xen_pagetable(l3t);

+unmap_domain_page(l3t);
+free_xen_pagetable_new(l3mfn);
  }
  
  free_xen_pagetable(rpt);

@@ -896,11 +902,14 @@ static void cleanup_cpu_root_pgt(unsigned int cpu)
  /* Also zap the stub mapping for this CPU. */
  if ( stub_linear )
  {
-l3_pgentry_t *l3t = l4e_to_l3e(common_pgt);
-l2_pgentry_t *l2t = l3e_to_l2e(l3t[l3_table_offset(stub_linear)]);
-l1_pgentry_t *l1t = l2e_to_l1e(l2t[l2_table_offset(stub_linear)]);
+l3_pgentry_t l3e = l3e_from_l4e(common_pgt,
+l3_table_offset(stub_linear));
+l2_pgentry_t l2e = l2e_from_l3e(l3e, l2_table_offset(stub_linear));
+l1_pgentry_t *l1t = map_l1t_from_l2e(l2e);
  
  l1t[l1_table_offset(stub_linear)] = l1e_empty();

+
+unmap_domain_page(l1t);
  }
  }
  



Cheers,

--
Julien Grall

Re: [PATCH 3/6] x86_64/mm: map and unmap page tables in subarch_memory_op

[Julien Grall]

Hi Hongyan,

On 17/04/2020 10:52, Hongyan Xia wrote:

From: Wei Liu 

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 


Reviewed-by: Julien Grall 

Cheers,

--
Julien Grall

Re: [PATCH 2/6] x86_64/mm: map and unmap page tables in subarch_init_memory

[Julien Grall]

Hi,

On 17/04/2020 10:52, Hongyan Xia wrote:

From: Wei Liu 

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 


Reviewed-by: Julien Grall 

Cheers,

--
Julien Grall

Re: [PATCH 1/6] x86_64/mm: map and unmap page tables in cleanup_frame_table

[Julien Grall]

On 17/04/2020 10:52, Hongyan Xia wrote:

From: Wei Liu 

Also fix a weird indentation.

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
---
  xen/arch/x86/x86_64/mm.c | 14 +++---
  1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c
index e85ef449f3..18210405f4 100644
--- a/xen/arch/x86/x86_64/mm.c
+++ b/xen/arch/x86/x86_64/mm.c
@@ -737,8 +737,8 @@ static void cleanup_frame_table(struct mem_hotadd_info 
*info)
  
  while (sva < eva)

  {
-l3e = l4e_to_l3e(idle_pg_table[l4_table_offset(sva)])[
-  l3_table_offset(sva)];
+l3e = l3e_from_l4e(idle_pg_table[l4_table_offset(sva)],
+   l3_table_offset(sva));
  if ( !(l3e_get_flags(l3e) & _PAGE_PRESENT) ||
   (l3e_get_flags(l3e) & _PAGE_PSE) )
  {
@@ -747,7 +747,7 @@ static void cleanup_frame_table(struct mem_hotadd_info 
*info)
  continue;
  }
  
-l2e = l3e_to_l2e(l3e)[l2_table_offset(sva)];

+l2e = l2e_from_l3e(l3e, l2_table_offset(sva));
  ASSERT(l2e_get_flags(l2e) & _PAGE_PRESENT);
  
  if ( (l2e_get_flags(l2e) & (_PAGE_PRESENT | _PAGE_PSE)) ==

@@ -763,10 +763,10 @@ static void cleanup_frame_table(struct mem_hotadd_info 
*info)
  continue;
  }
  
-ASSERT(l1e_get_flags(l2e_to_l1e(l2e)[l1_table_offset(sva)]) &

-_PAGE_PRESENT);
- sva = (sva & ~((1UL << PAGE_SHIFT) - 1)) +
-(1UL << PAGE_SHIFT);
+ASSERT(l1e_get_flags(l1e_from_l2e(l2e, l1_table_offset(sva))) &
+   _PAGE_PRESENT);
+
+sva = (sva & ~((1UL << PAGE_SHIFT) - 1)) + (1UL << PAGE_SHIFT);


NIT: While you are modifying the indentation. Couldn't we use PAGE_MASK 
and PAGE_SIZE here?


Cheers,

--
Julien Grall

Re: [PATCH 1/6] x86_64/mm: map and unmap page tables in cleanup_frame_table

[Julien Grall]

(resending)

On 17/04/2020 10:52, Hongyan Xia wrote:

From: Wei Liu 

Also fix a weird indentation.

Signed-off-by: Wei Liu 
Signed-off-by: Hongyan Xia 
---
  xen/arch/x86/x86_64/mm.c | 14 +++---
  1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c
index e85ef449f3..18210405f4 100644
--- a/xen/arch/x86/x86_64/mm.c
+++ b/xen/arch/x86/x86_64/mm.c
@@ -737,8 +737,8 @@ static void cleanup_frame_table(struct mem_hotadd_info 
*info)
  
  while (sva < eva)

  {
-l3e = l4e_to_l3e(idle_pg_table[l4_table_offset(sva)])[
-  l3_table_offset(sva)];
+l3e = l3e_from_l4e(idle_pg_table[l4_table_offset(sva)],
+   l3_table_offset(sva));


This macro doesn't exist yet in the tree. It would be good to spell out 
the dependencies in the cover letter so this doesn't get merged before 
the dependency is merged.


Reviewed-by: Julien Grall 

Cheers,

--
Julien Grall

Re: [PATCH 1/6] x86_64/mm: map and unmap page tables in cleanup_frame_table

[Julien Grall]

On 24/04/2020 09:58, Julien Grall wrote:

Hi,

On 17/04/2020 10:52, Hongyan Xia wrote:> diff --git 
a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c> index 
e85ef449f3..18210405f4 100644> --- a/xen/arch/x86/x86_64/mm.c> +++ 
b/xen/arch/x86/x86_64/mm.c> @@ -737,8 +737,8 @@ static void 
cleanup_frame_table(struct mem_hotadd_info *info)>   >   while (sva 
< eva)>   {> -    l3e = 
l4e_to_l3e(idle_pg_table[l4_table_offset(sva)])[> - 
l3_table_offset(sva)];> +    l3e = 
l3e_from_l4e(idle_pg_table[l4_table_offset(sva)],> +   
l3_table_offset(sva));
This macro doesn't exist yet in the tree. It would be good to spell out 
the dependencies in the cover letter so this doesn't get merged before 
the dependency is merged.


Reviewed-by: Julien Grall 


Argh, I screwed the reply. Sorry for that. I will resend it.

--
Julien Grall

Re: [PATCH 1/6] x86_64/mm: map and unmap page tables in cleanup_frame_table

[Julien Grall]

Hi,

On 17/04/2020 10:52, Hongyan Xia wrote:> diff --git 
a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c> index 
e85ef449f3..18210405f4 100644> --- a/xen/arch/x86/x86_64/mm.c> +++ 
b/xen/arch/x86/x86_64/mm.c> @@ -737,8 +737,8 @@ static void 
cleanup_frame_table(struct mem_hotadd_info *info)>   >   while (sva 
< eva)>   {> -l3e = 
l4e_to_l3e(idle_pg_table[l4_table_offset(sva)])[> - 
l3_table_offset(sva)];> +l3e = 
l3e_from_l4e(idle_pg_table[l4_table_offset(sva)],> + 
  l3_table_offset(sva));
This macro doesn't exist yet in the tree. It would be good to spell out 
the dependencies in the cover letter so this doesn't get merged before 
the dependency is merged.


Reviewed-by: Julien Grall 

Cheers,

--
Julien Grall

Re: [PATCH 0/6] convert more Xen page table code to the new API

[Hongyan Xia]

A gentle ping.

On Fri, 2020-04-17 at 10:52 +0100, Hongyan Xia wrote:
> From: Hongyan Xia 
> 
> Basically just rewriting functions using the new API to map and unmap
> PTEs. Each patch is independent.
> 
> Apart from mapping and unmapping page tables, no other functional
> change
> intended.
> 
> Wei Liu (6):
>   x86_64/mm: map and unmap page tables in cleanup_frame_table
>   x86_64/mm: map and unmap page tables in subarch_init_memory
>   x86_64/mm: map and unmap page tables in subarch_memory_op
>   x86/smpboot: map and unmap page tables in cleanup_cpu_root_pgt
>   x86/pv: map and unmap page tables in mark_pv_pt_pages_rdonly
>   x86/pv: map and unmap page table in dom0_construct_pv
> 
>  xen/arch/x86/pv/dom0_build.c | 38 
> 
>  xen/arch/x86/smpboot.c   | 25 
>  xen/arch/x86/x86_64/mm.c | 32 +++---
>  3 files changed, 58 insertions(+), 37 deletions(-)
> 

RE: [ANNOUNCE] Xen 4.14 Development Update

[Durrant, Paul]

> -Original Message-
> On 24.04.2020 09:38, Paul Durrant wrote:
> > *  Memory read caching in emulation (v5)
> >   -  Jan Beulich
> 
> Unless issues are found with this, it can be moved to "completed" -
> I did commit this yesterday.
> 

Someone just drew my attention to the commit so once it hits master it should 
indeed be marked as such.
Thanks,

  Paul

Re: [ANNOUNCE] Xen 4.14 Development Update

[Jan Beulich]

On 24.04.2020 09:38, Paul Durrant wrote:
> *  Memory read caching in emulation (v5)
>   -  Jan Beulich

Unless issues are found with this, it can be moved to "completed" -
I did commit this yesterday.

Jan

[ANNOUNCE] Xen 4.14 Development Update

[Paul Durrant]

*** ONE WEEK UNTIL LAST POSTING ***

This email only tracks big items for xen.git tree. Please reply for items
you would like to see in 4.14 so that people have an idea what
is going on and prioritise accordingly.

You're welcome to provide description and use cases of the feature you're
working on.

= Timeline =

We now adopt a fixed cut-off date scheme. We will release about every 8
 months.
The critical dates for Xen 4.14 are as follows:

---> We are here
* Last posting date: May 1st, 2020
* Hard code freeze: May 22nd, 2020
* Release: June 26th, 2020

Note that we don't have a freeze exception scheme anymore. All patches
that wish to go into 4.14 must be posted initially no later than the
last posting date and finally no later than the hard code freeze.
All patches posted after that date will be automatically queued into next
release.

RCs will be arranged immediately after freeze.

There is also a jira instance to track all the tasks (not only big)
for the project. See: https://xenproject.atlassian.net/projects/XEN/issues.

Some of the tasks tracked by this e-mail also have a corresponding jira task
referred by XEN-N.

There is a version number for patch series associated to each feature.
Can each owner send an update giving the latest version number if the
series has been re-posted? Also, can the owners of any completed items
please respond so that the item can be moved into the 'Completed' section.

= Projects =

== Hypervisor == 

*  Live-Updating Xen (RFC v2)
  -  David Woodhouse
  -  The latest code is available at 
https://xenbits.xen.org/gitweb/?p=people/dwmw2/xen.git;a=shortlog;h=refs/heads/lu-master
  -  Project wiki page at https://wiki.xenproject.org/wiki/Live-Updating_Xen

*  Non-Cooperative Live Migration (v7 (design docs))
  -  Paul Durrant

*  Secret Hiding (v5)
  -  Hongyan Xia

*  Hypervisor file system (v6)
  -  Juergen Gross

=== x86 === 

*  Linux stub domains (v4)
  -  Marek Marczykowski-Górecki

*  Virtualise MSR_ARCH_CAPS for guests
  -  Andrew Cooper

*  AMD hardware mitigations (Rome)
  -  Andrew Cooper

*  Xen ioreq server (v3)
  -  Roger Pau Monne

*  Memory read caching in emulation (v5)
  -  Jan Beulich

*  Instruction emulator improvements
  -  Jan Beulich

*  VM forking (v11)
  -  Tamas K Lengyel

=== ARM === 

== Completed == 


Paul Durrant