[Xen-devel] [xen-unstable-smoke test] 125819: trouble: blocked/broken/pass

2018-08-08 Thread osstest service owner
flight 125819 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/125819/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken

Regressions which are regarded as allowable (not blocking):
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 125729

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64-xsm   3 capture-logs  broken blocked in 125729
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  4b60c40659b34b6577a6bc91eb4115458a0e425f
baseline version:
 xen  ed5f8d9ca47e69e30221c37ec812f2b38af19d83

Last test of basis   125729  2018-08-01 11:00:39 Z7 days
Failing since125741  2018-08-02 10:01:09 Z6 days   22 attempts
Testing same since   125802  2018-08-08 11:00:47 Z0 days6 attempts


People who touched revisions under test:
  Alexandru Isaila 
  Andrew Cooper 
  Doug Goldstein 
  George Dunlap 
  Jan Beulich 
  Julien Grall 
  Kevin Tian 
  Razvan Cojocaru 
  Roger Pau Monné 
  Stefano Stabellini 
  Wei Liu 

jobs:
 build-arm64-xsm  broken  
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64-xsm broken
broken-step build-arm64-xsm hosts-allocate
broken-step build-arm64-xsm capture-logs

Not pushing.

(No revision log; it would be 512 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [libvirt test] 125797: regressions - trouble: blocked/broken/fail/pass

2018-08-08 Thread osstest service owner
flight 125797 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/125797/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken
 build-arm64  broken
 build-arm64-pvopsbroken
 build-i386-libvirt6 libvirt-buildfail REGR. vs. 123814
 build-amd64-libvirt   6 libvirt-buildfail REGR. vs. 123814
 build-armhf-libvirt   6 libvirt-buildfail REGR. vs. 123814

Regressions which are regarded as allowable (not blocking):
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 123814
 build-arm64   2 hosts-allocate broken REGR. vs. 123814
 build-arm64-pvops 2 hosts-allocate broken REGR. vs. 123814

Tests which did not succeed, but are not blocking:
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt  1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 build-arm64   3 capture-logs  broken blocked in 123814
 build-arm64-xsm   3 capture-logs  broken blocked in 123814
 build-arm64-pvops 3 capture-logs  broken blocked in 123814

version targeted for testing:
 libvirt  26cfb1a3cd39d731099ee7d5d1c47b3730ebde16
baseline version:
 libvirt  076a2b409667dd9f716a2a2085e1ffea9d58fe8b

Last test of basis   123814  2018-06-05 04:19:23 Z   64 days
Failing since123840  2018-06-06 04:19:28 Z   63 days   49 attempts
Testing same since   125797  2018-08-08 04:18:52 Z0 days1 attempts


People who touched revisions under test:
Ales Musil 
  Andrea Bolognani 
  Anya Harter 
  Bjoern Walk 
  Bobo Du 
  Boris Fiuczynski 
  Brijesh Singh 
  Changkuo Shi 
  Chen Hanxiao 
  Christian Ehrhardt 
  Clementine Hayat 
  Cole Robinson 
  Daniel Nicoletti 
  Daniel P. Berrangé 
  Daniel Veillard 
  Eric Blake 
  Erik Skultety 
  Fabiano Fidêncio 
  Filip Alac 
  Han Han 
  intrigeri 
  intrigeri 
  Jamie Strandboge 
  Jie Wang 
  Jiri Denemark 
  John Ferlan 
  Julio Faracco 
  Ján Tomko 
  Kashyap Chamarthy 
  Katerina Koukiou 
  Laine Stump 
  Laszlo Ersek 
  Luyao Huang 
  Marc Hartmayer 
  Marc Hartmayer 
  Marcos Paulo de Souza 
  Martin Kletzander 
  Matthias Bolte 
  Michal Privoznik 
  Michal Prívozník 
  Nikolay Shirokovskiy 
  Pavel Hrdina 
  Peter Krempa 
  Pino Toscano 
  Radostin Stoyanov 
  Ramy Elkest 
  ramyelkest 
  Richard W.M. Jones 
  Roman Bogorodskiy 
  Shi Lei 
  Shichangkuo 
  Simon Kobyda 
  Stefan Bader 
  Stefan Berger 
  Sukrit Bhatnagar 
  Tomáš Golembiovský 
  w00251574 
  Weilun Zhu 

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  broken  
 build-i386-xsm   pass
 build-amd64  pass
 build-arm64  broken  
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  fail
 build-arm64-libvirt  blocked 
 build-armhf-libvirt  fail
 build-i386-libvirt   fail
 build-amd64-pvopspass
 build-arm64-pvopsbroken  
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   blocked 
 

[Xen-devel] [linux-4.9 test] 125795: regressions - trouble: blocked/broken/fail/pass

2018-08-08 Thread osstest service owner
flight 125795 linux-4.9 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/125795/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken
 build-arm64  broken
 build-arm64-pvopsbroken
 build-armhf-libvirt   6 libvirt-buildfail REGR. vs. 125183

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemuu-ovmf-amd64 16 guest-localmigrate/x10 fail in 125781 
pass in 125795
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 16 guest-localmigrate/x10 fail 
pass in 125781

Regressions which are regarded as allowable (not blocking):
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 125183
 build-arm64   2 hosts-allocate broken REGR. vs. 125183
 build-arm64-pvops 2 hosts-allocate broken REGR. vs. 125183

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-arm64-arm64-examine  1 build-check(1)   blocked  n/a
 build-arm64   3 capture-logs  broken blocked in 125183
 build-arm64-xsm   3 capture-logs  broken blocked in 125183
 build-arm64-pvops 3 capture-logs  broken blocked in 125183
 test-armhf-armhf-libvirt13 migrate-support-check fail in 125781 never pass
 test-armhf-armhf-libvirt 14 saverestore-support-check fail in 125781 never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-check fail in 125781 never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-check fail in 125781 never 
pass
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 125183
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 125183
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 125183
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 125183
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 125183
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-amd64-xl-pvhv2-intel 12 guest-start fail never pass
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-amd64-xl-pvhv2-amd 12 guest-start  fail  never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop fail never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 

[Xen-devel] [linux-linus test] 125788: trouble: blocked/broken/fail/pass

2018-08-08 Thread osstest service owner
flight 125788 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/125788/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64  broken
 build-arm64-pvopsbroken
 build-arm64-xsm  broken

Regressions which are regarded as allowable (not blocking):
 build-arm64-pvops 2 hosts-allocate broken REGR. vs. 125702
 build-arm64   2 hosts-allocate broken REGR. vs. 125702
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 125702
 test-amd64-amd64-rumprun-amd64 17 rumprun-demo-xenstorels/xenstorels.repeat 
fail REGR. vs. 125702

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-examine  1 build-check(1)   blocked  n/a
 build-arm64   3 capture-logs  broken blocked in 125702
 build-arm64-pvops 3 capture-logs  broken blocked in 125702
 build-arm64-xsm   3 capture-logs  broken blocked in 125702
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 125702
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 125702
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 125702
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 125702
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 125702
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 125702
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 125702
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 125702
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass

version targeted for testing:
 linux1236568ee3cbb0d3ac62d0074a29b97ecf34cbbc
baseline version:
 linux 

[Xen-devel] [xen-unstable-smoke test] 125816: trouble: blocked/broken/pass

2018-08-08 Thread osstest service owner
flight 125816 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/125816/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken

Regressions which are regarded as allowable (not blocking):
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 125729

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64-xsm   3 capture-logs  broken blocked in 125729
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  4b60c40659b34b6577a6bc91eb4115458a0e425f
baseline version:
 xen  ed5f8d9ca47e69e30221c37ec812f2b38af19d83

Last test of basis   125729  2018-08-01 11:00:39 Z7 days
Failing since125741  2018-08-02 10:01:09 Z6 days   21 attempts
Testing same since   125802  2018-08-08 11:00:47 Z0 days5 attempts


People who touched revisions under test:
  Alexandru Isaila 
  Andrew Cooper 
  Doug Goldstein 
  George Dunlap 
  Jan Beulich 
  Julien Grall 
  Kevin Tian 
  Razvan Cojocaru 
  Roger Pau Monné 
  Stefano Stabellini 
  Wei Liu 

jobs:
 build-arm64-xsm  broken  
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64-xsm broken
broken-step build-arm64-xsm hosts-allocate
broken-step build-arm64-xsm capture-logs

Not pushing.

(No revision log; it would be 512 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [ovmf baseline-only test] 75055: tolerable FAIL

2018-08-08 Thread Platform Team regression test user
This run is configured for baseline tests only.

flight 75055 ovmf real [real]
http://osstest.xensource.com/osstest/logs/75055/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-ovmf-amd64 10 debian-hvm-install  fail like 75054
 test-amd64-amd64-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail like 75054

version targeted for testing:
 ovmf 9e6c4f1527e6f72d6ada10aa39854f2bdd40772f
baseline version:
 ovmf b6e48ec6412eab0f21fdff5a045a7ee516574d44

Last test of basis75054  2018-08-08 12:49:57 Z0 days
Testing same since75055  2018-08-08 22:53:08 Z0 days1 attempts


People who touched revisions under test:
  Jaben Carsey 
  Liming Gao 
  Star Zeng 
  Yunhua Feng 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 fail
 test-amd64-i386-xl-qemuu-ovmf-amd64  fail



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xensource.com/osstest/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


commit 9e6c4f1527e6f72d6ada10aa39854f2bdd40772f
Author: Star Zeng 
Date:   Tue Aug 7 18:01:12 2018 +0800

FmpDevicePkg FmpDxe: Lock variables in entrypoint instead of callback

Current code locks variables in PcdFmpDeviceLockEventGuid callback by
VariableLock protocol whose interface will be closed at EndOfDxe.
So the PcdFmpDeviceLockEventGuid callback needs be executed before
the EndOfDxe callback in Variable driver.
When PcdFmpDeviceLockEventGuid = gEfiEndOfDxeEventGroupGuid, the
callback's execution sequence depends on the callback's TPL and
registration sequence.
When PcdFmpDeviceLockEventGuid = gEfiEventReadyToBootGuid, the
PcdFmpDeviceLockEventGuid callback will be executed after the
EndOfDxe callback in Variable driver, the locking will fail.

The patch moves the variables locking logic to entrypoint.
The patch also moves the IsLockFmpDeviceAtLockEventGuidRequired ()
checking to entrypoint.

The entrypoint's final return status should be better to depend on
the return status of RegisterFmpInstaller/InstallFmpInstance, but not
gBS->CreateEventEx.
So the patch also moves the RegisterFmpInstaller/InstallFmpInstance
calling to the end of entrypoint.

Cc: Michael D Kinney 
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Star Zeng 
Reviewed-by: Michael D Kinney 

commit 6ec4d300fe5929a43b9118adc80aa909f61ab665
Author: Star Zeng 
Date:   Mon Aug 6 15:46:36 2018 +0800

MdeModulePkg ErstFmpDxe: Create ESRT in ReadyToBoot event

Current code just creates ESRT entry in FMP notification
and installs ESRT configuration table in ReadyToBoot event.

The LastAttemptVersion and LastAttemptStatus in ESRT will
be out of date after system continues to boot without reset
after capsule update (reset is not required or capsule update
is failed).

This patches updates the code to create ESRT based on all
FMP instances in ReadyToBoot event.

Cc: Michael D Kinney 
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Star Zeng 
Reviewed-by: Michael D Kinney 

commit 27e42bf61bb27a61b5b4dd053c6bc219c73c4cc8
Author: Star Zeng 
Date:   Mon Aug 6 15:44:59 2018 +0800

FmpDevicePkg FmpDxe: Need repopulate after SetImage is called

No need repopulate if SetImage is not called.
But need repopulate after SetImage is called to update
LastAttemptVersion and LastAttemptStatus

Cc: Michael D Kinney 
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Star Zeng 
Reviewed-by: Michael D Kinney 

commit cefc8d8821f0a5ec7995901146dd6b055d7b956a
Author: Yunhua Feng 
Date:   Mon Aug 6 09:02:25 2018 +0800

BaseTools: Use gGuidPattern for Guid regular expression

Use GlobalData.py gGuidPattern for Guid regular expression

Cc: Liming Gao 
Cc: Yonghong Zhu 
Contributed-under: TianoCore 

[Xen-devel] [xen-unstable test] 125789: regressions - trouble: blocked/broken/fail/pass

2018-08-08 Thread osstest service owner
flight 125789 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/125789/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken
 build-arm64-pvopsbroken
 build-arm64  broken
 build-i386-pvops  6 kernel-build   fail in 125770 REGR. vs. 125691

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-libvirt 18 guest-start/debian.repeat  fail pass in 125770

Regressions which are regarded as allowable (not blocking):
 build-arm64   2 hosts-allocate broken REGR. vs. 125691
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 125691
 build-arm64-pvops 2 hosts-allocate broken REGR. vs. 125691

Tests which did not succeed, but are not blocking:
 test-amd64-i386-pair  1 build-check(1)   blocked in 125770 n/a
 test-amd64-i386-qemut-rhel6hvm-intel  1 build-check(1)   blocked in 125770 n/a
 test-amd64-i386-xl-raw1 build-check(1)   blocked in 125770 n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)blocked in 125770 n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 1 build-check(1) blocked in 
125770 n/a
 test-amd64-i386-xl1 build-check(1)   blocked in 125770 n/a
 test-amd64-i386-xl-qemut-debianhvm-amd64 1 build-check(1) blocked in 125770 n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)blocked in 125770 n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked in 125770 n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1) blocked in 125770 n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64 1 build-check(1) blocked in 125770 n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked in 125770 n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked in 
125770 n/a
 test-amd64-i386-xl-pvshim 1 build-check(1)   blocked in 125770 n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked in 125770 n/a
 test-amd64-i386-xl-qemut-debianhvm-amd64-xsm 1 build-check(1) blocked in 
125770 n/a
 test-amd64-i386-migrupgrade   1 build-check(1)   blocked in 125770 n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)blocked in 125770 n/a
 test-amd64-i386-xl-qemut-ws16-amd64  1 build-check(1)blocked in 125770 n/a
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 1 build-check(1) blocked 
in 125770 n/a
 test-amd64-i386-xl-shadow 1 build-check(1)   blocked in 125770 n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1)   blocked in 125770 n/a
 test-amd64-i386-qemut-rhel6hvm-amd  1 build-check(1) blocked in 125770 n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked in 
125770 n/a
 test-amd64-i386-examine   1 build-check(1)   blocked in 125770 n/a
 test-amd64-i386-xl-qemut-win7-amd64  1 build-check(1)blocked in 125770 n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)blocked in 125770 n/a
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
in 125770 n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1) blocked in 125770 n/a
 test-amd64-i386-xl-qemut-win10-i386  1 build-check(1)blocked in 125770 n/a
 test-amd64-i386-rumprun-i386  1 build-check(1)   blocked in 125770 n/a
 test-amd64-i386-xl-qemuu-win10-i386  1 build-check(1)blocked in 125770 n/a
 test-amd64-i386-livepatch 1 build-check(1)   blocked in 125770 n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked in 125770 n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-arm64-arm64-examine  1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64   3 capture-logs  broken blocked in 125691
 build-arm64-pvops 3 capture-logs  broken blocked in 125691
 build-arm64-xsm   3 capture-logs  broken blocked in 125691
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 125691
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 125691
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 125691
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 125691
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 125691
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 125691
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 125691
 

[Xen-devel] [xen-unstable-smoke test] 125814: trouble: blocked/broken/pass

2018-08-08 Thread osstest service owner
flight 125814 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/125814/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken

Regressions which are regarded as allowable (not blocking):
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 125729

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64-xsm   3 capture-logs  broken blocked in 125729
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  4b60c40659b34b6577a6bc91eb4115458a0e425f
baseline version:
 xen  ed5f8d9ca47e69e30221c37ec812f2b38af19d83

Last test of basis   125729  2018-08-01 11:00:39 Z7 days
Failing since125741  2018-08-02 10:01:09 Z6 days   20 attempts
Testing same since   125802  2018-08-08 11:00:47 Z0 days4 attempts


People who touched revisions under test:
  Alexandru Isaila 
  Andrew Cooper 
  Doug Goldstein 
  George Dunlap 
  Jan Beulich 
  Julien Grall 
  Kevin Tian 
  Razvan Cojocaru 
  Roger Pau Monné 
  Stefano Stabellini 
  Wei Liu 

jobs:
 build-arm64-xsm  broken  
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64-xsm broken
broken-step build-arm64-xsm hosts-allocate
broken-step build-arm64-xsm capture-logs

Not pushing.

(No revision log; it would be 512 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [ovmf test] 125803: all pass - PUSHED

2018-08-08 Thread osstest service owner
flight 125803 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/125803/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf 9e6c4f1527e6f72d6ada10aa39854f2bdd40772f
baseline version:
 ovmf b6e48ec6412eab0f21fdff5a045a7ee516574d44

Last test of basis   125790  2018-08-07 14:59:27 Z1 days
Testing same since   125803  2018-08-08 12:29:12 Z0 days1 attempts


People who touched revisions under test:
  Jaben Carsey 
  Liming Gao 
  Star Zeng 
  Yunhua Feng 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   b6e48ec641..9e6c4f1527  9e6c4f1527e6f72d6ada10aa39854f2bdd40772f -> 
xen-tested-master

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [linux-4.14 test] 125786: trouble: blocked/broken/fail/pass

2018-08-08 Thread osstest service owner
flight 125786 linux-4.14 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/125786/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-pvopsbroken
 build-arm64  broken
 build-arm64-xsm  broken

Regressions which are regarded as allowable (not blocking):
 build-arm64   2 hosts-allocate broken REGR. vs. 125175
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 125175
 build-arm64-pvops 2 hosts-allocate broken REGR. vs. 125175

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-examine  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 build-arm64   3 capture-logs  broken blocked in 125175
 build-arm64-xsm   3 capture-logs  broken blocked in 125175
 build-arm64-pvops 3 capture-logs  broken blocked in 125175
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop fail never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail never pass
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop fail never pass
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass

version targeted for testing:
 linux2ae6c0413b4768f9d8fc6f718a732f9dae014b67
baseline version:
 linux1e92e813554a93741666e9f378a83d70405b9076

Last test of basis   125175  2018-07-15 08:04:36 Z   24 

[Xen-devel] [xen-unstable-smoke test] 125810: trouble: blocked/broken/pass

2018-08-08 Thread osstest service owner
flight 125810 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/125810/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken

Regressions which are regarded as allowable (not blocking):
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 125729

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64-xsm   3 capture-logs  broken blocked in 125729
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  4b60c40659b34b6577a6bc91eb4115458a0e425f
baseline version:
 xen  ed5f8d9ca47e69e30221c37ec812f2b38af19d83

Last test of basis   125729  2018-08-01 11:00:39 Z7 days
Failing since125741  2018-08-02 10:01:09 Z6 days   19 attempts
Testing same since   125802  2018-08-08 11:00:47 Z0 days3 attempts


People who touched revisions under test:
  Alexandru Isaila 
  Andrew Cooper 
  Doug Goldstein 
  George Dunlap 
  Jan Beulich 
  Julien Grall 
  Kevin Tian 
  Razvan Cojocaru 
  Roger Pau Monné 
  Stefano Stabellini 
  Wei Liu 

jobs:
 build-arm64-xsm  broken  
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64-xsm broken
broken-step build-arm64-xsm hosts-allocate
broken-step build-arm64-xsm capture-logs

Not pushing.

(No revision log; it would be 512 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH] x86/entry/64: Remove %ebx handling from error_entry/exit

2018-08-08 Thread Sarah Newman
commit b3681dd548d06deb2e1573890829dff4b15abf46 upstream.

This version applies to v4.9.

From Andy Lutomirski, original author:

error_entry and error_exit communicate the user vs kernel status of
the frame using %ebx.  This is unnecessary -- the information is in
regs->cs.  Just use regs->cs.

This makes error_entry simpler and makes error_exit more robust.

It also fixes a nasty bug.  Before all the Spectre nonsense, The
xen_failsafe_callback entry point returned like this:

ALLOC_PT_GPREGS_ON_STACK
SAVE_C_REGS
SAVE_EXTRA_REGS
ENCODE_FRAME_POINTER
jmp error_exit

And it did not go through error_entry.  This was bogus: RBX
contained garbage, and error_exit expected a flag in RBX.
Fortunately, it generally contained *nonzero* garbage, so the
correct code path was used.  As part of the Spectre fixes, code was
added to clear RBX to mitigate certain speculation attacks.  Now,
depending on kernel configuration, RBX got zeroed and, when running
some Wine workloads, the kernel crashes.  This was introduced by:

commit 3ac6d8c787b8 ("x86/entry/64: Clear registers for
exceptions/interrupts, to reduce speculation attack surface")

With this patch applied, RBX is no longer needed as a flag, and the
problem goes away.

I suspect that malicious userspace could use this bug to crash the
kernel even without the offending patch applied, though.

[Historical note: I wrote this patch as a cleanup before I was aware
 of the bug it fixed.]

[Note to stable maintainers: this should probably get applied to all
 kernels.]

Cc: Brian Gerst 
Cc: Borislav Petkov 
Cc: Dominik Brodowski 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Thomas Gleixner 
Cc: Boris Ostrovsky 
Cc: Juergen Gross 
Cc: xen-devel@lists.xenproject.org
Cc: x...@kernel.org
Cc: sta...@vger.kernel.org
Cc: Andy Lutomirski 
Fixes: 3ac6d8c787b8 ("x86/entry/64: Clear registers for exceptions/interrupts, 
to reduce speculation attack surface")
Reported-and-tested-by: "M. Vefa Bicakci" 
Signed-off-by: Andy Lutomirski 
Signed-off-by: Sarah Newman 
---
 arch/x86/entry/entry_64.S | 19 ---
 1 file changed, 4 insertions(+), 15 deletions(-)

diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
index d58d8dc..0dab47a 100644
--- a/arch/x86/entry/entry_64.S
+++ b/arch/x86/entry/entry_64.S
@@ -774,7 +774,7 @@ ENTRY(\sym)
 
call\do_sym
 
-   jmp error_exit  /* %ebx: no swapgs flag */
+   jmp error_exit
.endif
 END(\sym)
 .endm
@@ -1043,7 +1043,6 @@ END(paranoid_exit)
 
 /*
  * Save all registers in pt_regs, and switch gs if needed.
- * Return: EBX=0: came from user mode; EBX=1: otherwise
  */
 ENTRY(error_entry)
cld
@@ -1087,7 +1086,6 @@ ENTRY(error_entry)
 * for these here too.
 */
 .Lerror_kernelspace:
-   incl%ebx
leaqnative_irq_return_iret(%rip), %rcx
cmpq%rcx, RIP+8(%rsp)
je  .Lerror_bad_iret
@@ -1119,28 +1117,19 @@ ENTRY(error_entry)
 
/*
 * Pretend that the exception came from user mode: set up pt_regs
-* as if we faulted immediately after IRET and clear EBX so that
-* error_exit knows that we will be returning to user mode.
+* as if we faulted immediately after IRET.
 */
mov %rsp, %rdi
callfixup_bad_iret
mov %rax, %rsp
-   decl%ebx
jmp .Lerror_entry_from_usermode_after_swapgs
 END(error_entry)
 
-
-/*
- * On entry, EBX is a "return to kernel mode" flag:
- *   1: already in kernel mode, don't need SWAPGS
- *   0: user gsbase is loaded, we need SWAPGS and standard preparation for 
return to usermode
- */
 ENTRY(error_exit)
-   movl%ebx, %eax
DISABLE_INTERRUPTS(CLBR_NONE)
TRACE_IRQS_OFF
-   testl   %eax, %eax
-   jnz retint_kernel
+   testb   $3, CS(%rsp)
+   jz  retint_kernel
jmp retint_user
 END(error_exit)
 
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] xen/balloon: fix balloon initialization for PVH Dom0

2018-08-08 Thread Boris Ostrovsky
On 08/08/2018 07:46 AM, Roger Pau Monne wrote:
> The current balloon code tries to calculate a delta factor for the
> balloon target when running in HVM mode in order to account for memory
> used by the firmware.
>
> This workaround for memory accounting doesn't work properly on a PVH
> Dom0, that has a static-max value different from the target value even
> at startup. Note that this is not a problem for DomUs because guests are
> started with a static-max value that matches the amount of RAM in the
> memory map.
>
> Fix this by forcefully setting target_diff for Dom0, regardless of
> it's mode.
>
> Reported-by: Gabriel Bercarug 
> Signed-off-by: Roger Pau Monné 
>


Applied to for-linus-4.19

-boris

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 3/3] x86/spec-ctrl: Add support for modifying SSBD AMD VIA LS_CFG MSR

2018-08-08 Thread Brian Woods
On Tue, Aug 07, 2018 at 01:51:07AM -0600, Jan Beulich wrote:
> >>> On 06.08.18 at 21:07,  wrote:
> > On Thu, Aug 02, 2018 at 01:09:06AM -0600, Jan Beulich wrote:
> >> >>> On 02.08.18 at 00:20,  wrote:
> >> > On Tue, Jul 31, 2018 at 05:25:27AM -0600, Jan Beulich wrote:
> >> >> Code structure wise this looks to undo a fair part of what patch
> >> >> 1 has done. It would be nice to limit code churn.
> >> > 
> >> > Patch 1 stand alone just to improve reporting the capabilities of the
> >> > processor.  Currently Xen doesn't mention anything if SSBD is actually
> >> > enable on AMD processors.  Patch 2-3 add it where Xen can it
> >> > dynamically.
> >> 
> >> Which is all fine, but couldn't patch 1 be written in a way that the
> >> next one(s) don't have to turn code structure all over again.
> > 
> > Rather than change 1, I changed patch 3 (well now patch 2).
> > 
> >> >> Why can't this be called from init_speculation_mitigations()?
> >> > 
> >> > IIRC I was having memory init problems with.  It was moved to later in
> >> > the boot so that xmalloc would work correctly.  I haven't touched this
> >> > code in over month.  If you want a 100% tested answer I can retest
> >> > putting it in init_speculation_mitigations().
> >> 
> >> Would be nice if that could be arranged for, as the rather specialized
> >> call looks pretty odd in (iirc __start_xen(); iirc because you've dropped
> >> too much of the context in your reply, and I'm too lazy to dig out the
> >> original patch again).
> > 
> > It was __start_xen().  Well, IIRC xalloc didn't work (well writing to
> > the address given) until after arch_init_memory().  For it to work in
> > init_speculation_mitigations(), I'd assume you'd need to call
> > arch_init_memory() before init_speculation_mitigations().
> 
> I don't think that's a viable option, but it could certainly be explored.
> I wonder though whether you can't get away without allocation at
> this early point.

Well, the thing is that you could use DECLARE_PER_CPU but then you
have to initialize it during each cpu start up, but two logical cpus
share a lock and only one needs to touch it etc.  I found it better to
just initialize everything on the boot cpu before others are brought
up, but the whole thing is a little messy.  (See the last comment.)

> >> >> Still I wonder whether less code duplication wouldn't be better.
> >> > 
> >> > current_cpu_data isn't available when ssbd_amd_ls_cfg_init is called.
> >> 
> >> By "isn't available" you mean "hasn't been populated yet"? Else
> >> I don't understand.
> > 
> > It hasn't been populated yet.
> 
> Not even boot_cpu_data?

boot_cpu_data is, but current_cpu_data and nr_sockets is not.

> >> >> I find such a hard-coded upper bound quite concerning. Is nr_sockets
> >> >> not correct in the AMD case? If so, that would want fixing, such that
> >> >> you can use the variable here.
> >> > 
> >> > It's been a while since I wrote this but IIRC, it has to do with
> >> > nr_sockets either being off or not available when the code is run.
> >> > For Family 17h which the patches are for, there's a max of two sockets.
> >> 
> >> I've implied that from how you wrote the patches, but such hard coding
> >> will only make for more code churn later on. Try to be as generic as
> >> possible.
> > 
> > Well, nr_sockets gets set in smp_prepare_cpus, which gets called after
> > init_speculation_mitigations() and ssbd_amd_ls_cfg_init().  It could be
> > put later on in the boot, but it needs to be run at least before other
> > cpus are brought online.  I'm welcome to any suggestions about how to
> > restructure things though.
> 
> Did you consider using a presmp-initcall?
> 
> Jan

I've been looking at it, seems like that could work.  You'd still need
something in start_secondary but it would take the init call out of
__start_xen().  I'll test that.


-- 
Brian Woods

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC] OVMF on PVH

2018-08-08 Thread Anthony PERARD
On Thu, Aug 02, 2018 at 12:24:35PM +0100, Anthony PERARD wrote:
> That replace a section of the OVMF binary called VarStore by that ELF
> header.  It's empty and libxl doesn't know how to use it. (QEMU/KVM would
> have a flash device loading the store, so it can be written to.)

> ## Loading binary at 0x1 (1MB, like hvmloader on HVM)

Mostly notes for myself:
I've realize that moving the initial binary location might also move the
VarStore (depending on how OVMF build is configured). The VarStore
location is used later by OVMF, if that location looks like a VarStore
(there is a header) then OVMF use it, otherwise OVMF simply initialise
it.

Anyway, just a reminder that I need to pay attention to where it is, as
maybe someone is trying to use it on HVM.

-- 
Anthony PERARD

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v4 4/4] x86/iommu: add reserved dom0-iommu option to map reserved memory ranges

2018-08-08 Thread Roger Pau Monné
On Wed, Aug 08, 2018 at 07:15:54AM -0600, Jan Beulich wrote:
> >>> On 08.08.18 at 12:07,  wrote:
> > Several people have reported hardware issues (malfunctioning USB
> > controllers) due to iommu page faults on Intel hardware. Those faults
> > are caused by missing RMRR (VTd) entries in the ACPI tables. Those can
> > be worked around on VTd hardware by manually adding RMRR entries on
> > the command line, this is however limited to Intel hardware and quite
> > cumbersome to do.
> > 
> > In order to solve those issues add a new dom0-iommu=reserved option
> > that identity maps all regions marked as reserved in the memory map.
> 
> Considering that report we've had (yesterday? earlier today?), don't
> we need to go further and use the union of RMRRs and reserved
> regions? Iirc they had a case where an RMRR was not in a reserved
> region ...

AFAICT (and I could be reading the code wrong) RMRR regions not on
reserved regions are still correctly mapped to the guest.

> > --- a/docs/misc/xen-command-line.markdown
> > +++ b/docs/misc/xen-command-line.markdown
> > @@ -1205,7 +1205,7 @@ detection of systems known to misbehave upon accesses 
> > to that port.
> >  >> Enable IOMMU debugging code (implies `verbose`).
> >  
> >  ### dom0-iommu
> > -> `= List of [ none | strict | relaxed | inclusive ]`
> > +> `= List of [ none | strict | relaxed | inclusive | reserved ]`
> >  
> >  * `none`: disables DMA remapping for Dom0.
> >  
> > @@ -1233,6 +1233,15 @@ meaning:
> >option is only applicable to a PV Dom0 and is enabled by default on Intel
> >hardware.
> >  
> > +* `reserved`: sets up DMA remapping for all the reserved regions in the  
> > memory
> > +  map for Dom0. Use this to work around firmware issues providing incorrect
> > +  RMRR/IVMD entries. Rather than only mapping RAM pages for IOMMU accesses
> > +  for Dom0, all memory regions marked as reserved in the memory map that 
> > don't
> > +  overlap with any MMIO region from emulated devices will be identity 
> > mapped.
> > +  This option maps a subset of the memory that would be mapped when using 
> > the
> > +  `inclusive` option. This option is available to a PVH Dom0 and is 
> > enabled by
> > +  default on Intel hardware.
> 
> The sub-options so far were quite clear in their meanings, but
> "dom0-iommu=reserved" might mean all sorts of things to me, but quite
> certainly not "map all reserved regions". "map-reserved" perhaps?

Then we should also have 'map-inclusive' for symmetry IMO.

> > --- a/xen/arch/x86/hvm/io.c
> > +++ b/xen/arch/x86/hvm/io.c
> > @@ -404,6 +404,11 @@ static const struct hvm_mmcfg *vpci_mmcfg_find(const 
> > struct domain *d,
> >  return NULL;
> >  }
> >  
> > +bool vpci_mmcfg_address(const struct domain *d, paddr_t addr)
> > +{
> > +return vpci_mmcfg_find(d, addr);
> > +}
> 
> I think the function name should have an "is_" somewhere, to make
> clear address is a noun here and not a verb.
> 
> > --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
> > +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
> > @@ -256,6 +256,9 @@ static void __hwdom_init amd_iommu_hwdom_init(struct 
> > domain *d)
> >  /* Inclusive IOMMU mappings are disabled by default on AMD hardware. */
> >  iommu_dom0_inclusive = iommu_dom0_inclusive == -1 ? false
> >: 
> > iommu_dom0_inclusive;
> > +/* Reserved IOMMU mappings are disabled by default on AMD hardware. */
> > +iommu_dom0_reserved = iommu_dom0_reserved == -1 ? false
> > +: iommu_dom0_reserved;
> 
> Especially seeing these two together now, I think an if() each would
> be easier to read (not the least because of being shorter). To me,
> the main purpose of the conditional operator is to allow shortening
> simple if/else pairs, rather than lengthening simple if()-s.
> 
> > @@ -134,13 +135,67 @@ void arch_iommu_domain_destroy(struct domain *d)
> >  {
> >  }
> >  
> > +static bool __hwdom_init hwdom_iommu_map(const struct domain *d, unsigned 
> > long pfn,
> > + unsigned long max_pfn)
> > +{
> > +unsigned int i;
> > +
> > +/*
> > + * Ignore any address below 1MB, that's already identity mapped by the
> > + * domain builder for HVM.
> > + */
> > +if ( (is_hvm_domain(d) && pfn < PFN_DOWN(MB(1))) ||
> 
> Careful again here with the distinction between Dom0 and hwdom:
> The domain builder you refer to is - aiui - the in-Xen one, i.e. the
> one _only_ dealing with Dom0.

So this should instead be:

if ( (is_control_domain(d) && is_hvm_domain(d) && pfn < PFN_DOWN(MB(1))) ||

> > +/*
> > + * If dom0-strict mode is enabled or the guest type is PVH/HVM then 
> > exclude
> > + * conventional RAM and let the common code map dom0's pages.
> > + */
> > +if ( page_is_ram_type(pfn, RAM_TYPE_CONVENTIONAL) &&
> > + (iommu_dom0_strict || is_hvm_domain(d)) )
> > +return false;
> > +if ( 

Re: [Xen-devel] [PATCH v4 2/4] iommu: make iommu_inclusive_mapping a suboption of dom0-iommu

2018-08-08 Thread Roger Pau Monné
On Wed, Aug 08, 2018 at 06:32:00AM -0600, Jan Beulich wrote:
> >>> On 08.08.18 at 12:07,  wrote:
> > Introduce a new dom0-iommu=inclusive generic option that supersedes
> > iommu_inclusive_mapping. The previous behaviour is preserved and the
> > option should only be enabled by default on Intel hardware.
> 
> Why "should" instead of "is"?
> 
> > @@ -1221,6 +1221,18 @@ PV Dom0:
> >  Note that all the above options are mutually exclusive. Specifying more 
> > than
> >  one on the `dom0-iommu` command line will result in undefined behavior.
> >  
> > +The following options control whether non-RAM regions are added to the Dom0
> > +iommu tables. Note that they can be prefixed with `no-` to effect the 
> > inverse
> > +meaning:
> 
> I'm not particularly happy about the mentioning of "no-" here: Why is
> this better than the also permitted "=0" etc suffixes? Keep it generic,
> just like other options do.

Oh, I've just copied this text from the iommu option. Should I change
this to:

"The following boolean options control whether non-RAM regions are
added to the Dom0 iommu tables:"

> > +* `inclusive`: sets up DMA remapping for all the non-RAM memory below 4GB
> > +  except for unusable ranges. Use this to work around firmware issues 
> > providing
> > +  incorrect RMRR/IVMD entries. Rather than only mapping RAM pages for IOMMU
> > +  accesses for Dom0, with this option all pages up to 4GB, not marked as
> > +  unusable in the E820 table, will get a mapping established. Note that 
> > this
> > +  option is only applicable to a PV Dom0 and is enabled by default on Intel
> > +  hardware.
> 
> No word at all about the interaction with none/strict/relaxed? I think,
> as mentioned for patch 1, "none" renders this option meaningless as
> well. But for "relaxed" it's pretty unclear, because from E820 alone
> you can't judge whether e.g. a reserved region is RAM or MMIO. (As
> an implication, the mentioning of RAM in patch 1's doc for "relaxed"
> then looks symmetrically wrong, just like I've already asked to replace
> "memory" by "RAM" for "strict".)

Hm, I'm not sure I follow. 'relaxed' refers to how the regions marked
as RAM in the memory map (E820_RAM) will be mapped to the guest.

The inclusive option OTOH refers to if/how non-RAM regions in the
memory map will be mapped to the guest. It doesn't matter if a
reserved region (E820_RESERVED) is actually backed by RAM or MMIO, it
will be mapped to the guest because it's a reserved region on the
memory map.

> > --- a/xen/drivers/passthrough/arm/iommu.c
> > +++ b/xen/drivers/passthrough/arm/iommu.c
> > @@ -73,3 +73,7 @@ int arch_iommu_populate_page_table(struct domain *d)
> >  /* The IOMMU shares the p2m with the CPU */
> >  return -ENOSYS;
> >  }
> > +
> > +void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
> > +{
> > +}
> 
> The option being in common code, I think you also need to set it for
> ARM, so it won't remain at its default of -1.
> 
> > @@ -144,16 +145,23 @@ static int __init parse_dom0_iommu_param(const char 
> > *s)
> >  int rc = 0;
> >  
> >  do {
> > +bool val = !!strncmp(s, "no-", 3);
> 
> Oh, you do a literal comparison against "no-". Please don't, that's what
> we have parse_boolean() for.

Thanks, I will fix it. I've mostly cloned what the iommu option
currently does.

> > @@ -202,6 +210,13 @@ void __hwdom_init iommu_hwdom_init(struct domain *d)
> >  if ( !iommu_enabled )
> >  return;
> >  
> > +if ( iommu_dom0_inclusive == true && !is_pv_domain(d) )
> 
> Why the "== true"? It shouldn't have its initializer value of -1 anymore
> at this point.

It can have a value of -1, AFAICT the default values are set by
hd->platform_ops->hwdom_init which is called later in this function.

Thanks, Roger.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC] OVMF on PVH

2018-08-08 Thread Anthony PERARD
On Thu, Aug 02, 2018 at 06:21:31PM +0200, Roger Pau Monné wrote:
> On Thu, Aug 02, 2018 at 12:24:35PM +0100, Anthony PERARD wrote:
> > ## New binary, separated from QEMU/KVM one.
> > 
> > Right now, the rules to build the firmware are located in
> > `OvmfPkg/OvmfPkgX64.dsc`, a platform. I've created a new platform, without
> > KVM support, and would like to retire the Xen support from the current
> > platform.  For now, I have created OvmfPkg/XenOvmf.dsc. (The change to a
> > new platform had been proposed by upstream.)
> 
> Is it my understanding that after this there will be a generic
> platform (for QEMU, KVM and everything else) and a platform for Xen?

It's not really generic. As far as I can tell, it's only QEMU/KVM (and
Xen now). But yes, there would be a separate platform (or file when
built) for Xen.

It would be possible to keep Xen support in the current "generic"
platform, but that would mean that code would be duplicated.

I think the OVMF maintainer would be happier if the Xen support was
separated, even now without PVH.

> > That would simplify Xen support in OVMF, and allow to diverge more from
> > OVMF where needed. That would mostly be useful during the initial boot
> > phase where there lots of `if xen do that, else do that`.
> 
> I fear this might be dangerous because most developers will only test
> against the generic platform, so breakage could be introduced more
> easily to the Xen platform without upstream noticing?

That would not change anything. Upstream maintainers doesn't test booting
OVMF in a Xen guest so there are sometime where only Xen is broken, even
if they try hard to not break it.

As for build, I actually don't know if that would change anything.
Anyway, we've got osstest.

> > We can also choose some drivers that can work on both PVH and HVM, for
> > example, use a LAPIC timer instead of ACPI Timer.
> > 
> > All this mean that building OVMF for Xen would need to be change.
> 
> And distros will likely have to provide two different packages, or a
> single package with the generic binary and the Xen specific one.

Yes. But it is already an issue sometime. For example, I cann't use the
ovmf package from Arch Linux because it is cut in two part (vars and
code), and Xen only works when both part are together (/bin/cat vars
code).

> > 
> > ## ELF header
> > 
> > Adding an ELF header to OVMF so the blob can be loaded by libxl/libxc
> > without modifying those libs.
> > 
> > That replace a section of the OVMF binary called VarStore by that ELF
> > header.  It's empty and libxl doesn't know how to use it. (QEMU/KVM would
> > have a flash device loading the store, so it can be written to.)
> 
> Can't you just add the header without replacing anything? The ELF
> header is quite small, so I don't think size should be a problem (if
> that's why it's placed on top of the VarStore).

I don't think hvmloader can cope with a different size right now. And
that section doesn't contain anything useful, it would need to be
separated in order to be useful.

> > With the ELF header, I can test OVMF by simply adding this in the config
> > file:
> > 
> > type='pvh'
> > kernel='/path/to/OVMF.fd'
> 
> Yes, that's the expectation and is what firmware='ovmf|uefi' should
> do behind the scenes.
> 
> > We could use a modified `hvmloader` to load OVMF on PVH or teach libxc to
> > load it at a hardcoded location, but I don't see it as necessary, and if
> > we have one less firmware to load, it's probably better.
> > 
> > With the use of an ELF header comes another issue, which as to do with
> > were the binary needs to be loaded initially.
> > 
> > ## Loading binary at 0x1 (1MB, like hvmloader on HVM)
> 
> That works well for hvmloader because after it has finished running
> the memory can be overwritten without issues. The same is not true for
> ovmf which needs to keep data around so that boot services and runtime
> services can continue working.
> 
> > Right now, the OVMF blob is loaded (by hvmloader) just below 4GB, with
> > hvmloader moving some RAM around to allow that.  But with the use of an
> > ELF header, and let libxc load the blob, it was not possible to load the
> > blob just below 4GB. (Or at least without modification of the toolstack I
> > guess.)
> 
> libxc/libelf will load the binary in the position specified by the ELF
> program headers PhysAddr field (also called LMA). If you execute
> `readelf -l ` you should see the program headers and the
> address where they should be loaded.
> 
> You can easily modify the LMA in the linker script using the AT
> keyword.

I don't know what to answer to that. OVMF isn't an ELF. There isn't an
easy way to tell where it should be loaded. There is no linker script
(at least that link everything together), but a collection python script
and some program, and configuration. I don't need to run `readelf`
because it either doesn't work or it would print the value that I've set
manually.

The issue it that, if I set LMA to 0xffe0, 

Re: [Xen-devel] [PATCH v4 1/4] iommu: introduce dom0-iommu option

2018-08-08 Thread Roger Pau Monné
On Wed, Aug 08, 2018 at 06:10:39AM -0600, Jan Beulich wrote:
> >>> On 08.08.18 at 12:07,  wrote:
> > +Note that all the above options are mutually exclusive. Specifying more 
> > than
> > +one on the `dom0-iommu` command line will result in undefined behavior.
> 
> Isn't this more strict than it needs to be? "none", afaict, always takes
> precedence if enabled. What color a bike shed is simply doesn't matter
> when it doesn't exist.

Right, that's due to the current implementation and the way this is
stored, but I don't think we want to spell out any of this in order to
not give any guarantees. For example:

dom0-iommu=none,relaxed

Shouldn't be used, albeit with the current implementation relaxed will
be basically ignored I don't think we want to write this down
anywhere because people shouldn't rely on this behavior.

> > --- a/xen/arch/x86/x86_64/mm.c
> > +++ b/xen/arch/x86/x86_64/mm.c
> > @@ -1426,7 +1426,8 @@ int memory_add(unsigned long spfn, unsigned long 
> > epfn, unsigned int pxm)
> >  if ( ret )
> >  goto destroy_m2p;
> >  
> > -if ( iommu_enabled && !iommu_passthrough && 
> > !need_iommu(hardware_domain) )
> > +if ( iommu_enabled && !iommu_dom0_passthrough &&
> > + !need_iommu(hardware_domain) )
> 
> This makes already clear that you need to better distinguish Dom0 and
> hwdom here, but it's not immediately clear to me which direction the
> changes should be made: Do you mean truly only Dom0 throughout
> this patch, or hwdom? While the doc and command line option name can
> perhaps left as is, internal variable names should not say Dom0 when
> they don't mean Dom0. Otoh if you mean only Dom0, then the use of
> hardware_domain above (and elsewhere) is now wrong.

Hm, everything is kind of mixed here. Existing variables already use
_dom0_ (iommu_dom0_strict for example). I can rename them to
iommu_hwdom_, because AFAICT this applies to the hardware domain.

> > +static int __init parse_dom0_iommu_param(const char *s)
> > +{
> > +const char *ss;
> > +int rc = 0;
> > +
> > +do {
> > +ss = strchr(s, ',');
> > +if ( !ss )
> > +ss = strchr(s, '\0');
> > +
> > +if ( !strncmp(s, "none", ss - s) )
> > +iommu_dom0_passthrough = true;
> > +else if ( !strncmp(s, "strict", ss - s) )
> > +iommu_dom0_strict = true;
> > +else if ( !strncmp(s, "relaxed", ss - s) )
> > +iommu_dom0_strict = false;
> 
> Perhaps better just have one of the two, and make them truly
> boolean? Or would that conflict with further patches / plans?

I don't think this will cause a lot of conflicts, some rebasing
issues but no big deal. I've used this syntax as discussed
in a previous version and agreed with Paul and Kevin. I'm OK with
this, and I think it's clear, but I don't have a strong opinion so if
you think this is not clear I can change it.

I would just like to reach a consensus on the nomenclature of the
option ASAP, so the bikeshed can be as small as possible :).

Thanks, Roger.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 4/5] xen/blkback: move persistent grants flags to bool

2018-08-08 Thread Roger Pau Monné
On Wed, Aug 08, 2018 at 04:25:27PM +0200, Juergen Gross wrote:
> The struct persistent_gnt flags member is meant to be a bitfield of
> different flags. There is only PERSISTENT_GNT_ACTIVE flag left, so
> convert it to a bool named "active".
> 
> Signed-off-by: Juergen Gross 

Reviewed-by: Roger Pau Monné 

Thanks.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 2/5] xen/blkfront: cleanup stale persistent grants

2018-08-08 Thread Roger Pau Monné
On Wed, Aug 08, 2018 at 04:25:25PM +0200, Juergen Gross wrote:
> Add a periodic cleanup function to remove old persistent grants which
> are no longer in use on the backend side. This avoids starvation in
> case there are lots of persistent grants for a device which no longer
> is involved in I/O business.
> 
> Signed-off-by: Juergen Gross 

Reviewed-by: Roger Pau Monné 

Thanks, Roger.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 1/5] xen/blkback: don't keep persistent grants too long

2018-08-08 Thread Roger Pau Monné
On Wed, Aug 08, 2018 at 04:25:24PM +0200, Juergen Gross wrote:
> Persistent grants are allocated until a threshold per ring is being
> reached. Those grants won't be freed until the ring is being destroyed
> meaning there will be resources kept busy which might no longer be
> used.
> 
> Instead of freeing only persistent grants until the threshold is
> reached add a timestamp and remove all persistent grants not having
> been in use for a minute.
> 
> Signed-off-by: Juergen Gross 

Reviewed-by: Roger Pau Monné 

Thanks, Roger.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC] OVMF on PVH

2018-08-08 Thread Roger Pau Monné
On Wed, Aug 08, 2018 at 04:02:09PM +0100, Anthony PERARD wrote:
> On Thu, Aug 02, 2018 at 04:44:56PM +0100, Wei Liu wrote:
> > On Thu, Aug 02, 2018 at 12:24:35PM +0100, Anthony PERARD wrote:
> > > ## Loading binary at 0x1 (1MB, like hvmloader on HVM)
> > > 
> > > Right now, the OVMF blob is loaded (by hvmloader) just below 4GB, with
> > > hvmloader moving some RAM around to allow that.  But with the use of an
> > > ELF header, and let libxc load the blob, it was not possible to load the
> > > blob just below 4GB. (Or at least without modification of the toolstack I
> > > guess.) So I've attempted to have OVMF been started from a different place
> > > in memory. Then I had to hack a bit more in order to be able to start OVMF
> > > from two different location (the current one for HVM guest, and a new one
> > > that can be used for PVH). That works fine, I'll just have to find out
> > > what upstream think about that.
> > 
> > If you specify the virt_addr (?) in the ELF header, libxc should know
> > how to deal with that? What is missing in the current toolstack code?
> 
> RAM.
> 
> If I set the ELF header properly so that it is written that the blob
> must be loaded in the guest RAM just below 4G, libxc will not be able to
> do anything because there is no RAM, that space is suppose to be a mem
> hole, or something like that, I forgot the exact detail :(. I think also
> that libxl would try to load other data like the startinfo page past the
> kernel (OVMF here) and that would mean past 4G.
> 
> (In my notes from when I tried, I've written "libxc will not move ram
> like hvmloader would; [and with >4G for the guest] libxc can not map
> anything past 0xfee0, and I want 0xffe0", whatever that mean,
> can find out where I found that 0xfee0 address.)

This is arguably a bug of the current libxc implementation. libxc
should always populate the place where the kernel will be loaded
first, and then populate the rest of the memory map using the
remaining memory.

Given your other reply that mentions that the initial load region is
discarded afterwards because ovmf relocates itself I think I'm OK with
loading it wherever it's easier.

Thanks, Roger.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2] xen: move pv irq related functions under CONFIG_XEN_PV umbrella

2018-08-08 Thread Boris Ostrovsky
x86 maintainers, this needs you ack too even though it has "xen" tag in
the subject, thanks.


On 08/08/2018 02:19 AM, Juergen Gross wrote:
> All functions in arch/x86/xen/irq.c and arch/x86/xen/xen-asm*.S are
> specific to PV guests. Include them in the kernel with
> CONFIG_XEN_PV only.
>
> Make the PV specific code in arch/x86/entry/entry_*.S dependent on
> CONFIG_XEN_PV instead of CONFIG_XEN.
>
> The HVM specific code should depend on CONFIG_XEN_PVHVM.
>
> While at it reformat the Makefile to make it more readable.
>
> Signed-off-by: Juergen Gross 


Reviewed-by: Boris Ostrovsky 


> ---
>  arch/x86/entry/entry_32.S |  8 +---
>  arch/x86/entry/entry_64.S |  8 +---
>  arch/x86/xen/Makefile | 41 +++--
>  include/xen/events.h  |  2 ++
>  4 files changed, 43 insertions(+), 16 deletions(-)
>
> diff --git a/arch/x86/entry/entry_32.S b/arch/x86/entry/entry_32.S
> index c371bfee137a..cc4ec4fd58d2 100644
> --- a/arch/x86/entry/entry_32.S
> +++ b/arch/x86/entry/entry_32.S
> @@ -369,7 +369,7 @@ GLOBAL(__begin_SYSENTER_singlestep_region)
>   * will ignore all of the single-step traps generated in this range.
>   */
>  
> -#ifdef CONFIG_XEN
> +#ifdef CONFIG_XEN_PV
>  /*
>   * Xen doesn't set %esp to be precisely what the normal SYSENTER
>   * entry point expects, so fix it up before using the normal path.
> @@ -807,7 +807,7 @@ ENTRY(spurious_interrupt_bug)
>   jmp common_exception
>  END(spurious_interrupt_bug)
>  
> -#ifdef CONFIG_XEN
> +#ifdef CONFIG_XEN_PV
>  ENTRY(xen_hypervisor_callback)
>   pushl   $-1 /* orig_ax = -1 => not a system 
> call */
>   SAVE_ALL
> @@ -888,11 +888,13 @@ ENTRY(xen_failsafe_callback)
>   _ASM_EXTABLE(3b, 8b)
>   _ASM_EXTABLE(4b, 9b)
>  ENDPROC(xen_failsafe_callback)
> +#endif /* CONFIG_XEN_PV */
>  
> +#ifdef CONFIG_XEN_PVHVM
>  BUILD_INTERRUPT3(xen_hvm_callback_vector, HYPERVISOR_CALLBACK_VECTOR,
>xen_evtchn_do_upcall)
> +#endif
>  
> -#endif /* CONFIG_XEN */
>  
>  #if IS_ENABLED(CONFIG_HYPERV)
>  
> diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
> index 8ae7ffda8f98..3c2526e24dd8 100644
> --- a/arch/x86/entry/entry_64.S
> +++ b/arch/x86/entry/entry_64.S
> @@ -1049,7 +1049,7 @@ ENTRY(do_softirq_own_stack)
>   ret
>  ENDPROC(do_softirq_own_stack)
>  
> -#ifdef CONFIG_XEN
> +#ifdef CONFIG_XEN_PV
>  idtentry hypervisor_callback xen_do_hypervisor_callback has_error_code=0
>  
>  /*
> @@ -1129,11 +1129,13 @@ ENTRY(xen_failsafe_callback)
>   ENCODE_FRAME_POINTER
>   jmp error_exit
>  END(xen_failsafe_callback)
> +#endif /* CONFIG_XEN_PV */
>  
> +#ifdef CONFIG_XEN_PVHVM
>  apicinterrupt3 HYPERVISOR_CALLBACK_VECTOR \
>   xen_hvm_callback_vector xen_evtchn_do_upcall
> +#endif
>  
> -#endif /* CONFIG_XEN */
>  
>  #if IS_ENABLED(CONFIG_HYPERV)
>  apicinterrupt3 HYPERVISOR_CALLBACK_VECTOR \
> @@ -1150,7 +1152,7 @@ idtentry debug  do_debug
> has_error_code=0paranoid=1 shift_ist=DEBUG_STACK
>  idtentry int3do_int3 has_error_code=0
>  idtentry stack_segment   do_stack_segmenthas_error_code=1
>  
> -#ifdef CONFIG_XEN
> +#ifdef CONFIG_XEN_PV
>  idtentry xennmi  do_nmi  has_error_code=0
>  idtentry xendebugdo_debughas_error_code=0
>  idtentry xenint3 do_int3 has_error_code=0
> diff --git a/arch/x86/xen/Makefile b/arch/x86/xen/Makefile
> index d83cb5478f54..f723b5aa8f74 100644
> --- a/arch/x86/xen/Makefile
> +++ b/arch/x86/xen/Makefile
> @@ -12,25 +12,46 @@ endif
>  # Make sure early boot has no stackprotector
>  nostackp := $(call cc-option, -fno-stack-protector)
>  CFLAGS_enlighten_pv.o:= $(nostackp)
> -CFLAGS_mmu_pv.o  := $(nostackp)
> +CFLAGS_mmu_pv.o  := $(nostackp)
>  
> -obj-y:= enlighten.o multicalls.o mmu.o irq.o \
> - time.o xen-asm.o xen-asm_$(BITS).o \
> - grant-table.o suspend.o platform-pci-unplug.o
> +obj-y+= enlighten.o
> +obj-y+= multicalls.o
> +obj-y+= mmu.o
> +obj-y+= time.o
> +obj-y+= grant-table.o
> +obj-y+= suspend.o
> +obj-y+= platform-pci-unplug.o
>  
> -obj-$(CONFIG_XEN_PVHVM)  += enlighten_hvm.o mmu_hvm.o 
> suspend_hvm.o
> -obj-$(CONFIG_XEN_PV) += setup.o apic.o pmu.o suspend_pv.o \
> - p2m.o enlighten_pv.o mmu_pv.o
> -obj-$(CONFIG_XEN_PVH)+= enlighten_pvh.o
> +obj-$(CONFIG_XEN_PVHVM)  += enlighten_hvm.o
> +obj-$(CONFIG_XEN_PVHVM)  += mmu_hvm.o
> +obj-$(CONFIG_XEN_PVHVM)  += 

[Xen-devel] [qemu-mainline test] 125785: trouble: blocked/broken/fail/pass

2018-08-08 Thread osstest service owner
flight 125785 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/125785/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-pvopsbroken
 build-arm64-xsm  broken
 build-arm64  broken

Regressions which are regarded as allowable (not blocking):
 build-arm64-pvops 2 hosts-allocate broken REGR. vs. 125640
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 125640
 build-arm64   2 hosts-allocate broken REGR. vs. 125640

Tests which did not succeed, but are not blocking:
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 build-arm64-xsm   3 capture-logs  broken blocked in 125640
 build-arm64-pvops 3 capture-logs  broken blocked in 125640
 build-arm64   3 capture-logs  broken blocked in 125640
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 125640
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 125640
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 125640
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 125640
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 125640
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass

version targeted for testing:
 qemuu09d98b69804cfd9eccadb97951e8df64ada3bc7a
baseline version:
 qemuu18a398f6a39df4b08ff86ac0d38384193ca5f4cc

Last test of basis   125640  2018-07-27 22:16:44 Z   11 days
Failing since125675  2018-07-30 09:36:58 Z9 days6 attempts
Testing same since   125785  2018-08-07 11:08:43 Z1 days1 attempts


People who touched revisions under test:
  Alex Bennée 
  BALATON Zoltan 
  Christian Borntraeger 
  Cornelia Huck 
  David Gibson 
  Dou Liyang 
  Dr. David Alan Gilbert 
  Fam Zheng 
  Geert Uytterhoeven 
  Igor Mammedov 
  Jay Zhou 
  Kevin Wolf 
  KONRAD Frederic 
  Laurent Desnogues 
  Laurent Vivier 
  Leonid Bloch 
  liujunjie 
  Marc-André Lureau 
  Markus Armbruster 
  Max Filippov 
  Michael S. Tsirkin 
  Paolo Bonzini 
  Pavel Dovgalyuk 
  Peter Maydell 
  Philippe Mathieu-Daudé 
  Richard Henderson 
  

Re: [Xen-devel] [RFC] OVMF on PVH

2018-08-08 Thread Anthony PERARD
On Thu, Aug 02, 2018 at 04:44:56PM +0100, Wei Liu wrote:
> On Thu, Aug 02, 2018 at 12:24:35PM +0100, Anthony PERARD wrote:
> > ## New binary, separated from QEMU/KVM one.
> > 
> > Right now, the rules to build the firmware are located in
> > `OvmfPkg/OvmfPkgX64.dsc`, a platform. I've created a new platform, without
> > KVM support, and would like to retire the Xen support from the current
> > platform.  For now, I have created OvmfPkg/XenOvmf.dsc. (The change to a
> > new platform had been proposed by upstream.)
> > 
> > That would simplify Xen support in OVMF, and allow to diverge more from
> > OVMF where needed. That would mostly be useful during the initial boot
> > phase where there lots of `if xen do that, else do that`.
> > 
> > We can also choose some drivers that can work on both PVH and HVM, for
> > example, use a LAPIC timer instead of ACPI Timer.
> > 
> > All this mean that building OVMF for Xen would need to be change.
> 
> Changing things in xen.git is easy and fine. This is more about
> introducing a burden for downstream packagers because they need to build
> a separate packages.

Yeah, that would be a bit annoying for downstream. Maybe we could keep
in the current OVMF implementation for a little while longer and create
the new one which would be Xen specific, but which would introduce PVH
support. That would mean duplicated code upstream.

I don't think it would be possible to add PVH to the current OVMF.

> This is not saying I'm against this idea in any way, just something that
> comes to my mind.
> 
> > 
> > 
> > ## ELF header
> > 
> > Adding an ELF header to OVMF so the blob can be loaded by libxl/libxc
> > without modifying those libs.
> > 
> > That replace a section of the OVMF binary called VarStore by that ELF
> > header.  It's empty and libxl doesn't know how to use it. (QEMU/KVM would
> > have a flash device loading the store, so it can be written to.)
> 
> But long term we might want to support VarStore (that is used to stored
> config, right?). How are we going to do that in the future if we
> repurpose it now?

So, normally on QEMU/KVM, in order to use this VarStore, libvirt (or
user) would add two flash device to qemu, one which gives access to this
VarStore that a guest can write to it (and so would be baked by a file
in the host), and the second flash device would be the rest of OVMF, all
its code.

In Xen, we only ever load the sum of both, and I think hvmloader might
not work if the OVMF blob is smaller than expected (meaning the
toolstack load ovmf without the VarStore so it can attach to QEMU
instead).

I think that VarStore is just a placeholder anyway, in the current
binary.

> > 
> > With the ELF header, I can test OVMF by simply adding this in the config
> > file:
> > 
> > type='pvh'
> > kernel='/path/to/OVMF.fd'
> > 
> > We could use a modified `hvmloader` to load OVMF on PVH or teach libxc to
> > load it at a hardcoded location, but I don't see it as necessary, and if
> > we have one less firmware to load, it's probably better.
> > 
> > With the use of an ELF header comes another issue, which as to do with
> > were the binary needs to be loaded initially.
> > 
> > ## Loading binary at 0x1 (1MB, like hvmloader on HVM)
> > 
> > Right now, the OVMF blob is loaded (by hvmloader) just below 4GB, with
> > hvmloader moving some RAM around to allow that.  But with the use of an
> > ELF header, and let libxc load the blob, it was not possible to load the
> > blob just below 4GB. (Or at least without modification of the toolstack I
> > guess.) So I've attempted to have OVMF been started from a different place
> > in memory. Then I had to hack a bit more in order to be able to start OVMF
> > from two different location (the current one for HVM guest, and a new one
> > that can be used for PVH). That works fine, I'll just have to find out
> > what upstream think about that.
> 
> If you specify the virt_addr (?) in the ELF header, libxc should know
> how to deal with that? What is missing in the current toolstack code?

RAM.

If I set the ELF header properly so that it is written that the blob
must be loaded in the guest RAM just below 4G, libxc will not be able to
do anything because there is no RAM, that space is suppose to be a mem
hole, or something like that, I forgot the exact detail :(. I think also
that libxl would try to load other data like the startinfo page past the
kernel (OVMF here) and that would mean past 4G.

(In my notes from when I tried, I've written "libxc will not move ram
like hvmloader would; [and with >4G for the guest] libxc can not map
anything past 0xfee0, and I want 0xffe0", whatever that mean,
can find out where I found that 0xfee0 address.)

Anyway, it didn't make sense to be to move the RAM around for some
code/data that is only use very early and never used again. And it seams
easier to have initial blob somewhere else rather trying to teach libxc
about this weirdness.

Thanks,

-- 
Anthony PERARD


Re: [Xen-devel] Emulation and active (valid) interrupts

2018-08-08 Thread Razvan Cojocaru
> 2. Is it possible that something else in that path writes into
> VM_ENTRY_INTR_INFO (which the vmx_intr_assist() logic can't possibly
> prevent)? For example, in my Xen 4.7.5 sources, there's a
> pt_restore_timer(v); call in hvm_do_resume() before the vm_event
> emulation code.

Actually even handle_hvm_io_completion(v) could theoretically cause a
write into VM_ENTRY_INTR_INFO, because it emulates. I'm not sure how
probable this is, but theoretically the same issue with vm_event
emulation applies here: external interrupts could be lost.


Thanks,
Razvan

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 3/5] xen/blkfront: reorder tests in xlblk_init()

2018-08-08 Thread Juergen Gross
In case we don't want pv block devices we should not test parameters
for sanity and eventually print out error messages. So test precluding
conditions before checking parameters.

Signed-off-by: Juergen Gross 
Reviewed-by: Roger Pau Monné 
---
 drivers/block/xen-blkfront.c | 18 +-
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
index 7c4b1b9d3971..36e4ca41e765 100644
--- a/drivers/block/xen-blkfront.c
+++ b/drivers/block/xen-blkfront.c
@@ -2713,6 +2713,15 @@ static int __init xlblk_init(void)
if (!xen_domain())
return -ENODEV;
 
+   if (!xen_has_pv_disk_devices())
+   return -ENODEV;
+
+   if (register_blkdev(XENVBD_MAJOR, DEV_NAME)) {
+   pr_warn("xen_blk: can't get major %d with name %s\n",
+   XENVBD_MAJOR, DEV_NAME);
+   return -ENODEV;
+   }
+
if (xen_blkif_max_segments < BLKIF_MAX_SEGMENTS_PER_REQUEST)
xen_blkif_max_segments = BLKIF_MAX_SEGMENTS_PER_REQUEST;
 
@@ -2728,15 +2737,6 @@ static int __init xlblk_init(void)
xen_blkif_max_queues = nr_cpus;
}
 
-   if (!xen_has_pv_disk_devices())
-   return -ENODEV;
-
-   if (register_blkdev(XENVBD_MAJOR, DEV_NAME)) {
-   printk(KERN_WARNING "xen_blk: can't get major %d with name 
%s\n",
-  XENVBD_MAJOR, DEV_NAME);
-   return -ENODEV;
-   }
-
INIT_DELAYED_WORK(_work, blkfront_delay_work);
 
ret = xenbus_register_frontend(_driver);
-- 
2.13.7


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 2/5] xen/blkfront: cleanup stale persistent grants

2018-08-08 Thread Juergen Gross
Add a periodic cleanup function to remove old persistent grants which
are no longer in use on the backend side. This avoids starvation in
case there are lots of persistent grants for a device which no longer
is involved in I/O business.

Signed-off-by: Juergen Gross 
---
 drivers/block/xen-blkfront.c | 94 ++--
 1 file changed, 90 insertions(+), 4 deletions(-)

diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
index b5cedccb5d7d..7c4b1b9d3971 100644
--- a/drivers/block/xen-blkfront.c
+++ b/drivers/block/xen-blkfront.c
@@ -46,6 +46,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include 
 #include 
@@ -121,6 +122,8 @@ static inline struct blkif_req *blkif_req(struct request 
*rq)
 
 static DEFINE_MUTEX(blkfront_mutex);
 static const struct block_device_operations xlvbd_block_fops;
+static struct delayed_work blkfront_work;
+static LIST_HEAD(info_list);
 
 /*
  * Maximum number of segments in indirect requests, the actual value used by
@@ -216,6 +219,7 @@ struct blkfront_info
/* Save uncomplete reqs and bios for migration. */
struct list_head requests;
struct bio_list bio_list;
+   struct list_head info_list;
 };
 
 static unsigned int nr_minors;
@@ -1764,6 +1768,12 @@ static int write_per_ring_nodes(struct 
xenbus_transaction xbt,
return err;
 }
 
+static void free_info(struct blkfront_info *info)
+{
+   list_del(>info_list);
+   kfree(info);
+}
+
 /* Common code used when first setting up, and when resuming. */
 static int talk_to_blkback(struct xenbus_device *dev,
   struct blkfront_info *info)
@@ -1885,7 +1895,10 @@ static int talk_to_blkback(struct xenbus_device *dev,
  destroy_blkring:
blkif_free(info, 0);
 
-   kfree(info);
+   mutex_lock(_mutex);
+   free_info(info);
+   mutex_unlock(_mutex);
+
dev_set_drvdata(>dev, NULL);
 
return err;
@@ -1996,6 +2009,10 @@ static int blkfront_probe(struct xenbus_device *dev,
info->handle = simple_strtoul(strrchr(dev->nodename, '/')+1, NULL, 0);
dev_set_drvdata(>dev, info);
 
+   mutex_lock(_mutex);
+   list_add(>info_list, _list);
+   mutex_unlock(_mutex);
+
return 0;
 }
 
@@ -2306,6 +2323,12 @@ static void blkfront_gather_backend_features(struct 
blkfront_info *info)
if (indirect_segments <= BLKIF_MAX_SEGMENTS_PER_REQUEST)
indirect_segments = 0;
info->max_indirect_segments = indirect_segments;
+
+   if (info->feature_persistent) {
+   mutex_lock(_mutex);
+   schedule_delayed_work(_work, HZ * 10);
+   mutex_unlock(_mutex);
+   }
 }
 
 /*
@@ -2487,7 +2510,9 @@ static int blkfront_remove(struct xenbus_device *xbdev)
mutex_unlock(>mutex);
 
if (!bdev) {
-   kfree(info);
+   mutex_lock(_mutex);
+   free_info(info);
+   mutex_unlock(_mutex);
return 0;
}
 
@@ -2507,7 +2532,9 @@ static int blkfront_remove(struct xenbus_device *xbdev)
if (info && !bdev->bd_openers) {
xlvbd_release_gendisk(info);
disk->private_data = NULL;
-   kfree(info);
+   mutex_lock(_mutex);
+   free_info(info);
+   mutex_unlock(_mutex);
}
 
mutex_unlock(>bd_mutex);
@@ -2590,7 +2617,7 @@ static void blkif_release(struct gendisk *disk, fmode_t 
mode)
dev_info(disk_to_dev(bdev->bd_disk), "releasing disk\n");
xlvbd_release_gendisk(info);
disk->private_data = NULL;
-   kfree(info);
+   free_info(info);
}
 
 out:
@@ -2623,6 +2650,61 @@ static struct xenbus_driver blkfront_driver = {
.is_ready = blkfront_is_ready,
 };
 
+static void purge_persistent_grants(struct blkfront_info *info)
+{
+   unsigned int i;
+   unsigned long flags;
+
+   for (i = 0; i < info->nr_rings; i++) {
+   struct blkfront_ring_info *rinfo = >rinfo[i];
+   struct grant *gnt_list_entry, *tmp;
+
+   spin_lock_irqsave(>ring_lock, flags);
+
+   if (rinfo->persistent_gnts_c == 0) {
+   spin_unlock_irqrestore(>ring_lock, flags);
+   continue;
+   }
+
+   list_for_each_entry_safe(gnt_list_entry, tmp, >grants,
+node) {
+   if (gnt_list_entry->gref == GRANT_INVALID_REF ||
+   gnttab_query_foreign_access(gnt_list_entry->gref))
+   continue;
+
+   list_del(_list_entry->node);
+   gnttab_end_foreign_access(gnt_list_entry->gref, 0, 0UL);
+   rinfo->persistent_gnts_c--;
+   __free_page(gnt_list_entry->page);
+   kfree(gnt_list_entry);
+   }
+
+

[Xen-devel] Emulation and active (valid) interrupts

2018-08-08 Thread Razvan Cojocaru
Hello,

I'm seeing rare occasions where this backtrace occurs at a point in
__vmx_inject_exception() where there's already a valid interrupt set up
in VM_ENTRY_INTR_INFO (that is, once the value is __vmread(), intr_info
& INTR_INFO_VALID_MASK is non-zero):

Xen call trace:
   [] vmx.c#__vmx_inject_exception+0x47/0xd9
   [] vmx.c#vmx_inject_trap+0x1e1/0x29f
   [] hvm_inject_trap+0xb0/0xb5
   [] hvm_inject_page_fault+0x2a/0x2c
   [] hvm.c#__hvm_copy+0xdd/0x37f
   [] hvm_copy_from_guest_virt+0x14/0x16
   [] emulate.c#__hvmemul_read+0x12f/0x19f
   [] emulate.c#hvmemul_read+0x28/0x2a
   [] x86_emulate.c#read_ulong+0x13/0x15
   [] x86_emulate+0x16b1/0x11405
   [] emulate.c#_hvm_emulate_one+0x188/0x2bc
   [] hvm_emulate_one+0x10/0x12
   [] hvm_mem_access_emulate_one+0x7a/0xdd
   [] hvm_do_resume+0x246/0x3d3
   [] vmx_do_resume+0x102/0x119
   [] context_switch+0xf52/0xfad
   [] schedule.c#schedule+0x753/0x792
   [] softirq.c#__do_softirq+0x85/0x90
   [] do_softirq+0x13/0x15
   [] domain.c#idle_loop+0x61/0x6e

(this is a Xen 4.7.5 trace, but it applies to staging as well).

This was the initial culprit:

[] vmx.c#__vmx_inject_exception+0xa1/0xda
[] vmx_inject_extint+0x94/0x9f
[] vmx_intr_assist+0x4ee/0x5ad
[] vmx_asm_vmexit_handler+0xff/0x270

and I thought I'd fixed it by treating the emulation in a similar manner
to the single-step case: have vmx_intr_assist() block interrupts for the
duration of the emulation (please see the attached patch for staging).
However, a bit more rarely this time, we still end up overwriting an
interrupt via the above code path.

Obviously this works only if nothing has been written in
VM_ENTRY_INTR_INFO _before_ we block (return) in vmx_intr_assist().

My questions are:

1. Is it possible to already have a valid interrupt written in
VM_ENTRY_INTR_INFO at EXIT_REASON_EPT_VIOLATION-time in
vmx_vmexit_handler()?

2. Is it possible that something else in that path writes into
VM_ENTRY_INTR_INFO (which the vmx_intr_assist() logic can't possibly
prevent)? For example, in my Xen 4.7.5 sources, there's a
pt_restore_timer(v); call in hvm_do_resume() before the vm_event
emulation code.


Thanks,
Razvan
diff --git a/xen/arch/x86/hvm/vm_event.c b/xen/arch/x86/hvm/vm_event.c
index 28d08a6..8e7c737 100644
--- a/xen/arch/x86/hvm/vm_event.c
+++ b/xen/arch/x86/hvm/vm_event.c
@@ -124,6 +124,8 @@ void hvm_vm_event_do_resume(struct vcpu *v)
 
 w->do_write.msr = 0;
 }
+
+v->arch.vm_event->intr_block = false;
 }
 
 /*
diff --git a/xen/arch/x86/hvm/vmx/intr.c b/xen/arch/x86/hvm/vmx/intr.c
index eb9b288..97cecbf 100644
--- a/xen/arch/x86/hvm/vmx/intr.c
+++ b/xen/arch/x86/hvm/vmx/intr.c
@@ -22,6 +22,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -37,6 +38,7 @@
 #include 
 #include 
 #include 
+#include 
 
 /*
  * A few notes on virtual NMI and INTR delivery, and interactions with
@@ -231,6 +233,11 @@ void vmx_intr_assist(void)
 enum hvm_intblk intblk;
 int pt_vector;
 
+if ( unlikely(v->arch.vm_event) &&
+ vm_event_check_ring(v->domain->vm_event_monitor) &&
+ v->arch.vm_event->intr_block )
+return;
+
 /* Block event injection when single step with MTF. */
 if ( unlikely(v->arch.hvm_vcpu.single_step) )
 {
diff --git a/xen/common/monitor.c b/xen/common/monitor.c
index c606683..4263929 100644
--- a/xen/common/monitor.c
+++ b/xen/common/monitor.c
@@ -113,6 +113,12 @@ int monitor_traps(struct vcpu *v, bool sync, vm_event_request_t *req)
 if ( sync )
 {
 req->flags |= VM_EVENT_FLAG_VCPU_PAUSED;
+/*
+ * It only makes sense to block interrupts for the duration of
+ * processing blocking vm_events, since that is the only case where
+ * emulating the current instruction really applies.
+ */
+v->arch.vm_event->intr_block = true;
 vm_event_vcpu_pause(v);
 rc = 1;
 }
diff --git a/xen/include/asm-x86/vm_event.h b/xen/include/asm-x86/vm_event.h
index 39e73c8..2b36614 100644
--- a/xen/include/asm-x86/vm_event.h
+++ b/xen/include/asm-x86/vm_event.h
@@ -34,6 +34,12 @@ struct arch_vm_event {
 struct monitor_write_data write_data;
 struct vm_event_regs_x86 gprs;
 bool set_gprs;
+/*
+ * Block interrupts until this vm_event is done handling (after the
+ * fashion of single-step). Meant for the cases where the vm_event
+ * reply asks for emulation of the current instruction.
+ */
+bool intr_block;
 };
 
 int vm_event_init_domain(struct domain *d);
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [ovmf baseline-only test] 75054: tolerable FAIL

2018-08-08 Thread Platform Team regression test user
This run is configured for baseline tests only.

flight 75054 ovmf real [real]
http://osstest.xensource.com/osstest/logs/75054/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail blocked in 75052
 test-amd64-amd64-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail like 75052

version targeted for testing:
 ovmf b6e48ec6412eab0f21fdff5a045a7ee516574d44
baseline version:
 ovmf 91a5b13650752a54cf766791aa369495c3426485

Last test of basis75052  2018-08-07 15:19:56 Z0 days
Testing same since75054  2018-08-08 12:49:57 Z0 days1 attempts


People who touched revisions under test:
  Ruiyu Ni 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 fail
 test-amd64-i386-xl-qemuu-ovmf-amd64  fail



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xensource.com/osstest/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


commit b6e48ec6412eab0f21fdff5a045a7ee516574d44
Author: Ruiyu Ni 
Date:   Mon Aug 6 14:29:00 2018 +0800

ShellPkg/acpi: Fix XCODE5 X64 build failure

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Ruiyu Ni 
Reviewed-by: Dandan Bi 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 5/5] xen/blkback: remove unused pers_gnts_lock from struct xen_blkif_ring

2018-08-08 Thread Juergen Gross
pers_gnts_lock isn't being used anywhere. Remove it.

Signed-off-by: Juergen Gross 
Reviewed-by: Roger Pau Monné 
---
 drivers/block/xen-blkback/common.h | 1 -
 1 file changed, 1 deletion(-)

diff --git a/drivers/block/xen-blkback/common.h 
b/drivers/block/xen-blkback/common.h
index 2339b8d39c5e..1d3002d773f7 100644
--- a/drivers/block/xen-blkback/common.h
+++ b/drivers/block/xen-blkback/common.h
@@ -269,7 +269,6 @@ struct xen_blkif_ring {
wait_queue_head_t   pending_free_wq;
 
/* Tree to store persistent grants. */
-   spinlock_t  pers_gnts_lock;
struct rb_root  persistent_gnts;
unsigned intpersistent_gnt_c;
atomic_tpersistent_gnt_in_use;
-- 
2.13.7


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 1/5] xen/blkback: don't keep persistent grants too long

2018-08-08 Thread Juergen Gross
Persistent grants are allocated until a threshold per ring is being
reached. Those grants won't be freed until the ring is being destroyed
meaning there will be resources kept busy which might no longer be
used.

Instead of freeing only persistent grants until the threshold is
reached add a timestamp and remove all persistent grants not having
been in use for a minute.

Signed-off-by: Juergen Gross 
---
 Documentation/ABI/testing/sysfs-driver-xen-blkback | 10 +++
 drivers/block/xen-blkback/blkback.c| 88 --
 drivers/block/xen-blkback/common.h |  8 +-
 3 files changed, 60 insertions(+), 46 deletions(-)

diff --git a/Documentation/ABI/testing/sysfs-driver-xen-blkback 
b/Documentation/ABI/testing/sysfs-driver-xen-blkback
index 8bb43b66eb55..4e7babb3ba1f 100644
--- a/Documentation/ABI/testing/sysfs-driver-xen-blkback
+++ b/Documentation/ABI/testing/sysfs-driver-xen-blkback
@@ -15,3 +15,13 @@ Description:
 blkback. If the frontend tries to use more than
 max_persistent_grants, the LRU kicks in and starts
 removing 5% of max_persistent_grants every 100ms.
+
+What:   
/sys/module/xen_blkback/parameters/persistent_grant_unused_seconds
+Date:   August 2018
+KernelVersion:  4.19
+Contact:Roger Pau Monné 
+Description:
+How long a persistent grant is allowed to remain
+allocated without being in use. The time is in
+seconds, 0 means indefinitely long.
+The default is 60 seconds.
diff --git a/drivers/block/xen-blkback/blkback.c 
b/drivers/block/xen-blkback/blkback.c
index b55b245e8052..f341ac84b853 100644
--- a/drivers/block/xen-blkback/blkback.c
+++ b/drivers/block/xen-blkback/blkback.c
@@ -84,6 +84,18 @@ MODULE_PARM_DESC(max_persistent_grants,
  "Maximum number of grants to map persistently");
 
 /*
+ * How long a persistent grant is allowed to remain allocated without being in
+ * use. The time is in seconds, 0 means indefinitely long.
+ */
+
+unsigned int xen_blkif_pgrant_timeout = 60;
+module_param_named(persistent_grant_unused_seconds, xen_blkif_pgrant_timeout,
+  uint, 0644);
+MODULE_PARM_DESC(persistent_grant_unused_seconds,
+"Time in seconds an unused persistent grant is allowed to "
+"remain allocated. Default is 60, 0 means unlimited.");
+
+/*
  * Maximum number of rings/queues blkback supports, allow as many queues as 
there
  * are CPUs if user has not specified a value.
  */
@@ -123,6 +135,13 @@ module_param(log_stats, int, 0644);
 /* Number of free pages to remove on each call to gnttab_free_pages */
 #define NUM_BATCH_FREE_PAGES 10
 
+static inline bool persistent_gnt_timeout(struct persistent_gnt 
*persistent_gnt)
+{
+   return xen_blkif_pgrant_timeout &&
+  (jiffies - persistent_gnt->last_used >=
+   HZ * xen_blkif_pgrant_timeout);
+}
+
 static inline int get_free_page(struct xen_blkif_ring *ring, struct page 
**page)
 {
unsigned long flags;
@@ -278,7 +297,7 @@ static void put_persistent_gnt(struct xen_blkif_ring *ring,
 {
if(!test_bit(PERSISTENT_GNT_ACTIVE, persistent_gnt->flags))
pr_alert_ratelimited("freeing a grant already unused\n");
-   set_bit(PERSISTENT_GNT_WAS_ACTIVE, persistent_gnt->flags);
+   persistent_gnt->last_used = jiffies;
clear_bit(PERSISTENT_GNT_ACTIVE, persistent_gnt->flags);
atomic_dec(>persistent_gnt_in_use);
 }
@@ -371,26 +390,26 @@ static void purge_persistent_gnt(struct xen_blkif_ring 
*ring)
struct persistent_gnt *persistent_gnt;
struct rb_node *n;
unsigned int num_clean, total;
-   bool scan_used = false, clean_used = false;
+   bool scan_used = false;
struct rb_root *root;
 
-   if (ring->persistent_gnt_c < xen_blkif_max_pgrants ||
-   (ring->persistent_gnt_c == xen_blkif_max_pgrants &&
-   !ring->blkif->vbd.overflow_max_grants)) {
-   goto out;
-   }
-
if (work_busy(>persistent_purge_work)) {
pr_alert_ratelimited("Scheduled work from previous purge is 
still busy, cannot purge list\n");
goto out;
}
 
-   num_clean = (xen_blkif_max_pgrants / 100) * LRU_PERCENT_CLEAN;
-   num_clean = ring->persistent_gnt_c - xen_blkif_max_pgrants + num_clean;
-   num_clean = min(ring->persistent_gnt_c, num_clean);
-   if ((num_clean == 0) ||
-   (num_clean > (ring->persistent_gnt_c - 
atomic_read(>persistent_gnt_in_use
-   goto out;
+   if (ring->persistent_gnt_c < xen_blkif_max_pgrants ||
+   (ring->persistent_gnt_c == xen_blkif_max_pgrants &&
+   !ring->blkif->vbd.overflow_max_grants)) {
+   num_clean = 0;
+   } else {
+   num_clean = (xen_blkif_max_pgrants / 100) * LRU_PERCENT_CLEAN;
+   num_clean = ring->persistent_gnt_c - 

[Xen-devel] [PATCH v2 4/5] xen/blkback: move persistent grants flags to bool

2018-08-08 Thread Juergen Gross
The struct persistent_gnt flags member is meant to be a bitfield of
different flags. There is only PERSISTENT_GNT_ACTIVE flag left, so
convert it to a bool named "active".

Signed-off-by: Juergen Gross 
---
 drivers/block/xen-blkback/blkback.c | 13 ++---
 drivers/block/xen-blkback/common.h  |  7 +--
 2 files changed, 7 insertions(+), 13 deletions(-)

diff --git a/drivers/block/xen-blkback/blkback.c 
b/drivers/block/xen-blkback/blkback.c
index f341ac84b853..2483931a3e01 100644
--- a/drivers/block/xen-blkback/blkback.c
+++ b/drivers/block/xen-blkback/blkback.c
@@ -255,8 +255,7 @@ static int add_persistent_gnt(struct xen_blkif_ring *ring,
}
}
 
-   bitmap_zero(persistent_gnt->flags, PERSISTENT_GNT_FLAGS_SIZE);
-   set_bit(PERSISTENT_GNT_ACTIVE, persistent_gnt->flags);
+   persistent_gnt->active = true;
/* Add new node and rebalance tree. */
rb_link_node(&(persistent_gnt->node), parent, new);
rb_insert_color(&(persistent_gnt->node), >persistent_gnts);
@@ -280,11 +279,11 @@ static struct persistent_gnt *get_persistent_gnt(struct 
xen_blkif_ring *ring,
else if (gref > data->gnt)
node = node->rb_right;
else {
-   if(test_bit(PERSISTENT_GNT_ACTIVE, data->flags)) {
+   if (data->active) {
pr_alert_ratelimited("requesting a grant 
already in use\n");
return NULL;
}
-   set_bit(PERSISTENT_GNT_ACTIVE, data->flags);
+   data->active = true;
atomic_inc(>persistent_gnt_in_use);
return data;
}
@@ -295,10 +294,10 @@ static struct persistent_gnt *get_persistent_gnt(struct 
xen_blkif_ring *ring,
 static void put_persistent_gnt(struct xen_blkif_ring *ring,
struct persistent_gnt *persistent_gnt)
 {
-   if(!test_bit(PERSISTENT_GNT_ACTIVE, persistent_gnt->flags))
+   if (!persistent_gnt->active)
pr_alert_ratelimited("freeing a grant already unused\n");
persistent_gnt->last_used = jiffies;
-   clear_bit(PERSISTENT_GNT_ACTIVE, persistent_gnt->flags);
+   persistent_gnt->active = false;
atomic_dec(>persistent_gnt_in_use);
 }
 
@@ -429,7 +428,7 @@ static void purge_persistent_gnt(struct xen_blkif_ring 
*ring)
BUG_ON(persistent_gnt->handle ==
BLKBACK_INVALID_HANDLE);
 
-   if (test_bit(PERSISTENT_GNT_ACTIVE, persistent_gnt->flags))
+   if (persistent_gnt->active)
continue;
if (!scan_used && !persistent_gnt_timeout(persistent_gnt))
continue;
diff --git a/drivers/block/xen-blkback/common.h 
b/drivers/block/xen-blkback/common.h
index 7bff72db3b7e..2339b8d39c5e 100644
--- a/drivers/block/xen-blkback/common.h
+++ b/drivers/block/xen-blkback/common.h
@@ -233,11 +233,6 @@ struct xen_vbd {
 
 struct backend_info;
 
-/* Number of available flags */
-#define PERSISTENT_GNT_FLAGS_SIZE  1
-/* This persistent grant is currently in use */
-#define PERSISTENT_GNT_ACTIVE  0
-
 /* Number of requests that we can fit in a ring */
 #define XEN_BLKIF_REQS_PER_PAGE32
 
@@ -246,7 +241,7 @@ struct persistent_gnt {
grant_ref_t gnt;
grant_handle_t handle;
unsigned long last_used;
-   DECLARE_BITMAP(flags, PERSISTENT_GNT_FLAGS_SIZE);
+   bool active;
struct rb_node node;
struct list_head remove_node;
 };
-- 
2.13.7


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 0/5] xen/blk: persistent grant rework

2018-08-08 Thread Juergen Gross
Persistent grants are used in the Xen's blkfront/blkback drivers to
avoid mapping/unmapping of I/O buffers in the backend for each I/O.

While this speeds up processing quite a bit there are problems related
to persistent grants in some configurations: domains with multiple
block devices making use of persistent grants might suffer from a lack
of grants if each of the block devices experienced a high I/O load at
some time. This is due to the number of persistent grants per device
only to be limited by a rather high maximum value, but never being
released even in case of longer times without any I/O.

This series modifies xen-blkback to unmap any domU page mapped via a
persistent grant after a timeout (default: 60 seconds). The timeout
is set to its default value again when a persistent grant has been
used for an I/O.

xen-blkfront is modified to scan every 10 seconds for persistent grants
not in use by blkback any more and to remove such grants.

The last 3 patches are small cleanups of blkfront and blkback drivers.

V2:
- patch 1: added new module parameter doc
- patch 1: removed PERSISTENT_GNT_WAS_ACTIVE flag
- patch 2: removed global worker active flag
- added new patch 4

Juergen Gross (5):
  xen/blkback: don't keep persistent grants too long
  xen/blkfront: cleanup stale persistent grants
  xen/blkfront: reorder tests in xlblk_init()
  xen/blkback: move persistent grants flags to bool
  xen/blkback: remove unused pers_gnts_lock from struct xen_blkif_ring

 Documentation/ABI/testing/sysfs-driver-xen-blkback |  10 ++
 drivers/block/xen-blkback/blkback.c|  99 ++-
 drivers/block/xen-blkback/common.h |  14 +--
 drivers/block/xen-blkfront.c   | 110 ++---
 4 files changed, 163 insertions(+), 70 deletions(-)

-- 
2.13.7


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC] OVMF on PVH

2018-08-08 Thread Anthony PERARD
On Thu, Aug 02, 2018 at 06:23:54AM -0600, Jan Beulich wrote:
> >>> On 02.08.18 at 13:24,  wrote:
> > ## Loading binary at 0x1 (1MB, like hvmloader on HVM)
> > 
> > Right now, the OVMF blob is loaded (by hvmloader) just below 4GB, with
> > hvmloader moving some RAM around to allow that.  But with the use of an
> > ELF header, and let libxc load the blob, it was not possible to load the
> > blob just below 4GB. (Or at least without modification of the toolstack I
> > guess.) So I've attempted to have OVMF been started from a different place
> > in memory. Then I had to hack a bit more in order to be able to start OVMF
> > from two different location (the current one for HVM guest, and a new one
> > that can be used for PVH). That works fine, I'll just have to find out
> > what upstream think about that.
> 
> I'm of two minds here - as a firmware binary, it doesn't seem to make
> sense to load at 0x10, yet as a hvmloader replacement it might.
> However, parts of it will need to stay (runtime services code/data),
> and such permanent data better lives at higher addresses imo.
> 
> Jan

On Thu, Aug 02, 2018 at 06:21:31PM +0200, Roger Pau Monné wrote:
> That works well for hvmloader because after it has finished running
> the memory can be overwritten without issues. The same is not true for
> ovmf which needs to keep data around so that boot services and runtime
> services can continue working.
> 
> Thanks, Roger.

So there is something to know about the way the OVMF blob (it's actually
called flash device image, or FD), it that there is almost no text (code
to execute). Most of the FD is compressed, with a small section (called
SEC Phase modules) which takes care of bringing up the CPU from 16bits
to 64bits, then some code to find where the next phase is and decompress
the main module.

I don't think any of the initial blob is kept around once the
decompression is done. I did build my OVMF, and use the same exacte blob
for both HVM and PVH, in HVM the blob would be loaded below 4G (no
change), and for PVH it would be loaded at 1M. They both worked fine, so
it doesn't really matter where the initial blob is.

As for where the permanent data actually lives, I am not changing that.
OVMF setup a lot of stuff between 0x80-0x140 as this is where
its page tables are initialy setup (I don't know if they move later),
and all its code is.

Thanks,

-- 
Anthony PERARD

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v22 2/2] tools/libxenctrl: use new xenforeignmemory API to seed grant table

2018-08-08 Thread Paul Durrant
A previous patch added support for priv-mapping guest resources directly
(rather than having to foreign-map, which requires P2M modification for
HVM guests).

This patch makes use of the new API to seed the guest grant table unless
the underlying infrastructure (i.e. privcmd) doesn't support it, in which
case the old scheme is used.

NOTE: The call to xc_dom_gnttab_hvm_seed() in hvm_build_set_params() was
  actually unnecessary, as the grant table has already been seeded
  by a prior call to xc_dom_gnttab_init() made by libxl__build_dom().

Signed-off-by: Paul Durrant 
Acked-by: Marek Marczykowski-Górecki 
Reviewed-by: Roger Pau Monné 
Acked-by: Wei Liu 
---
Cc: Ian Jackson 
Cc: Andrew Cooper 

v22:
 - Addressed comments from Andrew (cosmetic changes only).

v18:
 - Trivial re-base.

v13:
 - Re-base.

v10:
 - Use new id constant for grant table.

v4:
 - Minor cosmetic fix suggested by Roger.

v3:
 - Introduced xc_dom_set_gnttab_entry() to avoid duplicated code.
---
 tools/libxc/include/xc_dom.h|  12 +--
 tools/libxc/xc_dom_boot.c   | 166 +++-
 tools/libxc/xc_sr_restore_x86_hvm.c |  10 +--
 tools/libxc/xc_sr_restore_x86_pv.c  |   2 +-
 tools/libxl/libxl_dom.c |   1 -
 tools/python/xen/lowlevel/xc/xc.c   |   6 +-
 6 files changed, 120 insertions(+), 77 deletions(-)

diff --git a/tools/libxc/include/xc_dom.h b/tools/libxc/include/xc_dom.h
index 8a66889c68..0b5a632d3c 100644
--- a/tools/libxc/include/xc_dom.h
+++ b/tools/libxc/include/xc_dom.h
@@ -337,14 +337,10 @@ void *xc_dom_boot_domU_map(struct xc_dom_image *dom, 
xen_pfn_t pfn,
 int xc_dom_boot_image(struct xc_dom_image *dom);
 int xc_dom_compat_check(struct xc_dom_image *dom);
 int xc_dom_gnttab_init(struct xc_dom_image *dom);
-int xc_dom_gnttab_hvm_seed(xc_interface *xch, uint32_t domid,
-   xen_pfn_t console_gmfn,
-   xen_pfn_t xenstore_gmfn,
-   uint32_t console_domid,
-   uint32_t xenstore_domid);
-int xc_dom_gnttab_seed(xc_interface *xch, uint32_t domid,
-   xen_pfn_t console_gmfn,
-   xen_pfn_t xenstore_gmfn,
+int xc_dom_gnttab_seed(xc_interface *xch, uint32_t guest_domid,
+   bool is_hvm,
+   xen_pfn_t console_gfn,
+   xen_pfn_t xenstore_gfn,
uint32_t console_domid,
uint32_t xenstore_domid);
 bool xc_dom_translated(const struct xc_dom_image *dom);
diff --git a/tools/libxc/xc_dom_boot.c b/tools/libxc/xc_dom_boot.c
index 2e5681dc5d..0f852237ee 100644
--- a/tools/libxc/xc_dom_boot.c
+++ b/tools/libxc/xc_dom_boot.c
@@ -256,71 +256,81 @@ static xen_pfn_t xc_dom_gnttab_setup(xc_interface *xch, 
uint32_t domid)
 return gmfn;
 }
 
-int xc_dom_gnttab_seed(xc_interface *xch, uint32_t domid,
-   xen_pfn_t console_gmfn,
-   xen_pfn_t xenstore_gmfn,
-   uint32_t console_domid,
-   uint32_t xenstore_domid)
+static void xc_dom_set_gnttab_entry(xc_interface *xch,
+grant_entry_v1_t *gnttab,
+unsigned int idx,
+uint32_t guest_domid,
+uint32_t backend_domid,
+xen_pfn_t guest_gfn)
 {
+if ( guest_domid == backend_domid || guest_gfn == -1 )
+return;
+
+xc_dom_printf(xch, "%s: [%u] -> 0x%"PRI_xen_pfn,
+  __func__, idx, guest_gfn);
+
+gnttab[idx].flags = GTF_permit_access;
+gnttab[idx].domid = backend_domid;
+gnttab[idx].frame = guest_gfn;
+}
 
-xen_pfn_t gnttab_gmfn;
+static int compat_gnttab_seed(xc_interface *xch, uint32_t domid,
+  xen_pfn_t console_gfn,
+  xen_pfn_t xenstore_gfn,
+  uint32_t console_domid,
+  uint32_t xenstore_domid)
+{
+
+xen_pfn_t gnttab_gfn;
 grant_entry_v1_t *gnttab;
 
-gnttab_gmfn = xc_dom_gnttab_setup(xch, domid);
-if ( gnttab_gmfn == -1 )
+gnttab_gfn = xc_dom_gnttab_setup(xch, domid);
+if ( gnttab_gfn == -1 )
 return -1;
 
 gnttab = xc_map_foreign_range(xch,
   domid,
   PAGE_SIZE,
   PROT_READ|PROT_WRITE,
-  gnttab_gmfn);
+  gnttab_gfn);
 if ( gnttab == NULL )
 {
 xc_dom_panic(xch, XC_INTERNAL_ERROR,
- "%s: failed to map domU grant table "
+ "%s: failed to map d%d grant table "
  "[errno=%d]\n",
- __FUNCTION__, errno);
+ __func__, domid, errno);
 return -1;
 }
 
-if ( domid != console_domid  && 

[Xen-devel] [PATCH v22 0/2] guest resource mapping (reprise)

2018-08-08 Thread Paul Durrant
These patches are the patches from my original resource mapping series
that did not make it into 4.11.

Paul Durrant (2):
  common: add a new mappable resource type: XENMEM_resource_grant_table
  tools/libxenctrl: use new xenforeignmemory API to seed grant table

 tools/libxc/include/xc_dom.h|  12 +--
 tools/libxc/xc_dom_boot.c   | 166 +++-
 tools/libxc/xc_sr_restore_x86_hvm.c |  10 +--
 tools/libxc/xc_sr_restore_x86_pv.c  |   2 +-
 tools/libxl/libxl_dom.c |   1 -
 tools/python/xen/lowlevel/xc/xc.c   |   6 +-
 xen/common/grant_table.c| 115 ++---
 xen/common/memory.c |  56 +++-
 xen/include/public/memory.h |   6 ++
 xen/include/xen/grant_table.h   |   4 +
 10 files changed, 286 insertions(+), 92 deletions(-)

-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v22 1/2] common: add a new mappable resource type: XENMEM_resource_grant_table

2018-08-08 Thread Paul Durrant
This patch allows grant table frames to be mapped using the
XENMEM_acquire_resource memory op.

NOTE: This patch expands the on-stack mfn_list array in acquire_resource()
  but it is still small enough to remain on-stack.

NOTE: This patch also removes a bogus comment above the
  grant_to_status_frames() function.

Signed-off-by: Paul Durrant 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: Wei Liu 

v22:
 - Remove bogus comment and update commit message accordingly.
 - Add ASSERTion that an invalid MFN is not passed back to the caller of
   XENMEM_acquire_resource.
 - Re-code the idx to nr calculation to try to make it more obvious and
   add explicit overflow checks.

v21:
 - Prevent PVH/HVM tools domains from mapping any non-caller-owned resource
   unless the tools domain is also the hardware domain.
 - Grow the grant table appropriately whether it is a shared frame or
   a status frame that is being mapped.
 - Fix comment style breakage in memory.h.
 - Move implicit version setting to gnttab_get_shared_frame().

v19:
 - Add test to prevent PVH/HVM tools domains mapping grant status frames
   this way as the mapping infrastructure in Xen does not yet implement the
   necessary reference counting to make this safe.
 - Make sure grant table version is set before any attempt to grow the table.

v18:
 - Non-trivial re-base of grant table code.
 - Dropped Jan's R-b because of the grant table changes.

v13:
 - Re-work the internals to avoid using the XENMAPIDX_grant_table_status
   hack.

v12:
 - Dropped limit checks as requested by Jan.

v10:
 - Addressed comments from Jan.

v8:
 - The functionality was originally incorporated into the earlier patch
   "x86/mm: add HYPERVISOR_memory_op to acquire guest resources".
---
 xen/common/grant_table.c  | 115 +-
 xen/common/memory.c   |  56 +++-
 xen/include/public/memory.h   |   6 +++
 xen/include/xen/grant_table.h |   4 ++
 4 files changed, 166 insertions(+), 15 deletions(-)

diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index d9ec711c73..9aab808d44 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -352,12 +352,17 @@ static inline void active_entry_release(struct 
active_grant_entry *act)
 
 #define GRANT_STATUS_PER_PAGE (PAGE_SIZE / sizeof(grant_status_t))
 #define GRANT_PER_PAGE (PAGE_SIZE / sizeof(grant_entry_v2_t))
-/* Number of grant table status entries. Caller must hold d's gr. table lock.*/
+
 static inline unsigned int grant_to_status_frames(unsigned int grant_frames)
 {
 return DIV_ROUND_UP(grant_frames * GRANT_PER_PAGE, GRANT_STATUS_PER_PAGE);
 }
 
+static inline unsigned int status_to_grant_frames(unsigned int status_frames)
+{
+return DIV_ROUND_UP(status_frames * GRANT_STATUS_PER_PAGE, GRANT_PER_PAGE);
+}
+
 /* Check if the page has been paged out, or needs unsharing.
If rc == GNTST_okay, *page contains the page struct with a ref taken.
Caller must do put_page(*page).
@@ -3860,6 +3865,66 @@ int mem_sharing_gref_to_gfn(struct grant_table *gt, 
grant_ref_t ref,
 }
 #endif
 
+/* caller must hold write lock */
+static int gnttab_get_status_frame_mfn(struct domain *d,
+   unsigned long idx, mfn_t *mfn)
+{
+const struct grant_table *gt = d->grant_table;
+
+ASSERT(gt->gt_version == 2);
+
+if ( idx >= nr_status_frames(gt) )
+{
+unsigned long nr_status;
+unsigned long nr_grant;
+
+nr_status = idx + 1; /* sufficient frames to make idx valid */
+nr_grant = status_to_grant_frames(nr_status);
+
+if ( nr_grant <= nr_grant_frames(gt) ) /* overflow check */
+return -EINVAL;
+
+if ( nr_grant <= gt->max_grant_frames )
+gnttab_grow_table(d, nr_grant);
+
+/* check whether gnttab_grow_table() succeeded */
+if ( idx >= nr_status_frames(gt) )
+return -EINVAL;
+}
+
+*mfn = _mfn(virt_to_mfn(gt->status[idx]));
+return 0;
+}
+
+/* caller must hold write lock */
+static int gnttab_get_shared_frame_mfn(struct domain *d,
+   unsigned long idx, mfn_t *mfn)
+{
+const struct grant_table *gt = d->grant_table;
+
+ASSERT(gt->gt_version != 0);
+
+if ( idx >= nr_grant_frames(gt) )
+{
+unsigned long nr_grant;
+
+nr_grant = idx + 1; /* sufficient frames to make idx valid */
+
+if ( nr_grant <= nr_grant_frames(gt) ) /* overflow check */
+return -EINVAL;
+
+if ( nr_grant <= gt->max_grant_frames )
+gnttab_grow_table(d, nr_grant);
+
+/* check whether gnttab_grow_table() succeeded */
+if ( idx >= nr_grant_frames(gt) )
+return -EINVAL;
+}
+
+*mfn = _mfn(virt_to_mfn(gt->shared_raw[idx]));
+return 0;
+}
+
 int gnttab_map_frame(struct domain *d, unsigned long idx, 

Re: [Xen-devel] [PATCH v5 01/15] iommu: turn need_iommu back into a boolean.

2018-08-08 Thread Paul Durrant
> -Original Message-
> From: Jan Beulich [mailto:jbeul...@suse.com]
> Sent: 08 August 2018 14:39
> To: Paul Durrant 
> Cc: Julien Grall ; Andrew Cooper
> ; Wei Liu ; George
> Dunlap ; Ian Jackson ;
> Stefano Stabellini ; xen-devel  de...@lists.xenproject.org>; Konrad Rzeszutek Wilk
> ; Tim (Xen.org) 
> Subject: Re: [PATCH v5 01/15] iommu: turn need_iommu back into a
> boolean.
> 
> >>> On 03.08.18 at 19:22,  wrote:
> > As noted in [1] the tri-state nature of need_iommu after commit 3e06b989
> is
> > confusing.
> >
> > Because arch_iommu_populate_page_table() removes pages from the
> page list
> > as it maps them it is actually safe to re-invoke multiple times without
> > the need for any specific indication it has been called before, so it
> > is actually safe to simply turn need_iommu back into a boolean with no
> > change to the population algorithm.
> >
> > [1] https://lists.xenproject.org/archives/html/xen-devel/2018-
> 07/msg01870.html
> 
> I have to admit that I wouldn't read "confusing" into that mail. And
> given the below, I continue to wonder whether you really, really
> need to change this.

Ok, I'll squash this patch this into the subsequent patch where I separate the 
ideas of a domain having IOMMU pagetables and requiring them to be kept in sync.

  Paul

> 
> > --- a/xen/drivers/passthrough/iommu.c
> > +++ b/xen/drivers/passthrough/iommu.c
> > @@ -214,14 +214,14 @@ void iommu_teardown(struct domain *d)
> >  {
> >  const struct domain_iommu *hd = dom_iommu(d);
> >
> > -d->need_iommu = 0;
> > +d->need_iommu = false;
> >  hd->platform_ops->teardown(d);
> >  tasklet_schedule(_pt_cleanup_tasklet);
> >  }
> >
> >  int iommu_construct(struct domain *d)
> >  {
> > -if ( need_iommu(d) > 0 )
> > +if ( need_iommu(d) )
> >  return 0;
> >
> >  if ( !iommu_use_hap_pt(d) )
> > @@ -233,7 +233,7 @@ int iommu_construct(struct domain *d)
> >  return rc;
> >  }
> >
> > -d->need_iommu = 1;
> > +d->need_iommu = true;
> 
> So with the setting to -1 gone from the caller, need_iommu(d) will
> continue to return false until this completion point is reached. The
> fundamental idea of the tristate was that once we've started
> populating the IOMMU page tables (recall, the domain is not
> paused if this is a hotplug), all _other_ operations requiring IOMMU
> page table manipulation (grant table code, for example) will
> DTRT (tm) despite the code here perhaps never getting to notice
> the relevant page.
> 
> Trust me, it wasn't a lightweight decision to make this a tristate,
> I just couldn't see a better approach (short of using a second
> boolean, which I would have liked even less), and I still can't.
> 
> > @@ -493,7 +493,7 @@ static void iommu_dump_p2m_table(unsigned char
> key)
> >  ops = iommu_get_ops();
> >  for_each_domain(d)
> >  {
> > -if ( is_hardware_domain(d) || need_iommu(d) <= 0 )
> > +if ( is_hardware_domain(d) || !need_iommu(d) )
> >  continue;
> 
> I don't think the original semantics of the dumping to be skipped for
> domains with their IOMMU page tables under construction is being
> retained here.
> 
> Jan
> 


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-unstable-smoke test] 125802: trouble: blocked/broken/pass

2018-08-08 Thread osstest service owner
flight 125802 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/125802/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken

Regressions which are regarded as allowable (not blocking):
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 125729

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64-xsm   3 capture-logs  broken blocked in 125729
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  4b60c40659b34b6577a6bc91eb4115458a0e425f
baseline version:
 xen  ed5f8d9ca47e69e30221c37ec812f2b38af19d83

Last test of basis   125729  2018-08-01 11:00:39 Z7 days
Failing since125741  2018-08-02 10:01:09 Z6 days   17 attempts
Testing same since   125802  2018-08-08 11:00:47 Z0 days1 attempts


People who touched revisions under test:
  Alexandru Isaila 
  Andrew Cooper 
  Doug Goldstein 
  George Dunlap 
  Jan Beulich 
  Julien Grall 
  Kevin Tian 
  Razvan Cojocaru 
  Roger Pau Monné 
  Stefano Stabellini 
  Wei Liu 

jobs:
 build-arm64-xsm  broken  
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64-xsm broken
broken-step build-arm64-xsm hosts-allocate
broken-step build-arm64-xsm capture-logs

Not pushing.

(No revision log; it would be 512 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [freebsd-master test] 125801: regressions - trouble: blocked/fail

2018-08-08 Thread osstest service owner
flight 125801 freebsd-master real [real]
http://logs.test-lab.xenproject.org/osstest/logs/125801/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-freebsd   7 freebsd-buildfail REGR. vs. 125317

Tests which did not succeed, but are not blocking:
 build-amd64-freebsd-again 1 build-check(1)   blocked  n/a

version targeted for testing:
 freebsd  d182352391ba0f4301781c5e1c98a8e31a9b8315
baseline version:
 freebsd  bf65d05707104df94117a293327d797d71a0118d

Last test of basis   125317  2018-07-18 09:19:47 Z   21 days
Failing since125467  2018-07-20 09:19:59 Z   19 days9 attempts
Testing same since   125801  2018-08-08 09:19:29 Z0 days1 attempts


People who touched revisions under test:
  0mp <0...@freebsd.org>
  alc 
  allanjude 
  andrew 
  antoine 
  araujo 
  asomers 
  avg 
  bapt 
  bde 
  bdrewery 
  br 
  brd 
  bwidawsk 
  bz 
  cem 
  cperciva 
  cy 
  dab 
  daichi 
  davidcs 
  delphij 
  dim 
  eadler 
  emaste 
  eugen 
  ganbold 
  gavin 
  gjb 
  glebius 
  hselasky 
  ian 
  imp 
  jhb 
  jhibbits 
  jhixson 
  jtl 
  kevans 
  kevlo 
  kib 
  kp 
  leitao 
  luporl 
  lwhsu 
  manu 
  marius 
  markj 
  Matthew Ahrens 
  mav 
  mckusick 
  mm 
  mmacy 
  mmel 
  np 
  obrien 
  oshogbo 
  pfg 
  pkelsey 
  pstef 
  rmacklem 
  royger 
  rpokala 
  rrs 
  sef 
  shurd 
  sjg 
  trasz 
  truckman 
  tsoome 
  tuexen 
  uqs 
  will 
  woodsb02 
  wulf 
  zleslie 

jobs:
 build-amd64-freebsd-againblocked 
 build-amd64-freebsd  fail



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 8122 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-4.7-testing test] 125784: regressions - trouble: blocked/broken/fail/pass

2018-08-08 Thread osstest service owner
flight 125784 xen-4.7-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/125784/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64  broken
 build-arm64-pvopsbroken
 build-arm64-xsm  broken
 test-amd64-i386-libvirt-pair 22 guest-migrate/src_host/dst_host fail REGR. vs. 
125057

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-credit2 16 guest-start/debian.repeat fail in 125708 pass 
in 125784
 test-xtf-amd64-amd64-3 50 xtf/test-hvm64-lbr-tsx-vmentry fail in 125765 pass 
in 125784
 test-amd64-amd64-libvirt-pair 22 guest-migrate/src_host/dst_host fail pass in 
125708
 test-xtf-amd64-amd64-4   50 xtf/test-hvm64-lbr-tsx-vmentry fail pass in 125765
 test-amd64-amd64-xl-qemut-win7-amd64 15 guest-saverestore.2 fail pass in 125765

Regressions which are regarded as allowable (not blocking):
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 125057
 build-arm64-pvops 2 hosts-allocate broken REGR. vs. 125057
 build-arm64   2 hosts-allocate broken REGR. vs. 125057

Tests which did not succeed, but are not blocking:
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64   3 capture-logs  broken blocked in 125057
 build-arm64-pvops 3 capture-logs  broken blocked in 125057
 build-arm64-xsm   3 capture-logs  broken blocked in 125057
 test-arm64-arm64-xl-credit2 13 migrate-support-check fail in 125708 never pass
 test-arm64-arm64-xl-credit2 14 saverestore-support-check fail in 125708 never 
pass
 test-arm64-arm64-xl-xsm 13 migrate-support-check fail in 125708 never pass
 test-arm64-arm64-xl 13 migrate-support-check fail in 125708 never pass
 test-arm64-arm64-xl-xsm 14 saverestore-support-check fail in 125708 never pass
 test-arm64-arm64-xl 14 saverestore-support-check fail in 125708 never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-check fail in 125708 never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-check fail in 125708 never 
pass
 test-xtf-amd64-amd64-2 50 xtf/test-hvm64-lbr-tsx-vmentry fail in 125765 like 
125057
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop  fail in 125765 like 125057
 test-xtf-amd64-amd64-1  50 xtf/test-hvm64-lbr-tsx-vmentry fail like 125057
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 125057
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 125057
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 125057
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 125057
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 125057
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop fail like 125057
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop fail like 125057
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 125057
 test-amd64-amd64-xl-rtds 10 debian-install   fail  like 125057
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeatfail  like 125057
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 125057
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 

Re: [Xen-devel] [PATCH v5 01/15] iommu: turn need_iommu back into a boolean.

2018-08-08 Thread Jan Beulich
>>> On 03.08.18 at 19:22,  wrote:
> As noted in [1] the tri-state nature of need_iommu after commit 3e06b989 is
> confusing.
> 
> Because arch_iommu_populate_page_table() removes pages from the page list
> as it maps them it is actually safe to re-invoke multiple times without
> the need for any specific indication it has been called before, so it
> is actually safe to simply turn need_iommu back into a boolean with no
> change to the population algorithm.
> 
> [1] 
> https://lists.xenproject.org/archives/html/xen-devel/2018-07/msg01870.html 

I have to admit that I wouldn't read "confusing" into that mail. And
given the below, I continue to wonder whether you really, really
need to change this.

> --- a/xen/drivers/passthrough/iommu.c
> +++ b/xen/drivers/passthrough/iommu.c
> @@ -214,14 +214,14 @@ void iommu_teardown(struct domain *d)
>  {
>  const struct domain_iommu *hd = dom_iommu(d);
>  
> -d->need_iommu = 0;
> +d->need_iommu = false;
>  hd->platform_ops->teardown(d);
>  tasklet_schedule(_pt_cleanup_tasklet);
>  }
>  
>  int iommu_construct(struct domain *d)
>  {
> -if ( need_iommu(d) > 0 )
> +if ( need_iommu(d) )
>  return 0;
>  
>  if ( !iommu_use_hap_pt(d) )
> @@ -233,7 +233,7 @@ int iommu_construct(struct domain *d)
>  return rc;
>  }
>  
> -d->need_iommu = 1;
> +d->need_iommu = true;

So with the setting to -1 gone from the caller, need_iommu(d) will
continue to return false until this completion point is reached. The
fundamental idea of the tristate was that once we've started
populating the IOMMU page tables (recall, the domain is not
paused if this is a hotplug), all _other_ operations requiring IOMMU
page table manipulation (grant table code, for example) will
DTRT (tm) despite the code here perhaps never getting to notice
the relevant page.

Trust me, it wasn't a lightweight decision to make this a tristate,
I just couldn't see a better approach (short of using a second
boolean, which I would have liked even less), and I still can't.

> @@ -493,7 +493,7 @@ static void iommu_dump_p2m_table(unsigned char key)
>  ops = iommu_get_ops();
>  for_each_domain(d)
>  {
> -if ( is_hardware_domain(d) || need_iommu(d) <= 0 )
> +if ( is_hardware_domain(d) || !need_iommu(d) )
>  continue;

I don't think the original semantics of the dumping to be skipped for
domains with their IOMMU page tables under construction is being
retained here.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v21 1/2] common: add a new mappable resource type: XENMEM_resource_grant_table

2018-08-08 Thread Andrew Cooper
On 08/08/18 14:15, Paul Durrant wrote:
>>
>> @@ -3906,6 +3943,38 @@ int gnttab_map_frame(struct domain *d,
> unsigned long idx, gfn_t gfn,
>>  return rc;
>>  }
>>
>> +int gnttab_get_shared_frame(struct domain *d, unsigned long idx,
>> +mfn_t *mfn)
>> +{
>> +struct grant_table *gt = d->grant_table;
>> +int rc;
>> +
>> +grant_write_lock(gt);
>> +
>> +if ( gt->gt_version == 0 )
>> +gt->gt_version = 1;
> Since you've moved this here instead of dropping it, what requirement
> have you found for this to be set (other than the ASSERT() you put in
> gnttab_get_shared_frame_mfn()?
>
 The code in patch #2 is executed before the grant table version is set. I
 could alternatively have libxl explicitly set the version to 1 before 
 trying
 to seed the table.
>>> But that's not my point. What's wrong with leaving it at zero?
>>>
>> I'm not particularly happy calling gnttab_grow_table() with version left at 0
>> but I can try it and see if it breaks.
> Actually, no. There is nowhere else that leaves it at 0 and I find that I 
> can't set the version explicitly from the toolstack as gnttab_set_version 
> doesn't take a domid as a parameter. I'll leave the version setting as-is.

Yeah - this looks like the best option for now, and I'll fix the
defaulting-to-1 in due course.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v21 1/2] common: add a new mappable resource type: XENMEM_resource_grant_table

2018-08-08 Thread Paul Durrant
> -Original Message-
> From: Xen-devel [mailto:xen-devel-boun...@lists.xenproject.org] On Behalf
> Of Paul Durrant
> Sent: 08 August 2018 12:33
> To: 'Jan Beulich' 
> Cc: Stefano Stabellini ; Wei Liu
> ; Andrew Cooper ; Tim
> (Xen.org) ; George Dunlap ; Ian
> Jackson ; xen-devel  de...@lists.xenproject.org>
> Subject: Re: [Xen-devel] [PATCH v21 1/2] common: add a new mappable
> resource type: XENMEM_resource_grant_table
> 
> > -Original Message-
> > From: Jan Beulich [mailto:jbeul...@suse.com]
> > Sent: 08 August 2018 11:43
> > To: Paul Durrant 
> > Cc: Andrew Cooper ; George Dunlap
> > ; Ian Jackson ; Wei
> Liu
> > ; Stefano Stabellini ; xen-
> > devel ; Konrad Rzeszutek Wilk
> > ; Tim (Xen.org) 
> > Subject: RE: [PATCH v21 1/2] common: add a new mappable resource type:
> > XENMEM_resource_grant_table
> >
> > >>> On 08.08.18 at 12:38,  wrote:
> > >> From: Jan Beulich [mailto:jbeul...@suse.com]
> > >> Sent: 08 August 2018 11:30
> > >>
> > >> >>> On 08.08.18 at 11:00,  wrote:
> > >> > @@ -3860,6 +3866,47 @@ int mem_sharing_gref_to_gfn(struct
> > >> grant_table *gt, grant_ref_t ref,
> > >> >  }
> > >> >  #endif
> > >> >
> > >> > +/* caller must hold write lock */
> > >> > +static int gnttab_get_status_frame_mfn(struct domain *d,
> > >> > +   unsigned long idx, mfn_t *mfn)
> > >> > +{
> > >> > +struct grant_table *gt = d->grant_table;
> > >>
> > >> const?
> > >>
> > >
> > > IIRC that didn't work because gnttab_grow_table() modifies the content.
> >
> > But you don't pass gt to the function:
> >
> > >> > +ASSERT(gt->gt_version == 2);
> > >> > +
> > >> > +if ( idx >= nr_status_frames(gt) )
> > >> > +{
> > >> > +unsigned long nr = status_to_grant_frames(idx + 1);
> > >> > +
> > >> > +if ( nr <= gt->max_grant_frames )
> > >> > +gnttab_grow_table(d, nr);
> >
> > ^^^
> 
> I know, I was just remembering that the compiler complained when I tried
> this before. My memory could be wrong so I'll try it again.
> 
> >
> > >> > @@ -3906,6 +3943,38 @@ int gnttab_map_frame(struct domain *d,
> > >> unsigned long idx, gfn_t gfn,
> > >> >  return rc;
> > >> >  }
> > >> >
> > >> > +int gnttab_get_shared_frame(struct domain *d, unsigned long idx,
> > >> > +mfn_t *mfn)
> > >> > +{
> > >> > +struct grant_table *gt = d->grant_table;
> > >> > +int rc;
> > >> > +
> > >> > +grant_write_lock(gt);
> > >> > +
> > >> > +if ( gt->gt_version == 0 )
> > >> > +gt->gt_version = 1;
> > >>
> > >> Since you've moved this here instead of dropping it, what requirement
> > >> have you found for this to be set (other than the ASSERT() you put in
> > >> gnttab_get_shared_frame_mfn()?
> > >>
> > >
> > > The code in patch #2 is executed before the grant table version is set. I
> > > could alternatively have libxl explicitly set the version to 1 before 
> > > trying
> > > to seed the table.
> >
> > But that's not my point. What's wrong with leaving it at zero?
> >
> 
> I'm not particularly happy calling gnttab_grow_table() with version left at 0
> but I can try it and see if it breaks.

Actually, no. There is nowhere else that leaves it at 0 and I find that I can't 
set the version explicitly from the toolstack as gnttab_set_version doesn't 
take a domid as a parameter. I'll leave the version setting as-is.

  Paul

> 
>   Paul
> 
> > Jan
> >
> 
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xenproject.org
> https://lists.xenproject.org/mailman/listinfo/xen-devel
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v4 4/4] x86/iommu: add reserved dom0-iommu option to map reserved memory ranges

2018-08-08 Thread Jan Beulich
>>> On 08.08.18 at 12:07,  wrote:
> Several people have reported hardware issues (malfunctioning USB
> controllers) due to iommu page faults on Intel hardware. Those faults
> are caused by missing RMRR (VTd) entries in the ACPI tables. Those can
> be worked around on VTd hardware by manually adding RMRR entries on
> the command line, this is however limited to Intel hardware and quite
> cumbersome to do.
> 
> In order to solve those issues add a new dom0-iommu=reserved option
> that identity maps all regions marked as reserved in the memory map.

Considering that report we've had (yesterday? earlier today?), don't
we need to go further and use the union of RMRRs and reserved
regions? Iirc they had a case where an RMRR was not in a reserved
region ...

> --- a/docs/misc/xen-command-line.markdown
> +++ b/docs/misc/xen-command-line.markdown
> @@ -1205,7 +1205,7 @@ detection of systems known to misbehave upon accesses 
> to that port.
>  >> Enable IOMMU debugging code (implies `verbose`).
>  
>  ### dom0-iommu
> -> `= List of [ none | strict | relaxed | inclusive ]`
> +> `= List of [ none | strict | relaxed | inclusive | reserved ]`
>  
>  * `none`: disables DMA remapping for Dom0.
>  
> @@ -1233,6 +1233,15 @@ meaning:
>option is only applicable to a PV Dom0 and is enabled by default on Intel
>hardware.
>  
> +* `reserved`: sets up DMA remapping for all the reserved regions in the  
> memory
> +  map for Dom0. Use this to work around firmware issues providing incorrect
> +  RMRR/IVMD entries. Rather than only mapping RAM pages for IOMMU accesses
> +  for Dom0, all memory regions marked as reserved in the memory map that 
> don't
> +  overlap with any MMIO region from emulated devices will be identity mapped.
> +  This option maps a subset of the memory that would be mapped when using the
> +  `inclusive` option. This option is available to a PVH Dom0 and is enabled 
> by
> +  default on Intel hardware.

The sub-options so far were quite clear in their meanings, but
"dom0-iommu=reserved" might mean all sorts of things to me, but quite
certainly not "map all reserved regions". "map-reserved" perhaps?

> --- a/xen/arch/x86/hvm/io.c
> +++ b/xen/arch/x86/hvm/io.c
> @@ -404,6 +404,11 @@ static const struct hvm_mmcfg *vpci_mmcfg_find(const 
> struct domain *d,
>  return NULL;
>  }
>  
> +bool vpci_mmcfg_address(const struct domain *d, paddr_t addr)
> +{
> +return vpci_mmcfg_find(d, addr);
> +}

I think the function name should have an "is_" somewhere, to make
clear address is a noun here and not a verb.

> --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
> +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
> @@ -256,6 +256,9 @@ static void __hwdom_init amd_iommu_hwdom_init(struct 
> domain *d)
>  /* Inclusive IOMMU mappings are disabled by default on AMD hardware. */
>  iommu_dom0_inclusive = iommu_dom0_inclusive == -1 ? false
>: iommu_dom0_inclusive;
> +/* Reserved IOMMU mappings are disabled by default on AMD hardware. */
> +iommu_dom0_reserved = iommu_dom0_reserved == -1 ? false
> +: iommu_dom0_reserved;

Especially seeing these two together now, I think an if() each would
be easier to read (not the least because of being shorter). To me,
the main purpose of the conditional operator is to allow shortening
simple if/else pairs, rather than lengthening simple if()-s.

> @@ -134,13 +135,67 @@ void arch_iommu_domain_destroy(struct domain *d)
>  {
>  }
>  
> +static bool __hwdom_init hwdom_iommu_map(const struct domain *d, unsigned 
> long pfn,
> + unsigned long max_pfn)
> +{
> +unsigned int i;
> +
> +/*
> + * Ignore any address below 1MB, that's already identity mapped by the
> + * domain builder for HVM.
> + */
> +if ( (is_hvm_domain(d) && pfn < PFN_DOWN(MB(1))) ||

Careful again here with the distinction between Dom0 and hwdom:
The domain builder you refer to is - aiui - the in-Xen one, i.e. the
one _only_ dealing with Dom0.

> +/*
> + * If dom0-strict mode is enabled or the guest type is PVH/HVM then 
> exclude
> + * conventional RAM and let the common code map dom0's pages.
> + */
> +if ( page_is_ram_type(pfn, RAM_TYPE_CONVENTIONAL) &&
> + (iommu_dom0_strict || is_hvm_domain(d)) )
> +return false;
> +if ( page_is_ram_type(pfn, RAM_TYPE_RESERVED) &&
> + !iommu_dom0_reserved && !iommu_dom0_inclusive )
> +return false;
> +if ( !page_is_ram_type(pfn, RAM_TYPE_UNUSABLE) &&
> + !page_is_ram_type(pfn, RAM_TYPE_RESERVED) &&
> + !page_is_ram_type(pfn, RAM_TYPE_CONVENTIONAL) &&
> + (!iommu_dom0_inclusive || pfn > max_pfn) )
> +return false;

As page_is_ram_type() is, especially on systems with many E820
entries, not really cheap, I think at least a minimum amount of
optimization is on order here - after all you do this for every

Re: [Xen-devel] [PATCH v21 2/2] tools/libxenctrl: use new xenforeignmemory API to seed grant table

2018-08-08 Thread Paul Durrant
> -Original Message-
> From: Andrew Cooper
> Sent: 08 August 2018 10:59
> To: Paul Durrant ; xen-devel@lists.xenproject.org
> Cc: Ian Jackson 
> Subject: Re: [Xen-devel] [PATCH v21 2/2] tools/libxenctrl: use new
> xenforeignmemory API to seed grant table
> 
> On 08/08/18 10:00, Paul Durrant wrote:
> > A previous patch added support for priv-mapping guest resources directly
> > (rather than having to foreign-map, which requires P2M modification for
> > HVM guests).
> >
> > This patch makes use of the new API to seed the guest grant table unless
> > the underlying infrastructure (i.e. privcmd) doesn't support it, in which
> > case the old scheme is used.
> >
> > NOTE: The call to xc_dom_gnttab_hvm_seed() in hvm_build_set_params()
> was
> >   actually unnecessary, as the grant table has already been seeded
> >   by a prior call to xc_dom_gnttab_init() made by libxl__build_dom().
> >
> > Signed-off-by: Paul Durrant 
> > Acked-by: Marek Marczykowski-Górecki
> 
> > Reviewed-by: Roger Pau Monné 
> > Acked-by: Wei Liu 
> 
> Some minor style issues, all of which can fixed on commit (probably).
> Everything else looks fine.

Cool.

> 
> > diff --git a/tools/libxc/include/xc_dom.h b/tools/libxc/include/xc_dom.h
> > index 8a66889c68..a8a0c0da66 100644
> > --- a/tools/libxc/include/xc_dom.h
> > +++ b/tools/libxc/include/xc_dom.h
> > @@ -337,12 +337,8 @@ void *xc_dom_boot_domU_map(struct
> xc_dom_image *dom, xen_pfn_t pfn,
> >  int xc_dom_boot_image(struct xc_dom_image *dom);
> >  int xc_dom_compat_check(struct xc_dom_image *dom);
> >  int xc_dom_gnttab_init(struct xc_dom_image *dom);
> > -int xc_dom_gnttab_hvm_seed(xc_interface *xch, uint32_t domid,
> > -   xen_pfn_t console_gmfn,
> > -   xen_pfn_t xenstore_gmfn,
> > -   uint32_t console_domid,
> > -   uint32_t xenstore_domid);
> > -int xc_dom_gnttab_seed(xc_interface *xch, uint32_t domid,
> > +int xc_dom_gnttab_seed(xc_interface *xch, uint32_t guest_domid,
> > +   bool is_hvm,
> > xen_pfn_t console_gmfn,
> > xen_pfn_t xenstore_gmfn,
> 
> As you're rewriting most of the functionality, can we switch to gfn to
> correct the terminology?

Sure. I'll restrict mods to the bits I'm touching though.

> 
> > uint32_t console_domid,
> > diff --git a/tools/libxc/xc_dom_boot.c b/tools/libxc/xc_dom_boot.c
> > index 2e5681dc5d..8307ebeaf6 100644
> > --- a/tools/libxc/xc_dom_boot.c
> > +++ b/tools/libxc/xc_dom_boot.c
> > @@ -256,11 +256,29 @@ static xen_pfn_t
> xc_dom_gnttab_setup(xc_interface *xch, uint32_t domid)
> >  return gmfn;
> >  }
> >
> > -int xc_dom_gnttab_seed(xc_interface *xch, uint32_t domid,
> > -   xen_pfn_t console_gmfn,
> > -   xen_pfn_t xenstore_gmfn,
> > -   uint32_t console_domid,
> > -   uint32_t xenstore_domid)
> > +static void xc_dom_set_gnttab_entry(xc_interface *xch,
> > +grant_entry_v1_t *gnttab,
> > +unsigned int idx,
> > +uint32_t guest_domid,
> > +uint32_t backend_domid,
> > +xen_pfn_t backend_gmfn)
> 
> gfn
> 
> > +{
> > +if ( guest_domid == backend_domid || backend_gmfn == -1)
> 
> Space at the end.
> 
> > +return;
> > +
> > +xc_dom_printf(xch, "%s: [%u] -> 0x%"PRI_xen_pfn,
> > +  __FUNCTION__, idx, backend_gmfn);
> 
> __func__ rather than __FUNCTION__.  Also, "[%u] => d%d 0x"PRI_xen_pfn
> would be more helpful for diagnostics.  (I do realise that backend
> domid's were retrofitted without adjusting the printf().)
> 
> > +
> > +gnttab[idx].flags = GTF_permit_access;
> > +gnttab[idx].domid = backend_domid;
> > +gnttab[idx].frame = backend_gmfn;
> > +}
> > +
> > +static int compat_gnttab_seed(xc_interface *xch, uint32_t domid,
> 
> compat_gnttab_pv_seed() ?
> 

This is actually used in the HVM compat case too. I'll deal with the other 
stylistic issues and add explicit version setting as discussed on the other 
patch thread.

  Paul

> > +  xen_pfn_t console_gmfn,
> > +  xen_pfn_t xenstore_gmfn,
> 
> gfn
> 
> > +  uint32_t console_domid,
> > +  uint32_t xenstore_domid)
> >  {
> >
> >  xen_pfn_t gnttab_gmfn;
> > @@ -313,11 +323,11 @@ int xc_dom_gnttab_seed(xc_interface *xch,
> uint32_t domid,
> >  return 0;
> >  }
> >
> > -int xc_dom_gnttab_hvm_seed(xc_interface *xch, uint32_t domid,
> > -   xen_pfn_t console_gpfn,
> > -   xen_pfn_t xenstore_gpfn,
> > -   uint32_t console_domid,
> > -   uint32_t xenstore_domid)
> > +static int 

Re: [Xen-devel] [PATCH v4 3/4] dom0/pvh: change the order of the MMCFG initialization

2018-08-08 Thread Jan Beulich
>>> On 08.08.18 at 12:07,  wrote:
> So it's done before the iommu is initialized. This is required in
> order to be able to fetch the MMCFG regions from the domain struct.

Is this a useful change to make? Regions not reported through the MCFG
table will need punching holes anyway, so why not punch holes uniformly
in all cases, allowing the hole punching code to be tested even on systems
without non-boot-time-available regions?

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v4 2/4] iommu: make iommu_inclusive_mapping a suboption of dom0-iommu

2018-08-08 Thread Jan Beulich
>>> On 08.08.18 at 12:07,  wrote:
> Introduce a new dom0-iommu=inclusive generic option that supersedes
> iommu_inclusive_mapping. The previous behaviour is preserved and the
> option should only be enabled by default on Intel hardware.

Why "should" instead of "is"?

> @@ -1221,6 +1221,18 @@ PV Dom0:
>  Note that all the above options are mutually exclusive. Specifying more than
>  one on the `dom0-iommu` command line will result in undefined behavior.
>  
> +The following options control whether non-RAM regions are added to the Dom0
> +iommu tables. Note that they can be prefixed with `no-` to effect the inverse
> +meaning:

I'm not particularly happy about the mentioning of "no-" here: Why is
this better than the also permitted "=0" etc suffixes? Keep it generic,
just like other options do.

> +* `inclusive`: sets up DMA remapping for all the non-RAM memory below 4GB
> +  except for unusable ranges. Use this to work around firmware issues 
> providing
> +  incorrect RMRR/IVMD entries. Rather than only mapping RAM pages for IOMMU
> +  accesses for Dom0, with this option all pages up to 4GB, not marked as
> +  unusable in the E820 table, will get a mapping established. Note that this
> +  option is only applicable to a PV Dom0 and is enabled by default on Intel
> +  hardware.

No word at all about the interaction with none/strict/relaxed? I think,
as mentioned for patch 1, "none" renders this option meaningless as
well. But for "relaxed" it's pretty unclear, because from E820 alone
you can't judge whether e.g. a reserved region is RAM or MMIO. (As
an implication, the mentioning of RAM in patch 1's doc for "relaxed"
then looks symmetrically wrong, just like I've already asked to replace
"memory" by "RAM" for "strict".)

> --- a/xen/drivers/passthrough/arm/iommu.c
> +++ b/xen/drivers/passthrough/arm/iommu.c
> @@ -73,3 +73,7 @@ int arch_iommu_populate_page_table(struct domain *d)
>  /* The IOMMU shares the p2m with the CPU */
>  return -ENOSYS;
>  }
> +
> +void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
> +{
> +}

The option being in common code, I think you also need to set it for
ARM, so it won't remain at its default of -1.

> @@ -144,16 +145,23 @@ static int __init parse_dom0_iommu_param(const char *s)
>  int rc = 0;
>  
>  do {
> +bool val = !!strncmp(s, "no-", 3);

Oh, you do a literal comparison against "no-". Please don't, that's what
we have parse_boolean() for.

> @@ -202,6 +210,13 @@ void __hwdom_init iommu_hwdom_init(struct domain *d)
>  if ( !iommu_enabled )
>  return;
>  
> +if ( iommu_dom0_inclusive == true && !is_pv_domain(d) )

Why the "== true"? It shouldn't have its initializer value of -1 anymore
at this point.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [ovmf test] 125790: all pass - PUSHED

2018-08-08 Thread osstest service owner
flight 125790 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/125790/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf b6e48ec6412eab0f21fdff5a045a7ee516574d44
baseline version:
 ovmf 91a5b13650752a54cf766791aa369495c3426485

Last test of basis   125775  2018-08-06 19:11:00 Z1 days
Testing same since   125790  2018-08-07 14:59:27 Z0 days1 attempts


People who touched revisions under test:
  Ruiyu Ni 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   91a5b13650..b6e48ec641  b6e48ec6412eab0f21fdff5a045a7ee516574d44 -> 
xen-tested-master

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v4 1/4] iommu: introduce dom0-iommu option

2018-08-08 Thread Jan Beulich
>>> On 08.08.18 at 12:07,  wrote:
> @@ -1198,6 +1204,23 @@ detection of systems known to misbehave upon accesses 
> to that port.
>  
>  >> Enable IOMMU debugging code (implies `verbose`).
>  
> +### dom0-iommu

This is now misplaced, as the file is (meant to be) alphabetically
sorted.

> +> `= List of [ none | strict | relaxed ]`
> +
> +* `none`: disables DMA remapping for Dom0.
> +
> +The following two options control how RAM regions are mapped in the iommu for
> +PV Dom0:
> +
> +* `strict`: sets up DMA remapping only for the memory Dom0 actually got
> +  assigned.

s/memory/RAM/ ?

> +* `relaxed`: sets DMA remapping for all the host RAM except regions in use by
> +  Xen. This is the default iommu behaviour.

Drop "iommu" here?

> +Note that all the above options are mutually exclusive. Specifying more than
> +one on the `dom0-iommu` command line will result in undefined behavior.

Isn't this more strict than it needs to be? "none", afaict, always takes
precedence if enabled. What color a bike shed is simply doesn't matter
when it doesn't exist.

> --- a/xen/arch/x86/x86_64/mm.c
> +++ b/xen/arch/x86/x86_64/mm.c
> @@ -1426,7 +1426,8 @@ int memory_add(unsigned long spfn, unsigned long epfn, 
> unsigned int pxm)
>  if ( ret )
>  goto destroy_m2p;
>  
> -if ( iommu_enabled && !iommu_passthrough && !need_iommu(hardware_domain) 
> )
> +if ( iommu_enabled && !iommu_dom0_passthrough &&
> + !need_iommu(hardware_domain) )

This makes already clear that you need to better distinguish Dom0 and
hwdom here, but it's not immediately clear to me which direction the
changes should be made: Do you mean truly only Dom0 throughout
this patch, or hwdom? While the doc and command line option name can
perhaps left as is, internal variable names should not say Dom0 when
they don't mean Dom0. Otoh if you mean only Dom0, then the use of
hardware_domain above (and elsewhere) is now wrong.

Of course I won't demand (but even less so object to) you renaming
the other related variable that is affected here.

> --- a/xen/drivers/passthrough/iommu.c
> +++ b/xen/drivers/passthrough/iommu.c
> @@ -22,6 +22,7 @@
>  #include 
>  
>  static int parse_iommu_param(const char *s);
> +static int parse_dom0_iommu_param(const char *s);

Please don't. Instead ...

> @@ -72,6 +71,10 @@ bool_t __read_mostly iommu_hap_pt_share = 1;
>  bool_t __read_mostly iommu_debug;
>  bool_t __read_mostly amd_iommu_perdev_intremap = 1;
>  
> +custom_param("dom0-iommu", parse_dom0_iommu_param);

... move this immediately after (with no intervening blank line)
parse_dom0_iommu_param()'s definition.

> +static int __init parse_dom0_iommu_param(const char *s)
> +{
> +const char *ss;
> +int rc = 0;
> +
> +do {
> +ss = strchr(s, ',');
> +if ( !ss )
> +ss = strchr(s, '\0');
> +
> +if ( !strncmp(s, "none", ss - s) )
> +iommu_dom0_passthrough = true;
> +else if ( !strncmp(s, "strict", ss - s) )
> +iommu_dom0_strict = true;
> +else if ( !strncmp(s, "relaxed", ss - s) )
> +iommu_dom0_strict = false;

Perhaps better just have one of the two, and make them truly
boolean? Or would that conflict with further patches / plans?

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] xen/balloon: fix balloon initialization for PVH Dom0

2018-08-08 Thread Juergen Gross
On 08/08/18 13:46, Roger Pau Monne wrote:
> The current balloon code tries to calculate a delta factor for the
> balloon target when running in HVM mode in order to account for memory
> used by the firmware.
> 
> This workaround for memory accounting doesn't work properly on a PVH
> Dom0, that has a static-max value different from the target value even
> at startup. Note that this is not a problem for DomUs because guests are
> started with a static-max value that matches the amount of RAM in the
> memory map.
> 
> Fix this by forcefully setting target_diff for Dom0, regardless of
> it's mode.
> 
> Reported-by: Gabriel Bercarug 
> Signed-off-by: Roger Pau Monné 

Reviewed-by: Juergen Gross 


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3] x86/hvm: remove default ioreq server

2018-08-08 Thread Jan Beulich
>>> On 08.08.18 at 13:38,  wrote:
> On 08/08/18 11:48, Jan Beulich wrote:
> On 08.08.18 at 12:39,  wrote:
 From: Jan Beulich [mailto:jbeul...@suse.com]
 Sent: 08 August 2018 11:11

>>> On 07.08.18 at 17:42,  wrote:
> --- a/xen/include/public/hvm/params.h
> +++ b/xen/include/public/hvm/params.h
> @@ -81,9 +81,13 @@
>  #define HVM_PARAM_PAE_ENABLED  4
>
>  #define HVM_PARAM_IOREQ_PFN5
> -
>  #define HVM_PARAM_BUFIOREQ_PFN 6
> +
> +#ifdef __XEN__
> +/* These parameters is deprecated and its meaning is undefined */
> +#define HVM_PARAM_DM_DOMAIN 13
>  #define HVM_PARAM_BUFIOREQ_EVTCHN 26
> +#endif
 The comment was only partly switched to plural.

>>> Oh, so it was. I guess this can be fixed on commit unless you'd prefer I 
>>> send a v4.
>> Oh, of course it can - I meant to say so, but then forgot.
> 
> One note however.  This patch cannot be committed until the associated
> patch is committed into qemu-trad, which is waiting on IanJ as the
> maintainer of that tree.

And not just committed, but also has passed that other push gate.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH] xen/balloon: fix balloon initialization for PVH Dom0

2018-08-08 Thread Roger Pau Monne
The current balloon code tries to calculate a delta factor for the
balloon target when running in HVM mode in order to account for memory
used by the firmware.

This workaround for memory accounting doesn't work properly on a PVH
Dom0, that has a static-max value different from the target value even
at startup. Note that this is not a problem for DomUs because guests are
started with a static-max value that matches the amount of RAM in the
memory map.

Fix this by forcefully setting target_diff for Dom0, regardless of
it's mode.

Reported-by: Gabriel Bercarug 
Signed-off-by: Roger Pau Monné 
---
Cc: Gabriel Ercarug 
Cc: Boris Ostrovsky 
Cc: Juergen Gross 
Cc: xen-devel@lists.xenproject.org
---
 drivers/xen/xen-balloon.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/xen/xen-balloon.c b/drivers/xen/xen-balloon.c
index b437fccd4e62..294f35ce9e46 100644
--- a/drivers/xen/xen-balloon.c
+++ b/drivers/xen/xen-balloon.c
@@ -81,7 +81,7 @@ static void watch_target(struct xenbus_watch *watch,
static_max = new_target;
else
static_max >>= PAGE_SHIFT - 10;
-   target_diff = xen_pv_domain() ? 0
+   target_diff = (xen_pv_domain() || xen_initial_domain()) ? 0
: static_max - balloon_stats.target_pages;
}
 
-- 
2.18.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3] x86/hvm: remove default ioreq server

2018-08-08 Thread Andrew Cooper
On 08/08/18 11:48, Jan Beulich wrote:
 On 08.08.18 at 12:39,  wrote:
>>> From: Jan Beulich [mailto:jbeul...@suse.com]
>>> Sent: 08 August 2018 11:11
>>>
>> On 07.08.18 at 17:42,  wrote:
 --- a/xen/include/public/hvm/params.h
 +++ b/xen/include/public/hvm/params.h
 @@ -81,9 +81,13 @@
  #define HVM_PARAM_PAE_ENABLED  4

  #define HVM_PARAM_IOREQ_PFN5
 -
  #define HVM_PARAM_BUFIOREQ_PFN 6
 +
 +#ifdef __XEN__
 +/* These parameters is deprecated and its meaning is undefined */
 +#define HVM_PARAM_DM_DOMAIN 13
  #define HVM_PARAM_BUFIOREQ_EVTCHN 26
 +#endif
>>> The comment was only partly switched to plural.
>>>
>> Oh, so it was. I guess this can be fixed on commit unless you'd prefer I 
>> send a v4.
> Oh, of course it can - I meant to say so, but then forgot.

One note however.  This patch cannot be committed until the associated
patch is committed into qemu-trad, which is waiting on IanJ as the
maintainer of that tree.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v21 1/2] common: add a new mappable resource type: XENMEM_resource_grant_table

2018-08-08 Thread Paul Durrant
> -Original Message-
> From: Andrew Cooper
> Sent: 08 August 2018 12:34
> To: Jan Beulich 
> Cc: George Dunlap ; Ian Jackson
> ; Paul Durrant ; Wei Liu
> ; Stefano Stabellini ; xen-
> devel ; Konrad Rzeszutek Wilk
> ; Tim (Xen.org) 
> Subject: Re: [PATCH v21 1/2] common: add a new mappable resource type:
> XENMEM_resource_grant_table
> 
> On 08/08/18 11:55, Jan Beulich wrote:
>  On 08.08.18 at 12:46,  wrote:
> >> On 08/08/18 11:43, Jan Beulich wrote:
> >> On 08.08.18 at 12:38,  wrote:
> > From: Jan Beulich [mailto:jbeul...@suse.com]
> > Sent: 08 August 2018 11:30
> >
>  On 08.08.18 at 11:00,  wrote:
> >> +int gnttab_get_shared_frame(struct domain *d, unsigned long idx,
> >> +mfn_t *mfn)
> >> +{
> >> +struct grant_table *gt = d->grant_table;
> >> +int rc;
> >> +
> >> +grant_write_lock(gt);
> >> +
> >> +if ( gt->gt_version == 0 )
> >> +gt->gt_version = 1;
> > Since you've moved this here instead of dropping it, what
> requirement
> > have you found for this to be set (other than the ASSERT() you put in
> > gnttab_get_shared_frame_mfn()?
> >
>  The code in patch #2 is executed before the grant table version is set. I
>  could alternatively have libxl explicitly set the version to 1 before 
>  trying
>  to seed the table.
> >>> But that's not my point. What's wrong with leaving it at zero?
> >> On a tangent, why does a gnttab version of 0 exist at all?  Why don't we
> >> explicitly initialise it to 1 in the hypervisor?
> > Fair question, which unfortunately I can't answer.
> >
> >> We've had plenty of XSAs to do with mishandling of a gnttab version of
> >> 0.  Why not fix the problem at its source, and simplify the gnttab code
> >> while we are at it.
> > I don't mind, unless a reason for the seemingly strange behavior can be
> > found.
> 
> gt_version only came in with grant table v2, so the toolstack never
> previously set a version.  That ended up with cases where dom0 tries to
> map the store/cons grants before the guest driver has chosen a version.
> 
> I expect its like this because grant table v2 was a giant pile of poorly
> reviewed hacks, rather than for any better reason.
> 
> If noone else gets to it, I'll fold it into my series to mess thoroughly
> with parameter handling in DOMCTL_createdomain.

If that's going to take a while then I can add the explicit version set in 
patch #2.

  Paul

> 
> ~Andrew
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v21 1/2] common: add a new mappable resource type: XENMEM_resource_grant_table

2018-08-08 Thread Andrew Cooper
On 08/08/18 11:55, Jan Beulich wrote:
 On 08.08.18 at 12:46,  wrote:
>> On 08/08/18 11:43, Jan Beulich wrote:
>> On 08.08.18 at 12:38,  wrote:
> From: Jan Beulich [mailto:jbeul...@suse.com]
> Sent: 08 August 2018 11:30
>
 On 08.08.18 at 11:00,  wrote:
>> +int gnttab_get_shared_frame(struct domain *d, unsigned long idx,
>> +mfn_t *mfn)
>> +{
>> +struct grant_table *gt = d->grant_table;
>> +int rc;
>> +
>> +grant_write_lock(gt);
>> +
>> +if ( gt->gt_version == 0 )
>> +gt->gt_version = 1;
> Since you've moved this here instead of dropping it, what requirement
> have you found for this to be set (other than the ASSERT() you put in
> gnttab_get_shared_frame_mfn()?
>
 The code in patch #2 is executed before the grant table version is set. I 
 could alternatively have libxl explicitly set the version to 1 before 
 trying 
 to seed the table.
>>> But that's not my point. What's wrong with leaving it at zero?
>> On a tangent, why does a gnttab version of 0 exist at all?  Why don't we
>> explicitly initialise it to 1 in the hypervisor?
> Fair question, which unfortunately I can't answer.
>
>> We've had plenty of XSAs to do with mishandling of a gnttab version of
>> 0.  Why not fix the problem at its source, and simplify the gnttab code
>> while we are at it.
> I don't mind, unless a reason for the seemingly strange behavior can be
> found.

gt_version only came in with grant table v2, so the toolstack never
previously set a version.  That ended up with cases where dom0 tries to
map the store/cons grants before the guest driver has chosen a version.

I expect its like this because grant table v2 was a giant pile of poorly
reviewed hacks, rather than for any better reason.

If noone else gets to it, I'll fold it into my series to mess thoroughly
with parameter handling in DOMCTL_createdomain.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v21 1/2] common: add a new mappable resource type: XENMEM_resource_grant_table

2018-08-08 Thread Paul Durrant
> -Original Message-
> From: Jan Beulich [mailto:jbeul...@suse.com]
> Sent: 08 August 2018 11:43
> To: Paul Durrant 
> Cc: Andrew Cooper ; George Dunlap
> ; Ian Jackson ; Wei Liu
> ; Stefano Stabellini ; xen-
> devel ; Konrad Rzeszutek Wilk
> ; Tim (Xen.org) 
> Subject: RE: [PATCH v21 1/2] common: add a new mappable resource type:
> XENMEM_resource_grant_table
> 
> >>> On 08.08.18 at 12:38,  wrote:
> >> From: Jan Beulich [mailto:jbeul...@suse.com]
> >> Sent: 08 August 2018 11:30
> >>
> >> >>> On 08.08.18 at 11:00,  wrote:
> >> > @@ -3860,6 +3866,47 @@ int mem_sharing_gref_to_gfn(struct
> >> grant_table *gt, grant_ref_t ref,
> >> >  }
> >> >  #endif
> >> >
> >> > +/* caller must hold write lock */
> >> > +static int gnttab_get_status_frame_mfn(struct domain *d,
> >> > +   unsigned long idx, mfn_t *mfn)
> >> > +{
> >> > +struct grant_table *gt = d->grant_table;
> >>
> >> const?
> >>
> >
> > IIRC that didn't work because gnttab_grow_table() modifies the content.
> 
> But you don't pass gt to the function:
> 
> >> > +ASSERT(gt->gt_version == 2);
> >> > +
> >> > +if ( idx >= nr_status_frames(gt) )
> >> > +{
> >> > +unsigned long nr = status_to_grant_frames(idx + 1);
> >> > +
> >> > +if ( nr <= gt->max_grant_frames )
> >> > +gnttab_grow_table(d, nr);
> 
> ^^^

I know, I was just remembering that the compiler complained when I tried this 
before. My memory could be wrong so I'll try it again.

> 
> >> > @@ -3906,6 +3943,38 @@ int gnttab_map_frame(struct domain *d,
> >> unsigned long idx, gfn_t gfn,
> >> >  return rc;
> >> >  }
> >> >
> >> > +int gnttab_get_shared_frame(struct domain *d, unsigned long idx,
> >> > +mfn_t *mfn)
> >> > +{
> >> > +struct grant_table *gt = d->grant_table;
> >> > +int rc;
> >> > +
> >> > +grant_write_lock(gt);
> >> > +
> >> > +if ( gt->gt_version == 0 )
> >> > +gt->gt_version = 1;
> >>
> >> Since you've moved this here instead of dropping it, what requirement
> >> have you found for this to be set (other than the ASSERT() you put in
> >> gnttab_get_shared_frame_mfn()?
> >>
> >
> > The code in patch #2 is executed before the grant table version is set. I
> > could alternatively have libxl explicitly set the version to 1 before trying
> > to seed the table.
> 
> But that's not my point. What's wrong with leaving it at zero?
> 

I'm not particularly happy calling gnttab_grow_table() with version left at 0 
but I can try it and see if it breaks.

  Paul

> Jan
> 


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v21 1/2] common: add a new mappable resource type: XENMEM_resource_grant_table

2018-08-08 Thread Paul Durrant
> -Original Message-
> From: Jan Beulich [mailto:jbeul...@suse.com]
> Sent: 08 August 2018 11:47
> To: Andrew Cooper ; Paul Durrant
> 
> Cc: Wei Liu ; George Dunlap
> ; Ian Jackson ;
> Stefano Stabellini ; xen-devel  de...@lists.xenproject.org>; Konrad Rzeszutek Wilk
> ; Tim (Xen.org) 
> Subject: Re: [PATCH v21 1/2] common: add a new mappable resource type:
> XENMEM_resource_grant_table
> 
> >>> On 08.08.18 at 12:10,  wrote:
> > On 08/08/18 10:00, Paul Durrant wrote:
> >> +static int gnttab_get_status_frame_mfn(struct domain *d,
> >> +   unsigned long idx, mfn_t *mfn)
> >> +{
> >> +struct grant_table *gt = d->grant_table;
> >> +
> >> +ASSERT(gt->gt_version == 2);
> >> +
> >> +if ( idx >= nr_status_frames(gt) )
> >> +{
> >> +unsigned long nr = status_to_grant_frames(idx + 1);
> >
> > Why the +1 ? Won't that cause a failure if you attempt to map the
> > maximum valid index?
> 
> That's the normal index-of-something to count-of-something
> conversion (or else the table may get grown by too little). I've
> instead been considering the badness of overflow here, but
> decided to leave it uncommented as the check further down
> would at least not make this insecure. However, with ...
> 
> >> +
> >> +if ( nr <= gt->max_grant_frames )
> >> +gnttab_grow_table(d, nr);
> >
> > You want to capture the return value of grow_table(), which at least
> > distinguishes between ENODEV and ENOMEM.
> >
> >> +
> >> +if ( idx >= nr_status_frames(gt) )
> >> +return -EINVAL;
> >
> > This can probably(?) be asserted if gnttab_grow_table() returns
> > successfully.
> 
> ... these two a potential overflow above would then have a
> chance of triggering the assertion you suggest to add.
> 
> As to the grow_table() return value check - I'd prefer if the
> patch here didn't alter original behavior. If we want it altered,
> better in a separate patch.

Ok. I'll leave the return value of gnttab_grow_table() unchecked as-is.

> 
> Jan
> 


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v21 1/2] common: add a new mappable resource type: XENMEM_resource_grant_table

2018-08-08 Thread Jan Beulich
>>> On 08.08.18 at 12:46,  wrote:
> On 08/08/18 11:43, Jan Beulich wrote:
> On 08.08.18 at 12:38,  wrote:
 From: Jan Beulich [mailto:jbeul...@suse.com]
 Sent: 08 August 2018 11:30

>>> On 08.08.18 at 11:00,  wrote:
> +int gnttab_get_shared_frame(struct domain *d, unsigned long idx,
> +mfn_t *mfn)
> +{
> +struct grant_table *gt = d->grant_table;
> +int rc;
> +
> +grant_write_lock(gt);
> +
> +if ( gt->gt_version == 0 )
> +gt->gt_version = 1;
 Since you've moved this here instead of dropping it, what requirement
 have you found for this to be set (other than the ASSERT() you put in
 gnttab_get_shared_frame_mfn()?

>>> The code in patch #2 is executed before the grant table version is set. I 
>>> could alternatively have libxl explicitly set the version to 1 before 
>>> trying 
>>> to seed the table.
>> But that's not my point. What's wrong with leaving it at zero?
> 
> On a tangent, why does a gnttab version of 0 exist at all?  Why don't we
> explicitly initialise it to 1 in the hypervisor?

Fair question, which unfortunately I can't answer.

> We've had plenty of XSAs to do with mishandling of a gnttab version of
> 0.  Why not fix the problem at its source, and simplify the gnttab code
> while we are at it.

I don't mind, unless a reason for the seemingly strange behavior can be
found.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3] x86/hvm: remove default ioreq server

2018-08-08 Thread Jan Beulich
>>> On 08.08.18 at 12:39,  wrote:
>> From: Jan Beulich [mailto:jbeul...@suse.com]
>> Sent: 08 August 2018 11:11
>> 
>> >>> On 07.08.18 at 17:42,  wrote:
>> > --- a/xen/include/public/hvm/params.h
>> > +++ b/xen/include/public/hvm/params.h
>> > @@ -81,9 +81,13 @@
>> >  #define HVM_PARAM_PAE_ENABLED  4
>> >
>> >  #define HVM_PARAM_IOREQ_PFN5
>> > -
>> >  #define HVM_PARAM_BUFIOREQ_PFN 6
>> > +
>> > +#ifdef __XEN__
>> > +/* These parameters is deprecated and its meaning is undefined */
>> > +#define HVM_PARAM_DM_DOMAIN 13
>> >  #define HVM_PARAM_BUFIOREQ_EVTCHN 26
>> > +#endif
>> 
>> The comment was only partly switched to plural.
>> 
> 
> Oh, so it was. I guess this can be fixed on commit unless you'd prefer I 
> send a v4.

Oh, of course it can - I meant to say so, but then forgot.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v21 1/2] common: add a new mappable resource type: XENMEM_resource_grant_table

2018-08-08 Thread Jan Beulich
>>> On 08.08.18 at 12:10,  wrote:
> On 08/08/18 10:00, Paul Durrant wrote:
>> +static int gnttab_get_status_frame_mfn(struct domain *d,
>> +   unsigned long idx, mfn_t *mfn)
>> +{
>> +struct grant_table *gt = d->grant_table;
>> +
>> +ASSERT(gt->gt_version == 2);
>> +
>> +if ( idx >= nr_status_frames(gt) )
>> +{
>> +unsigned long nr = status_to_grant_frames(idx + 1);
> 
> Why the +1 ? Won't that cause a failure if you attempt to map the
> maximum valid index?

That's the normal index-of-something to count-of-something
conversion (or else the table may get grown by too little). I've
instead been considering the badness of overflow here, but
decided to leave it uncommented as the check further down
would at least not make this insecure. However, with ...

>> +
>> +if ( nr <= gt->max_grant_frames )
>> +gnttab_grow_table(d, nr);
> 
> You want to capture the return value of grow_table(), which at least
> distinguishes between ENODEV and ENOMEM.
> 
>> +
>> +if ( idx >= nr_status_frames(gt) )
>> +return -EINVAL;
> 
> This can probably(?) be asserted if gnttab_grow_table() returns
> successfully.

... these two a potential overflow above would then have a
chance of triggering the assertion you suggest to add.

As to the grow_table() return value check - I'd prefer if the
patch here didn't alter original behavior. If we want it altered,
better in a separate patch.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v21 1/2] common: add a new mappable resource type: XENMEM_resource_grant_table

2018-08-08 Thread Andrew Cooper
On 08/08/18 11:43, Jan Beulich wrote:
 On 08.08.18 at 12:38,  wrote:
>>> From: Jan Beulich [mailto:jbeul...@suse.com]
>>> Sent: 08 August 2018 11:30
>>>
>> On 08.08.18 at 11:00,  wrote:
 @@ -3860,6 +3866,47 @@ int mem_sharing_gref_to_gfn(struct
>>> grant_table *gt, grant_ref_t ref,
  }
  #endif

 +/* caller must hold write lock */
 +static int gnttab_get_status_frame_mfn(struct domain *d,
 +   unsigned long idx, mfn_t *mfn)
 +{
 +struct grant_table *gt = d->grant_table;
>>> const?
>>>
>> IIRC that didn't work because gnttab_grow_table() modifies the content.
> But you don't pass gt to the function:
>
 +ASSERT(gt->gt_version == 2);
 +
 +if ( idx >= nr_status_frames(gt) )
 +{
 +unsigned long nr = status_to_grant_frames(idx + 1);
 +
 +if ( nr <= gt->max_grant_frames )
 +gnttab_grow_table(d, nr);
> ^^^
>
 @@ -3906,6 +3943,38 @@ int gnttab_map_frame(struct domain *d,
>>> unsigned long idx, gfn_t gfn,
  return rc;
  }

 +int gnttab_get_shared_frame(struct domain *d, unsigned long idx,
 +mfn_t *mfn)
 +{
 +struct grant_table *gt = d->grant_table;
 +int rc;
 +
 +grant_write_lock(gt);
 +
 +if ( gt->gt_version == 0 )
 +gt->gt_version = 1;
>>> Since you've moved this here instead of dropping it, what requirement
>>> have you found for this to be set (other than the ASSERT() you put in
>>> gnttab_get_shared_frame_mfn()?
>>>
>> The code in patch #2 is executed before the grant table version is set. I 
>> could alternatively have libxl explicitly set the version to 1 before trying 
>> to seed the table.
> But that's not my point. What's wrong with leaving it at zero?

On a tangent, why does a gnttab version of 0 exist at all?  Why don't we
explicitly initialise it to 1 in the hypervisor?

We've had plenty of XSAs to do with mishandling of a gnttab version of
0.  Why not fix the problem at its source, and simplify the gnttab code
while we are at it.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v21 1/2] common: add a new mappable resource type: XENMEM_resource_grant_table

2018-08-08 Thread Andrew Cooper
On 08/08/18 11:19, Paul Durrant wrote:
>
>>> +static inline unsigned int status_to_grant_frames(unsigned int
>> status_frames)
>>> +{
>>> +return DIV_ROUND_UP(status_frames * GRANT_STATUS_PER_PAGE,
>> GRANT_PER_PAGE);
>>> +}
>>> +
>>>  /* Check if the page has been paged out, or needs unsharing.
>>> If rc == GNTST_okay, *page contains the page struct with a ref taken.
>>> Caller must do put_page(*page).
>>> @@ -3860,6 +3866,47 @@ int mem_sharing_gref_to_gfn(struct
>> grant_table *gt, grant_ref_t ref,
>>>  }
>>>  #endif
>>>
>>> +/* caller must hold write lock */
>>> +static int gnttab_get_status_frame_mfn(struct domain *d,
>>> +   unsigned long idx, mfn_t *mfn)
>>> +{
>>> +struct grant_table *gt = d->grant_table;
>>> +
>>> +ASSERT(gt->gt_version == 2);
>>> +
>>> +if ( idx >= nr_status_frames(gt) )
>>> +{
>>> +unsigned long nr = status_to_grant_frames(idx + 1);
>> Why the +1 ? Won't that cause a failure if you attempt to map the
>> maximum valid index?
> That's kind of the idea...

To force a failure when mapping a legitimate index?  Whatever the old
code did, this smells like broken boundary case.

A caller is going to want to map frames 0 to N-1, based on some
knowledge of either how many frames the guest has, or what the total
number of expected frames is.  nr_status_frames() OTOH, is 1-based,
which is why this looks wrong.

How about a comment along the lines of

/* idx is 0-based, nr_* is 1-based. */

which might help reduce the confusion?

>>> +
>>> +if ( idx >= nr_grant_frames(gt) )
>>> +return -EINVAL;
>>> +
>>> +*mfn = _mfn(virt_to_mfn(gt->shared_raw[idx]));
>>> +return 0;
>>> +}
>>> +
>>>  int gnttab_map_frame(struct domain *d, unsigned long idx, gfn_t gfn,
>>>   mfn_t *mfn)
>>>  {
>>> diff --git a/xen/common/memory.c b/xen/common/memory.c
>>> index e29d596727..427f65a5e1 100644
>>> --- a/xen/common/memory.c
>>> +++ b/xen/common/memory.c
>>> @@ -23,6 +23,7 @@
>>>  #include 
>>>  #include 
>>>  #include 
>>> +#include 
>>>  #include 
>>>  #include 
>>>  #include 
>>> @@ -982,6 +983,43 @@ static long xatp_permission_check(struct domain
>> *d, unsigned int space)
>>>  return xsm_add_to_physmap(XSM_TARGET, current->domain, d);
>>>  }
>>>
>>> +static int acquire_grant_table(struct domain *d, unsigned int id,
>>> +   unsigned long frame,
>>> +   unsigned int nr_frames,
>>> +   xen_pfn_t mfn_list[])
>>> +{
>>> +unsigned int i = nr_frames;
>>> +
>>> +/* Iterate backwards in case table needs to grow */
>>> +while ( i-- != 0 )
>>> +{
>>> +mfn_t mfn = INVALID_MFN;
>>> +int rc;
>>> +
>>> +switch ( id )
>>> +{
>>> +case XENMEM_resource_grant_table_id_shared:
>>> +rc = gnttab_get_shared_frame(d, frame + i, );
>>> +break;
>>> +
>>> +case XENMEM_resource_grant_table_id_status:
>>> +rc = gnttab_get_status_frame(d, frame + i, );
>>> +break;
>>> +
>>> +default:
>>> +rc = -EINVAL;
>>> +break;
>>> +}
>>> +
>>> +if ( rc )
>> Would it perhaps be prudent to have || mfn_eq(mfn, INVALID_MFN)
>> here?  I
>> don't think anything good will come of handing INVALID_MFN back to the
>> caller.
> Why? The value of mfn will be overwritten if rc == 0 and will be left as 
> INVALID_MFN if rc != 0. I can ASSERT that if you'd like.

Yeah - an ASSERT() would be nice.

>>> +!is_hardware_domain(currd) )
>>> +return -EOPNOTSUPP;
>> EPERM perhaps?
>>
> I debated that. I'm really not sure which one would be best.

Hmm, nor me.  Lets leave it like that for now.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v21 1/2] common: add a new mappable resource type: XENMEM_resource_grant_table

2018-08-08 Thread Jan Beulich
>>> On 08.08.18 at 12:38,  wrote:
>> From: Jan Beulich [mailto:jbeul...@suse.com]
>> Sent: 08 August 2018 11:30
>> 
>> >>> On 08.08.18 at 11:00,  wrote:
>> > @@ -3860,6 +3866,47 @@ int mem_sharing_gref_to_gfn(struct
>> grant_table *gt, grant_ref_t ref,
>> >  }
>> >  #endif
>> >
>> > +/* caller must hold write lock */
>> > +static int gnttab_get_status_frame_mfn(struct domain *d,
>> > +   unsigned long idx, mfn_t *mfn)
>> > +{
>> > +struct grant_table *gt = d->grant_table;
>> 
>> const?
>> 
> 
> IIRC that didn't work because gnttab_grow_table() modifies the content.

But you don't pass gt to the function:

>> > +ASSERT(gt->gt_version == 2);
>> > +
>> > +if ( idx >= nr_status_frames(gt) )
>> > +{
>> > +unsigned long nr = status_to_grant_frames(idx + 1);
>> > +
>> > +if ( nr <= gt->max_grant_frames )
>> > +gnttab_grow_table(d, nr);

^^^

>> > @@ -3906,6 +3943,38 @@ int gnttab_map_frame(struct domain *d,
>> unsigned long idx, gfn_t gfn,
>> >  return rc;
>> >  }
>> >
>> > +int gnttab_get_shared_frame(struct domain *d, unsigned long idx,
>> > +mfn_t *mfn)
>> > +{
>> > +struct grant_table *gt = d->grant_table;
>> > +int rc;
>> > +
>> > +grant_write_lock(gt);
>> > +
>> > +if ( gt->gt_version == 0 )
>> > +gt->gt_version = 1;
>> 
>> Since you've moved this here instead of dropping it, what requirement
>> have you found for this to be set (other than the ASSERT() you put in
>> gnttab_get_shared_frame_mfn()?
>> 
> 
> The code in patch #2 is executed before the grant table version is set. I 
> could alternatively have libxl explicitly set the version to 1 before trying 
> to seed the table.

But that's not my point. What's wrong with leaving it at zero?

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3] x86/hvm: remove default ioreq server

2018-08-08 Thread Paul Durrant
> -Original Message-
> From: Jan Beulich [mailto:jbeul...@suse.com]
> Sent: 08 August 2018 11:11
> To: Paul Durrant 
> Cc: xen-devel 
> Subject: Re: [PATCH v3] x86/hvm: remove default ioreq server
> 
> >>> On 07.08.18 at 17:42,  wrote:
> > My recent patch [1] to qemu-xen-traditional removes the last use of the
> > 'default' ioreq server in Xen. (This is a catch-all ioreq server that is
> > used if no explicitly registered I/O range is targetted).
> >
> > This patch can be applied once that patch is committed, to remove the
> > (>100 lines of) redundant code in Xen.
> >
> > NOTE: The removal of the special case for HVM_PARAM_DM_DOMAIN in
> >   hvm_allow_set_param() is not directly related to removal of
> >   default ioreq servers. It could have been cleaned up at any time
> >   after commit 9a422c03 "x86/hvm: stop passing explicit domid to
> >   hvm_create_ioreq_server()". It is now added to the new
> >   deprecated sets introduced by this patch.
> >
> > [1] https://lists.xenproject.org/archives/html/xen-devel/2018-
> 08/msg00270.html
> >
> > Signed-off-by: Paul Durrant 
> > Acked-by: Andrew Cooper 
> 
> Reviewed-by: Jan Beulich 

Thanks.

> with one more cosmetic adjustment:
> 
> > --- a/xen/include/public/hvm/params.h
> > +++ b/xen/include/public/hvm/params.h
> > @@ -81,9 +81,13 @@
> >  #define HVM_PARAM_PAE_ENABLED  4
> >
> >  #define HVM_PARAM_IOREQ_PFN5
> > -
> >  #define HVM_PARAM_BUFIOREQ_PFN 6
> > +
> > +#ifdef __XEN__
> > +/* These parameters is deprecated and its meaning is undefined */
> > +#define HVM_PARAM_DM_DOMAIN 13
> >  #define HVM_PARAM_BUFIOREQ_EVTCHN 26
> > +#endif
> 
> The comment was only partly switched to plural.
> 

Oh, so it was. I guess this can be fixed on commit unless you'd prefer I send a 
v4.

  Paul

> Jan
> 


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-unstable-smoke test] 125798: trouble: blocked/broken/fail/pass

2018-08-08 Thread osstest service owner
flight 125798 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/125798/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemuu-debianhvm-i386 16 guest-localmigrate/x10 fail pass 
in 125796

Regressions which are regarded as allowable (not blocking):
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 125729

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64-xsm   3 capture-logs  broken blocked in 125729
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  e752f28409678ce3ade49986b39309178fb2a6d6
baseline version:
 xen  ed5f8d9ca47e69e30221c37ec812f2b38af19d83

Last test of basis   125729  2018-08-01 11:00:39 Z6 days
Failing since125741  2018-08-02 10:01:09 Z6 days   16 attempts
Testing same since   125772  2018-08-06 16:00:37 Z1 days   14 attempts


People who touched revisions under test:
  Alexandru Isaila 
  Andrew Cooper 
  Doug Goldstein 
  George Dunlap 
  Jan Beulich 
  Julien Grall 
  Kevin Tian 
  Razvan Cojocaru 
  Roger Pau Monné 
  Stefano Stabellini 

jobs:
 build-arm64-xsm  broken  
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386 fail
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64-xsm broken
broken-step build-arm64-xsm hosts-allocate
broken-step build-arm64-xsm capture-logs
broken-job build-arm64-xsm broken

Not pushing.

(No revision log; it would be 480 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v21 1/2] common: add a new mappable resource type: XENMEM_resource_grant_table

2018-08-08 Thread Paul Durrant
> -Original Message-
> From: Jan Beulich [mailto:jbeul...@suse.com]
> Sent: 08 August 2018 11:30
> To: Paul Durrant 
> Cc: Andrew Cooper ; Wei Liu
> ; George Dunlap ; Ian
> Jackson ; Stefano Stabellini
> ; xen-devel ;
> Konrad Rzeszutek Wilk ; Tim (Xen.org)
> 
> Subject: Re: [PATCH v21 1/2] common: add a new mappable resource type:
> XENMEM_resource_grant_table
> 
> >>> On 08.08.18 at 11:00,  wrote:
> > --- a/xen/common/grant_table.c
> > +++ b/xen/common/grant_table.c
> > @@ -358,6 +358,12 @@ static inline unsigned int
> grant_to_status_frames(unsigned int grant_frames)
> >  return DIV_ROUND_UP(grant_frames * GRANT_PER_PAGE,
> GRANT_STATUS_PER_PAGE);
> >  }
> >
> > +/* Number of grant table entries. Caller must hold d's gr. table lock.*/
> > +static inline unsigned int status_to_grant_frames(unsigned int
> status_frames)
> > +{
> > +return DIV_ROUND_UP(status_frames * GRANT_STATUS_PER_PAGE,
> GRANT_PER_PAGE);
> > +}
> 
> Seeing no use of the grant table (it's not even passed in) I'm confused
> by the comment you add. I guess you've simply copied
> grant_to_status_frames()'es, which looks similarly wrong.
> 

Indeed.

> > @@ -3860,6 +3866,47 @@ int mem_sharing_gref_to_gfn(struct
> grant_table *gt, grant_ref_t ref,
> >  }
> >  #endif
> >
> > +/* caller must hold write lock */
> > +static int gnttab_get_status_frame_mfn(struct domain *d,
> > +   unsigned long idx, mfn_t *mfn)
> > +{
> > +struct grant_table *gt = d->grant_table;
> 
> const?
> 

IIRC that didn't work because gnttab_grow_table() modifies the content.

> > +ASSERT(gt->gt_version == 2);
> > +
> > +if ( idx >= nr_status_frames(gt) )
> > +{
> > +unsigned long nr = status_to_grant_frames(idx + 1);
> > +
> > +if ( nr <= gt->max_grant_frames )
> > +gnttab_grow_table(d, nr);
> > +
> > +if ( idx >= nr_status_frames(gt) )
> > +return -EINVAL;
> > +}
> > +
> > +*mfn = _mfn(virt_to_mfn(gt->status[idx]));
> > +return 0;
> > +}
> > +
> > +/* caller must hold write lock */
> > +static int gnttab_get_shared_frame_mfn(struct domain *d,
> > +   unsigned long idx, mfn_t *mfn)
> > +{
> > +struct grant_table *gt = d->grant_table;
> 
> Again?
> 
> > @@ -3906,6 +3943,38 @@ int gnttab_map_frame(struct domain *d,
> unsigned long idx, gfn_t gfn,
> >  return rc;
> >  }
> >
> > +int gnttab_get_shared_frame(struct domain *d, unsigned long idx,
> > +mfn_t *mfn)
> > +{
> > +struct grant_table *gt = d->grant_table;
> > +int rc;
> > +
> > +grant_write_lock(gt);
> > +
> > +if ( gt->gt_version == 0 )
> > +gt->gt_version = 1;
> 
> Since you've moved this here instead of dropping it, what requirement
> have you found for this to be set (other than the ASSERT() you put in
> gnttab_get_shared_frame_mfn()?
> 

The code in patch #2 is executed before the grant table version is set. I could 
alternatively have libxl explicitly set the version to 1 before trying to seed 
the table.

> > @@ -1046,6 +1089,16 @@ static int acquire_resource(
> >  xen_pfn_t gfn_list[ARRAY_SIZE(mfn_list)];
> >  unsigned int i;
> >
> > +/*
> > + * FIXME: until foreign pages inserted into the P2M are properly
> > + *reference counted, it is unsafe to allow mapping of
> > + *non-caller-owned resource pages unless the caller is
> > + *the hardware domain.
> > + */
> > +if (!(xmar.flags & XENMEM_rsrc_acq_caller_owned) &&
> > +!is_hardware_domain(currd) )
> 
> Missing blank and as a result improper indentation. Also the U in
> "until" wants to be upper case I think.
> 

Ok.

> The cosmetic aspects could of course be taken care of while
> committing, but the version related question needs to be
> clarified first.
> 

It's ok, I'm happy to send v22. Just need to know whether you'd prefer explicit 
version setting in the toolstack.

  Paul

> Jan
> 


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v21 1/2] common: add a new mappable resource type: XENMEM_resource_grant_table

2018-08-08 Thread Jan Beulich
>>> On 08.08.18 at 11:00,  wrote:
> --- a/xen/common/grant_table.c
> +++ b/xen/common/grant_table.c
> @@ -358,6 +358,12 @@ static inline unsigned int 
> grant_to_status_frames(unsigned int grant_frames)
>  return DIV_ROUND_UP(grant_frames * GRANT_PER_PAGE, 
> GRANT_STATUS_PER_PAGE);
>  }
>  
> +/* Number of grant table entries. Caller must hold d's gr. table lock.*/
> +static inline unsigned int status_to_grant_frames(unsigned int status_frames)
> +{
> +return DIV_ROUND_UP(status_frames * GRANT_STATUS_PER_PAGE, 
> GRANT_PER_PAGE);
> +}

Seeing no use of the grant table (it's not even passed in) I'm confused
by the comment you add. I guess you've simply copied
grant_to_status_frames()'es, which looks similarly wrong.

> @@ -3860,6 +3866,47 @@ int mem_sharing_gref_to_gfn(struct grant_table *gt, 
> grant_ref_t ref,
>  }
>  #endif
>  
> +/* caller must hold write lock */
> +static int gnttab_get_status_frame_mfn(struct domain *d,
> +   unsigned long idx, mfn_t *mfn)
> +{
> +struct grant_table *gt = d->grant_table;

const?

> +ASSERT(gt->gt_version == 2);
> +
> +if ( idx >= nr_status_frames(gt) )
> +{
> +unsigned long nr = status_to_grant_frames(idx + 1);
> +
> +if ( nr <= gt->max_grant_frames )
> +gnttab_grow_table(d, nr);
> +
> +if ( idx >= nr_status_frames(gt) )
> +return -EINVAL;
> +}
> +
> +*mfn = _mfn(virt_to_mfn(gt->status[idx]));
> +return 0;
> +}
> +
> +/* caller must hold write lock */
> +static int gnttab_get_shared_frame_mfn(struct domain *d,
> +   unsigned long idx, mfn_t *mfn)
> +{
> +struct grant_table *gt = d->grant_table;

Again?

> @@ -3906,6 +3943,38 @@ int gnttab_map_frame(struct domain *d, unsigned long 
> idx, gfn_t gfn,
>  return rc;
>  }
>  
> +int gnttab_get_shared_frame(struct domain *d, unsigned long idx,
> +mfn_t *mfn)
> +{
> +struct grant_table *gt = d->grant_table;
> +int rc;
> +
> +grant_write_lock(gt);
> +
> +if ( gt->gt_version == 0 )
> +gt->gt_version = 1;

Since you've moved this here instead of dropping it, what requirement
have you found for this to be set (other than the ASSERT() you put in
gnttab_get_shared_frame_mfn()?

> @@ -1046,6 +1089,16 @@ static int acquire_resource(
>  xen_pfn_t gfn_list[ARRAY_SIZE(mfn_list)];
>  unsigned int i;
>  
> +/*
> + * FIXME: until foreign pages inserted into the P2M are properly
> + *reference counted, it is unsafe to allow mapping of
> + *non-caller-owned resource pages unless the caller is
> + *the hardware domain.
> + */
> +if (!(xmar.flags & XENMEM_rsrc_acq_caller_owned) &&
> +!is_hardware_domain(currd) )

Missing blank and as a result improper indentation. Also the U in
"until" wants to be upper case I think.

The cosmetic aspects could of course be taken care of while
committing, but the version related question needs to be
clarified first.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v21 1/2] common: add a new mappable resource type: XENMEM_resource_grant_table

2018-08-08 Thread Paul Durrant
> -Original Message-
> From: Andrew Cooper
> Sent: 08 August 2018 11:11
> To: Paul Durrant ; xen-devel@lists.xenproject.org
> Cc: Jan Beulich ; George Dunlap
> ; Ian Jackson ; Konrad
> Rzeszutek Wilk ; Stefano Stabellini
> ; Tim (Xen.org) ; Wei Liu
> 
> Subject: Re: [PATCH v21 1/2] common: add a new mappable resource type:
> XENMEM_resource_grant_table
> 
> On 08/08/18 10:00, Paul Durrant wrote:
> > diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
> > index d9ec711c73..8e623ea08e 100644
> > --- a/xen/common/grant_table.c
> > +++ b/xen/common/grant_table.c
> > @@ -358,6 +358,12 @@ static inline unsigned int
> grant_to_status_frames(unsigned int grant_frames)
> >  return DIV_ROUND_UP(grant_frames * GRANT_PER_PAGE,
> GRANT_STATUS_PER_PAGE);
> >  }
> >
> > +/* Number of grant table entries. Caller must hold d's gr. table lock.*/
> 
> Really? this is some straight arithmetic on the integer parameter.
> 

Good point. I'd just adapted the comment from the reverse translation above it 
but the comment is indeed bogus.

> > +static inline unsigned int status_to_grant_frames(unsigned int
> status_frames)
> > +{
> > +return DIV_ROUND_UP(status_frames * GRANT_STATUS_PER_PAGE,
> GRANT_PER_PAGE);
> > +}
> > +
> >  /* Check if the page has been paged out, or needs unsharing.
> > If rc == GNTST_okay, *page contains the page struct with a ref taken.
> > Caller must do put_page(*page).
> > @@ -3860,6 +3866,47 @@ int mem_sharing_gref_to_gfn(struct
> grant_table *gt, grant_ref_t ref,
> >  }
> >  #endif
> >
> > +/* caller must hold write lock */
> > +static int gnttab_get_status_frame_mfn(struct domain *d,
> > +   unsigned long idx, mfn_t *mfn)
> > +{
> > +struct grant_table *gt = d->grant_table;
> > +
> > +ASSERT(gt->gt_version == 2);
> > +
> > +if ( idx >= nr_status_frames(gt) )
> > +{
> > +unsigned long nr = status_to_grant_frames(idx + 1);
> 
> Why the +1 ? Won't that cause a failure if you attempt to map the
> maximum valid index?

That's kind of the idea...

> 
> > +
> > +if ( nr <= gt->max_grant_frames )
> > +gnttab_grow_table(d, nr);
> 
> You want to capture the return value of grow_table(), which at least
> distinguishes between ENODEV and ENOMEM.
> 
> > +
> > +if ( idx >= nr_status_frames(gt) )
> > +return -EINVAL;
> 
> This can probably(?) be asserted if gnttab_grow_table() returns
> successfully.
> 

... which is why this is an if and not an ASSERT. I'm just following the 
analogy of the way the table is grown in gnttab_get_shared_frame_mfn() which is 
the way it was done before. If you'd rather I change things to use the return 
value from gnttab_grow_table() then I guess I could do that.

> > +}
> > +
> > +*mfn = _mfn(virt_to_mfn(gt->status[idx]));
> > +return 0;
> > +}
> > +
> > +/* caller must hold write lock */
> > +static int gnttab_get_shared_frame_mfn(struct domain *d,
> > +   unsigned long idx, mfn_t *mfn)
> > +{
> > +struct grant_table *gt = d->grant_table;
> > +
> > +ASSERT(gt->gt_version != 0);
> > +
> > +if ( (idx >= nr_grant_frames(gt)) && (idx < gt->max_grant_frames) )
> > +gnttab_grow_table(d, idx + 1);
> 
> Similarly wrt rc.
> 
> > +
> > +if ( idx >= nr_grant_frames(gt) )
> > +return -EINVAL;
> > +
> > +*mfn = _mfn(virt_to_mfn(gt->shared_raw[idx]));
> > +return 0;
> > +}
> > +
> >  int gnttab_map_frame(struct domain *d, unsigned long idx, gfn_t gfn,
> >   mfn_t *mfn)
> >  {
> > diff --git a/xen/common/memory.c b/xen/common/memory.c
> > index e29d596727..427f65a5e1 100644
> > --- a/xen/common/memory.c
> > +++ b/xen/common/memory.c
> > @@ -23,6 +23,7 @@
> >  #include 
> >  #include 
> >  #include 
> > +#include 
> >  #include 
> >  #include 
> >  #include 
> > @@ -982,6 +983,43 @@ static long xatp_permission_check(struct domain
> *d, unsigned int space)
> >  return xsm_add_to_physmap(XSM_TARGET, current->domain, d);
> >  }
> >
> > +static int acquire_grant_table(struct domain *d, unsigned int id,
> > +   unsigned long frame,
> > +   unsigned int nr_frames,
> > +   xen_pfn_t mfn_list[])
> > +{
> > +unsigned int i = nr_frames;
> > +
> > +/* Iterate backwards in case table needs to grow */
> > +while ( i-- != 0 )
> > +{
> > +mfn_t mfn = INVALID_MFN;
> > +int rc;
> > +
> > +switch ( id )
> > +{
> > +case XENMEM_resource_grant_table_id_shared:
> > +rc = gnttab_get_shared_frame(d, frame + i, );
> > +break;
> > +
> > +case XENMEM_resource_grant_table_id_status:
> > +rc = gnttab_get_status_frame(d, frame + i, );
> > +break;
> > +
> > +default:
> > +rc = -EINVAL;
> > +break;
> > +}
> > +
> > +if ( rc )
> 
> Would it 

Re: [Xen-devel] PVH dom0 creation fails - the system freezes

2018-08-08 Thread bercarug

On 08/08/2018 01:11 PM, Roger Pau Monné wrote:

On Wed, Aug 08, 2018 at 11:44:28AM +0200, Roger Pau Monné wrote:

I just realized that I've dropped a chunk from my series while
rebasing, could you please try again with the following diff applied
on top of my series?

diff --git a/xen/drivers/passthrough/x86/iommu.c 
b/xen/drivers/passthrough/x86/iommu.c
index 6aec43ed1a..6271d8b671 100644
--- a/xen/drivers/passthrough/x86/iommu.c
+++ b/xen/drivers/passthrough/x86/iommu.c
@@ -209,7 +209,13 @@ void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
  if ( !hwdom_iommu_map(d, pfn, max_pfn) )
  continue;
  
-rc = iommu_map_page(d, pfn, pfn, IOMMUF_readable|IOMMUF_writable);

+if ( iommu_use_hap_pt(d) )
+{
+ASSERT(is_hvm_domain(d));
+rc = set_identity_p2m_entry(d, pfn, p2m_access_rw, 0);
+}
+else
+rc = iommu_map_page(d, pfn, pfn, IOMMUF_readable|IOMMUF_writable);
  if ( rc )
  printk(XENLOG_WARNING " d%d: IOMMU mapping failed: %d\n",
 d->domain_id, rc);

I've pushed a new version that has this chunk, so it might be easier
for you to just fetch and test:

git://xenbits.xen.org/people/royger/xen.git iommu_inclusive_v4

Thanks, Roger.

I already recompiled Xen using this patch, and the USB devices are 
functional again.



Thanks,

Gabriel




Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar 
Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in 
Romania. Registration number J22/2621/2005.
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] PVH dom0 creation fails - the system freezes

2018-08-08 Thread Roger Pau Monné
On Wed, Aug 08, 2018 at 11:44:28AM +0200, Roger Pau Monné wrote:
> I just realized that I've dropped a chunk from my series while
> rebasing, could you please try again with the following diff applied
> on top of my series?
> 
> diff --git a/xen/drivers/passthrough/x86/iommu.c 
> b/xen/drivers/passthrough/x86/iommu.c
> index 6aec43ed1a..6271d8b671 100644
> --- a/xen/drivers/passthrough/x86/iommu.c
> +++ b/xen/drivers/passthrough/x86/iommu.c
> @@ -209,7 +209,13 @@ void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
>  if ( !hwdom_iommu_map(d, pfn, max_pfn) )
>  continue;
>  
> -rc = iommu_map_page(d, pfn, pfn, IOMMUF_readable|IOMMUF_writable);
> +if ( iommu_use_hap_pt(d) )
> +{
> +ASSERT(is_hvm_domain(d));
> +rc = set_identity_p2m_entry(d, pfn, p2m_access_rw, 0);
> +}
> +else
> +rc = iommu_map_page(d, pfn, pfn, 
> IOMMUF_readable|IOMMUF_writable);
>  if ( rc )
>  printk(XENLOG_WARNING " d%d: IOMMU mapping failed: %d\n",
> d->domain_id, rc);

I've pushed a new version that has this chunk, so it might be easier
for you to just fetch and test:

git://xenbits.xen.org/people/royger/xen.git iommu_inclusive_v4

Thanks, Roger.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3] x86/hvm: remove default ioreq server

2018-08-08 Thread Jan Beulich
>>> On 07.08.18 at 17:42,  wrote:
> My recent patch [1] to qemu-xen-traditional removes the last use of the
> 'default' ioreq server in Xen. (This is a catch-all ioreq server that is
> used if no explicitly registered I/O range is targetted).
> 
> This patch can be applied once that patch is committed, to remove the
> (>100 lines of) redundant code in Xen.
> 
> NOTE: The removal of the special case for HVM_PARAM_DM_DOMAIN in
>   hvm_allow_set_param() is not directly related to removal of
>   default ioreq servers. It could have been cleaned up at any time
>   after commit 9a422c03 "x86/hvm: stop passing explicit domid to
>   hvm_create_ioreq_server()". It is now added to the new
>   deprecated sets introduced by this patch.
> 
> [1] 
> https://lists.xenproject.org/archives/html/xen-devel/2018-08/msg00270.html 
> 
> Signed-off-by: Paul Durrant 
> Acked-by: Andrew Cooper 

Reviewed-by: Jan Beulich 
with one more cosmetic adjustment:

> --- a/xen/include/public/hvm/params.h
> +++ b/xen/include/public/hvm/params.h
> @@ -81,9 +81,13 @@
>  #define HVM_PARAM_PAE_ENABLED  4
>  
>  #define HVM_PARAM_IOREQ_PFN5
> -
>  #define HVM_PARAM_BUFIOREQ_PFN 6
> +
> +#ifdef __XEN__
> +/* These parameters is deprecated and its meaning is undefined */
> +#define HVM_PARAM_DM_DOMAIN 13
>  #define HVM_PARAM_BUFIOREQ_EVTCHN 26
> +#endif

The comment was only partly switched to plural.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v21 1/2] common: add a new mappable resource type: XENMEM_resource_grant_table

2018-08-08 Thread Andrew Cooper
On 08/08/18 10:00, Paul Durrant wrote:
> diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
> index d9ec711c73..8e623ea08e 100644
> --- a/xen/common/grant_table.c
> +++ b/xen/common/grant_table.c
> @@ -358,6 +358,12 @@ static inline unsigned int 
> grant_to_status_frames(unsigned int grant_frames)
>  return DIV_ROUND_UP(grant_frames * GRANT_PER_PAGE, 
> GRANT_STATUS_PER_PAGE);
>  }
>  
> +/* Number of grant table entries. Caller must hold d's gr. table lock.*/

Really? this is some straight arithmetic on the integer parameter.

> +static inline unsigned int status_to_grant_frames(unsigned int status_frames)
> +{
> +return DIV_ROUND_UP(status_frames * GRANT_STATUS_PER_PAGE, 
> GRANT_PER_PAGE);
> +}
> +
>  /* Check if the page has been paged out, or needs unsharing.
> If rc == GNTST_okay, *page contains the page struct with a ref taken.
> Caller must do put_page(*page).
> @@ -3860,6 +3866,47 @@ int mem_sharing_gref_to_gfn(struct grant_table *gt, 
> grant_ref_t ref,
>  }
>  #endif
>  
> +/* caller must hold write lock */
> +static int gnttab_get_status_frame_mfn(struct domain *d,
> +   unsigned long idx, mfn_t *mfn)
> +{
> +struct grant_table *gt = d->grant_table;
> +
> +ASSERT(gt->gt_version == 2);
> +
> +if ( idx >= nr_status_frames(gt) )
> +{
> +unsigned long nr = status_to_grant_frames(idx + 1);

Why the +1 ? Won't that cause a failure if you attempt to map the
maximum valid index?

> +
> +if ( nr <= gt->max_grant_frames )
> +gnttab_grow_table(d, nr);

You want to capture the return value of grow_table(), which at least
distinguishes between ENODEV and ENOMEM.

> +
> +if ( idx >= nr_status_frames(gt) )
> +return -EINVAL;

This can probably(?) be asserted if gnttab_grow_table() returns
successfully.

> +}
> +
> +*mfn = _mfn(virt_to_mfn(gt->status[idx]));
> +return 0;
> +}
> +
> +/* caller must hold write lock */
> +static int gnttab_get_shared_frame_mfn(struct domain *d,
> +   unsigned long idx, mfn_t *mfn)
> +{
> +struct grant_table *gt = d->grant_table;
> +
> +ASSERT(gt->gt_version != 0);
> +
> +if ( (idx >= nr_grant_frames(gt)) && (idx < gt->max_grant_frames) )
> +gnttab_grow_table(d, idx + 1);

Similarly wrt rc.

> +
> +if ( idx >= nr_grant_frames(gt) )
> +return -EINVAL;
> +
> +*mfn = _mfn(virt_to_mfn(gt->shared_raw[idx]));
> +return 0;
> +}
> +
>  int gnttab_map_frame(struct domain *d, unsigned long idx, gfn_t gfn,
>   mfn_t *mfn)
>  {
> diff --git a/xen/common/memory.c b/xen/common/memory.c
> index e29d596727..427f65a5e1 100644
> --- a/xen/common/memory.c
> +++ b/xen/common/memory.c
> @@ -23,6 +23,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>  #include 
>  #include 
>  #include 
> @@ -982,6 +983,43 @@ static long xatp_permission_check(struct domain *d, 
> unsigned int space)
>  return xsm_add_to_physmap(XSM_TARGET, current->domain, d);
>  }
>  
> +static int acquire_grant_table(struct domain *d, unsigned int id,
> +   unsigned long frame,
> +   unsigned int nr_frames,
> +   xen_pfn_t mfn_list[])
> +{
> +unsigned int i = nr_frames;
> +
> +/* Iterate backwards in case table needs to grow */
> +while ( i-- != 0 )
> +{
> +mfn_t mfn = INVALID_MFN;
> +int rc;
> +
> +switch ( id )
> +{
> +case XENMEM_resource_grant_table_id_shared:
> +rc = gnttab_get_shared_frame(d, frame + i, );
> +break;
> +
> +case XENMEM_resource_grant_table_id_status:
> +rc = gnttab_get_status_frame(d, frame + i, );
> +break;
> +
> +default:
> +rc = -EINVAL;
> +break;
> +}
> +
> +if ( rc )

Would it perhaps be prudent to have || mfn_eq(mfn, INVALID_MFN) here?  I
don't think anything good will come of handing INVALID_MFN back to the
caller.

> +return rc;
> +
> +mfn_list[i] = mfn_x(mfn);
> +}
> +
> +return 0;
> +}
> +
>  static int acquire_resource(
>  XEN_GUEST_HANDLE_PARAM(xen_mem_acquire_resource_t) arg)
>  {
> @@ -1046,6 +1089,16 @@ static int acquire_resource(
>  xen_pfn_t gfn_list[ARRAY_SIZE(mfn_list)];
>  unsigned int i;
>  
> +/*
> + * FIXME: until foreign pages inserted into the P2M are properly
> + *reference counted, it is unsafe to allow mapping of
> + *non-caller-owned resource pages unless the caller is
> + *the hardware domain.
> + */
> +if (!(xmar.flags & XENMEM_rsrc_acq_caller_owned) &&

Space.

> +!is_hardware_domain(currd) )
> +return -EOPNOTSUPP;

EPERM perhaps?

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org

[Xen-devel] [PATCH v4 4/4] x86/iommu: add reserved dom0-iommu option to map reserved memory ranges

2018-08-08 Thread Roger Pau Monne
Several people have reported hardware issues (malfunctioning USB
controllers) due to iommu page faults on Intel hardware. Those faults
are caused by missing RMRR (VTd) entries in the ACPI tables. Those can
be worked around on VTd hardware by manually adding RMRR entries on
the command line, this is however limited to Intel hardware and quite
cumbersome to do.

In order to solve those issues add a new dom0-iommu=reserved option
that identity maps all regions marked as reserved in the memory map.
Note that regions used by devices emulated by Xen (LAPIC, IO-APIC or
PCIe MCFG regions) are specifically avoided. Note that this option is
available to a PVH Dom0 (as opposed to the inclusive option which only
works for PV Dom0).

Signed-off-by: Roger Pau Monné 
Reviewed-by: Paul Durrant 
---
Changes since v3:
 - Add mappings if the iommu page tables are shared.

Changes since v2:
 - Fix comment regarding dom0-strict.
 - Change documentation style of xen command line.
 - Rename iommu_map to hwdom_iommu_map.
 - Move all the checks to hwdom_iommu_map.

Changes since v1:
 - Introduce a new reserved option instead of abusing the inclusive
   option.
 - Use the same helper function for PV and PVH in order to decide if a
   page should be added to the domain page tables.
 - Use the data inside of the domain struct to detect overlaps with
   emulated MMIO regions.
---
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Jan Beulich 
Cc: Julien Grall 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: Wei Liu 
---
 docs/misc/xen-command-line.markdown | 11 ++-
 xen/arch/x86/hvm/io.c   |  5 ++
 xen/drivers/passthrough/amd/pci_amd_iommu.c |  3 +
 xen/drivers/passthrough/iommu.c |  3 +
 xen/drivers/passthrough/vtd/iommu.c |  3 +
 xen/drivers/passthrough/x86/iommu.c | 92 ++---
 xen/include/asm-x86/hvm/io.h|  3 +
 xen/include/xen/iommu.h |  2 +-
 8 files changed, 91 insertions(+), 31 deletions(-)

diff --git a/docs/misc/xen-command-line.markdown 
b/docs/misc/xen-command-line.markdown
index 90b32fe3f0..59ec2afc5d 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -1205,7 +1205,7 @@ detection of systems known to misbehave upon accesses to 
that port.
 >> Enable IOMMU debugging code (implies `verbose`).
 
 ### dom0-iommu
-> `= List of [ none | strict | relaxed | inclusive ]`
+> `= List of [ none | strict | relaxed | inclusive | reserved ]`
 
 * `none`: disables DMA remapping for Dom0.
 
@@ -1233,6 +1233,15 @@ meaning:
   option is only applicable to a PV Dom0 and is enabled by default on Intel
   hardware.
 
+* `reserved`: sets up DMA remapping for all the reserved regions in the memory
+  map for Dom0. Use this to work around firmware issues providing incorrect
+  RMRR/IVMD entries. Rather than only mapping RAM pages for IOMMU accesses
+  for Dom0, all memory regions marked as reserved in the memory map that don't
+  overlap with any MMIO region from emulated devices will be identity mapped.
+  This option maps a subset of the memory that would be mapped when using the
+  `inclusive` option. This option is available to a PVH Dom0 and is enabled by
+  default on Intel hardware.
+
 ### iommu\_dev\_iotlb\_timeout
 > `= `
 
diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c
index bf4d8748d3..5e01c33890 100644
--- a/xen/arch/x86/hvm/io.c
+++ b/xen/arch/x86/hvm/io.c
@@ -404,6 +404,11 @@ static const struct hvm_mmcfg *vpci_mmcfg_find(const 
struct domain *d,
 return NULL;
 }
 
+bool vpci_mmcfg_address(const struct domain *d, paddr_t addr)
+{
+return vpci_mmcfg_find(d, addr);
+}
+
 static unsigned int vpci_mmcfg_decode_addr(const struct hvm_mmcfg *mmcfg,
paddr_t addr, pci_sbdf_t *sbdf)
 {
diff --git a/xen/drivers/passthrough/amd/pci_amd_iommu.c 
b/xen/drivers/passthrough/amd/pci_amd_iommu.c
index 0e0c99c942..2c2867d088 100644
--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
@@ -256,6 +256,9 @@ static void __hwdom_init amd_iommu_hwdom_init(struct domain 
*d)
 /* Inclusive IOMMU mappings are disabled by default on AMD hardware. */
 iommu_dom0_inclusive = iommu_dom0_inclusive == -1 ? false
   : iommu_dom0_inclusive;
+/* Reserved IOMMU mappings are disabled by default on AMD hardware. */
+iommu_dom0_reserved = iommu_dom0_reserved == -1 ? false
+: iommu_dom0_reserved;
 
 if ( allocate_domain_resources(dom_iommu(d)) )
 BUG();
diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
index f15c94be42..9c991bd2cf 100644
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -75,6 +75,7 @@ custom_param("dom0-iommu", parse_dom0_iommu_param);
 bool __hwdom_initdata iommu_dom0_strict;
 bool 

[Xen-devel] [PATCH v4 2/4] iommu: make iommu_inclusive_mapping a suboption of dom0-iommu

2018-08-08 Thread Roger Pau Monne
Introduce a new dom0-iommu=inclusive generic option that supersedes
iommu_inclusive_mapping. The previous behaviour is preserved and the
option should only be enabled by default on Intel hardware.

No functional change intended.

Signed-off-by: Roger Pau Monné 
Reviewed-by: Paul Durrant 
---
Changes since v2:
 - Fix typo in commit message.
 - Change style and text of the documentation in xen command line.
 - Set the defaults in {intel/amd}_iommu_hwdom_init for inclusive.
 - Re-add the iommu_dom0_passthrough || !is_pv_domain(d) check.

Changes since v1:
 - Use dom0-iommu instead of the iommu option.
 - Only enable by default on Intel hardware.
---
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Jan Beulich 
Cc: Julien Grall 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: Wei Liu 
Cc: Kevin Tian 
---
 docs/misc/xen-command-line.markdown | 17 +-
 xen/drivers/passthrough/amd/pci_amd_iommu.c |  4 ++
 xen/drivers/passthrough/arm/iommu.c |  4 ++
 xen/drivers/passthrough/iommu.c | 23 ++--
 xen/drivers/passthrough/vtd/extern.h|  2 -
 xen/drivers/passthrough/vtd/iommu.c |  8 ++-
 xen/drivers/passthrough/vtd/x86/vtd.c   | 58 +---
 xen/drivers/passthrough/x86/iommu.c | 59 +
 xen/include/xen/iommu.h |  2 +
 9 files changed, 109 insertions(+), 68 deletions(-)

diff --git a/docs/misc/xen-command-line.markdown 
b/docs/misc/xen-command-line.markdown
index ea451f088e..90b32fe3f0 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -1205,7 +1205,7 @@ detection of systems known to misbehave upon accesses to 
that port.
 >> Enable IOMMU debugging code (implies `verbose`).
 
 ### dom0-iommu
-> `= List of [ none | strict | relaxed ]`
+> `= List of [ none | strict | relaxed | inclusive ]`
 
 * `none`: disables DMA remapping for Dom0.
 
@@ -1221,6 +1221,18 @@ PV Dom0:
 Note that all the above options are mutually exclusive. Specifying more than
 one on the `dom0-iommu` command line will result in undefined behavior.
 
+The following options control whether non-RAM regions are added to the Dom0
+iommu tables. Note that they can be prefixed with `no-` to effect the inverse
+meaning:
+
+* `inclusive`: sets up DMA remapping for all the non-RAM memory below 4GB
+  except for unusable ranges. Use this to work around firmware issues providing
+  incorrect RMRR/IVMD entries. Rather than only mapping RAM pages for IOMMU
+  accesses for Dom0, with this option all pages up to 4GB, not marked as
+  unusable in the E820 table, will get a mapping established. Note that this
+  option is only applicable to a PV Dom0 and is enabled by default on Intel
+  hardware.
+
 ### iommu\_dev\_iotlb\_timeout
 > `= `
 
@@ -1233,6 +1245,9 @@ wait descriptor timed out', try increasing this value.
 ### iommu\_inclusive\_mapping (VT-d)
 > `= `
 
+**WARNING: This command line option is deprecated, and superseded by
+_dom0-iommu=inclusive_ - using both options in combination is undefined.**
+
 > Default: `true`
 
 Use this to work around firmware issues providing incorrect RMRR entries.
diff --git a/xen/drivers/passthrough/amd/pci_amd_iommu.c 
b/xen/drivers/passthrough/amd/pci_amd_iommu.c
index eeacf713e4..0e0c99c942 100644
--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
@@ -253,6 +253,10 @@ static void __hwdom_init amd_iommu_hwdom_init(struct 
domain *d)
 unsigned long i; 
 const struct amd_iommu *iommu;
 
+/* Inclusive IOMMU mappings are disabled by default on AMD hardware. */
+iommu_dom0_inclusive = iommu_dom0_inclusive == -1 ? false
+  : iommu_dom0_inclusive;
+
 if ( allocate_domain_resources(dom_iommu(d)) )
 BUG();
 
diff --git a/xen/drivers/passthrough/arm/iommu.c 
b/xen/drivers/passthrough/arm/iommu.c
index 95b1abb972..325997b19f 100644
--- a/xen/drivers/passthrough/arm/iommu.c
+++ b/xen/drivers/passthrough/arm/iommu.c
@@ -73,3 +73,7 @@ int arch_iommu_populate_page_table(struct domain *d)
 /* The IOMMU shares the p2m with the CPU */
 return -ENOSYS;
 }
+
+void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
+{
+}
diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
index 830560bdcf..f15c94be42 100644
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -74,6 +74,7 @@ bool_t __read_mostly amd_iommu_perdev_intremap = 1;
 custom_param("dom0-iommu", parse_dom0_iommu_param);
 bool __hwdom_initdata iommu_dom0_strict;
 bool __read_mostly iommu_dom0_passthrough;
+int8_t __hwdom_initdata iommu_dom0_inclusive = -1;
 
 DEFINE_PER_CPU(bool_t, iommu_dont_flush_iotlb);
 
@@ -144,16 +145,23 @@ static int __init parse_dom0_iommu_param(const char *s)
 int rc = 0;
 
 do {
+bool val = !!strncmp(s, "no-", 3);
+
+if ( !val )
+s += 3;
+
 ss = 

[Xen-devel] [PATCH v4 0/4] 86/iommu: PVH Dom0 workarounds for missing RMRR entries

2018-08-08 Thread Roger Pau Monne
Hello,

The following series implement a workaround for missing RMRR
entries for a PVH Dom0. It's based on the iommu_inclusive_mapping VTd
option.

The PVH workaround identity maps all regions marked as reserved in the
memory map.

Note that this workaround is enabled by default on Intel hardware. It's
also available to AMD hardware, although it's disabled by default in
that case.

The series can be found at:

git://xenbits.xen.org/people/royger/xen.git iommu_inclusive_v4

Thanks.

Roger Pau Monne (4):
  iommu: introduce dom0-iommu option
  iommu: make iommu_inclusive_mapping a suboption of dom0-iommu
  dom0/pvh: change the order of the MMCFG initialization
  x86/iommu: add reserved dom0-iommu option to map reserved memory
ranges

 docs/misc/xen-command-line.markdown | 47 +++
 xen/arch/x86/hvm/dom0_build.c   |  9 +-
 xen/arch/x86/hvm/io.c   |  5 ++
 xen/arch/x86/x86_64/mm.c|  3 +-
 xen/drivers/passthrough/amd/iommu_init.c|  2 +-
 xen/drivers/passthrough/amd/pci_amd_iommu.c | 11 ++-
 xen/drivers/passthrough/arm/iommu.c |  4 +
 xen/drivers/passthrough/iommu.c | 62 --
 xen/drivers/passthrough/vtd/extern.h|  2 -
 xen/drivers/passthrough/vtd/iommu.c | 25 +++---
 xen/drivers/passthrough/vtd/x86/vtd.c   | 58 +
 xen/drivers/passthrough/x86/iommu.c | 93 +
 xen/include/asm-x86/hvm/io.h|  3 +
 xen/include/xen/iommu.h |  8 +-
 14 files changed, 246 insertions(+), 86 deletions(-)

-- 
2.18.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v4 1/4] iommu: introduce dom0-iommu option

2018-08-08 Thread Roger Pau Monne
To select the iommu configuration used by Dom0. This option supersedes
iommu=dom0-strict|dom0-passthrough.

No functional change.

Signed-off-by: Roger Pau Monné 
Reviewed-by: Paul Durrant 
---
Changes since v2:
 - Change the style and text used in the xen command line document.
 - Don't allow none, strict or relaxed to use the no- prefix.

Changes since v1:
 - New in this version.
---
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Jan Beulich 
Cc: Julien Grall 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: Wei Liu 
Cc: Suravee Suthikulpanit 
Cc: Brian Woods 
Cc: Kevin Tian 
---
 docs/misc/xen-command-line.markdown | 23 +++
 xen/arch/x86/x86_64/mm.c|  3 +-
 xen/drivers/passthrough/amd/iommu_init.c|  2 +-
 xen/drivers/passthrough/amd/pci_amd_iommu.c |  4 +-
 xen/drivers/passthrough/iommu.c | 42 +
 xen/drivers/passthrough/vtd/iommu.c | 16 
 xen/include/xen/iommu.h |  6 ++-
 7 files changed, 75 insertions(+), 21 deletions(-)

diff --git a/docs/misc/xen-command-line.markdown 
b/docs/misc/xen-command-line.markdown
index 65b4754418..ea451f088e 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -1150,12 +1150,18 @@ detection of systems known to misbehave upon accesses 
to that port.
 
 > `dom0-passthrough`
 
+> **WARNING: This command line option is deprecated, and superseded by
+> _dom0-iommu=none_ - using both options in combination is undefined.**
+
 > Default: `false`
 
 >> Control whether to disable DMA remapping for Dom0.
 
 > `dom0-strict`
 
+> **WARNING: This command line option is deprecated, and superseded by
+> _dom0-iommu=strict_ - using both options in combination is undefined.**
+
 > Default: `false`
 
 >> Control whether to set up DMA remapping only for the memory Dom0 actually
@@ -1198,6 +1204,23 @@ detection of systems known to misbehave upon accesses to 
that port.
 
 >> Enable IOMMU debugging code (implies `verbose`).
 
+### dom0-iommu
+> `= List of [ none | strict | relaxed ]`
+
+* `none`: disables DMA remapping for Dom0.
+
+The following two options control how RAM regions are mapped in the iommu for
+PV Dom0:
+
+* `strict`: sets up DMA remapping only for the memory Dom0 actually got
+  assigned.
+
+* `relaxed`: sets DMA remapping for all the host RAM except regions in use by
+  Xen. This is the default iommu behaviour.
+
+Note that all the above options are mutually exclusive. Specifying more than
+one on the `dom0-iommu` command line will result in undefined behavior.
+
 ### iommu\_dev\_iotlb\_timeout
 > `= `
 
diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c
index cca4ae926e..84226b3326 100644
--- a/xen/arch/x86/x86_64/mm.c
+++ b/xen/arch/x86/x86_64/mm.c
@@ -1426,7 +1426,8 @@ int memory_add(unsigned long spfn, unsigned long epfn, 
unsigned int pxm)
 if ( ret )
 goto destroy_m2p;
 
-if ( iommu_enabled && !iommu_passthrough && !need_iommu(hardware_domain) )
+if ( iommu_enabled && !iommu_dom0_passthrough &&
+ !need_iommu(hardware_domain) )
 {
 for ( i = spfn; i < epfn; i++ )
 if ( iommu_map_page(hardware_domain, i, i, 
IOMMUF_readable|IOMMUF_writable) )
diff --git a/xen/drivers/passthrough/amd/iommu_init.c 
b/xen/drivers/passthrough/amd/iommu_init.c
index 474992a75a..ad8c48be1c 100644
--- a/xen/drivers/passthrough/amd/iommu_init.c
+++ b/xen/drivers/passthrough/amd/iommu_init.c
@@ -1062,7 +1062,7 @@ static void __init amd_iommu_init_cleanup(void)
 radix_tree_destroy(_maps, xfree);
 
 iommu_enabled = 0;
-iommu_passthrough = 0;
+iommu_dom0_passthrough = false;
 iommu_intremap = 0;
 iommuv2_enabled = 0;
 }
diff --git a/xen/drivers/passthrough/amd/pci_amd_iommu.c 
b/xen/drivers/passthrough/amd/pci_amd_iommu.c
index 12d2695b89..eeacf713e4 100644
--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
@@ -121,7 +121,7 @@ static void amd_iommu_setup_domain_device(
 BUG_ON( !hd->arch.root_table || !hd->arch.paging_mode ||
 !iommu->dev_table.buffer );
 
-if ( iommu_passthrough && is_hardware_domain(domain) )
+if ( iommu_dom0_passthrough && is_hardware_domain(domain) )
 valid = 0;
 
 if ( ats_enabled )
@@ -256,7 +256,7 @@ static void __hwdom_init amd_iommu_hwdom_init(struct domain 
*d)
 if ( allocate_domain_resources(dom_iommu(d)) )
 BUG();
 
-if ( !iommu_passthrough && !need_iommu(d) )
+if ( !iommu_dom0_passthrough && !need_iommu(d) )
 {
 int rc = 0;
 
diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
index 70d218f910..830560bdcf 100644
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -22,6 +22,7 @@
 #include 
 
 static int parse_iommu_param(const char *s);
+static int parse_dom0_iommu_param(const char *s);
 static void iommu_dump_p2m_table(unsigned char 

[Xen-devel] [PATCH v4 3/4] dom0/pvh: change the order of the MMCFG initialization

2018-08-08 Thread Roger Pau Monne
So it's done before the iommu is initialized. This is required in
order to be able to fetch the MMCFG regions from the domain struct.

No functional change.

Signed-off-by: Roger Pau Monné 
---
Changes since v1:
 - New in this version.
---
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 xen/arch/x86/hvm/dom0_build.c | 9 +++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/hvm/dom0_build.c b/xen/arch/x86/hvm/dom0_build.c
index f0cd63b1ec..5065729106 100644
--- a/xen/arch/x86/hvm/dom0_build.c
+++ b/xen/arch/x86/hvm/dom0_build.c
@@ -1100,6 +1100,13 @@ int __init dom0_construct_pvh(struct domain *d, const 
module_t *image,
 return rc;
 }
 
+/*
+ * NB: MMCFG initialization needs to be performed before iommu
+ * initialization so the iommu code can fetch the MMCFG regions used by the
+ * domain.
+ */
+pvh_setup_mmcfg(d);
+
 iommu_hwdom_init(d);
 
 rc = pvh_load_kernel(d, image, image_headroom, initrd, 
bootstrap_map(image),
@@ -1124,8 +1131,6 @@ int __init dom0_construct_pvh(struct domain *d, const 
module_t *image,
 return rc;
 }
 
-pvh_setup_mmcfg(d);
-
 printk("WARNING: PVH is an experimental mode with limited 
functionality\n");
 return 0;
 }
-- 
2.18.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC PATCH 2/2] xen/arm: Add MESON UART driver for Amlogic S905 SoC

2018-08-08 Thread Jan Beulich
>>> On 07.08.18 at 19:07,  wrote:
> This patch adds driver for UART controller present on Amlogic S905 SoC.
> https://dn.odroid.com/S905/DataSheet/S905_Public_Datasheet_V1.1.4.pdf 
> 
> Signed-off-by: Amit Singh Tomar 
> ---
>  xen/drivers/char/Kconfig  |   8 ++
>  xen/drivers/char/Makefile |   1 +
>  xen/drivers/char/meson-uart.c | 290 
> ++
>  3 files changed, 299 insertions(+)
>  create mode 100644 xen/drivers/char/meson-uart.c

The driver being ARM-specific, you will want to update
./MAINTAINERS to also list this new file as ARM-specific.

> --- a/xen/drivers/char/Kconfig
> +++ b/xen/drivers/char/Kconfig
> @@ -20,6 +20,14 @@ config HAS_MVEBU
> This selects the Marvell MVEBU UART. If you have a ARMADA 3700
> based board, say Y.
>  
> +config HAS_MESON
> +bool
> +default y
> +depends on ARM_64
> +help
> +  This selects the Marvell MESON UART. If you have a Amlogic S905
> +  based board, say Y.
> +
>  config HAS_PL011
>   bool
>   default y

Please fix indentation to match that of surrounding code. Also
please use "def_bool y" rather than its longer two line equivalent.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] PVH dom0 creation fails - the system freezes

2018-08-08 Thread Jan Beulich
>>> On 08.08.18 at 10:51,  wrote:
> On Wed, Aug 08, 2018 at 09:43:39AM +0100, Paul Durrant wrote:
>> > -Original Message-
>> > From: Roger Pau Monne
>> > Sent: 08 August 2018 09:08
>> > To: berca...@amazon.com 
>> > Cc: Paul Durrant ; xen-devel > > de...@lists.xenproject.org>; David Woodhouse ;
>> > Jan Beulich ; Belgun, Adrian 
>> > Subject: Re: [Xen-devel] PVH dom0 creation fails - the system freezes
>> > 
>> > On Wed, Aug 08, 2018 at 10:46:40AM +0300, berca...@amazon.com wrote:
>> > > On 08/02/2018 04:55 PM, Roger Pau Monné wrote:
>> > > > Please try to avoid top posting.
>> > > >
>> > > > On Thu, Aug 02, 2018 at 11:36:26AM +, Bercaru, Gabriel wrote:
>> > > > > I applied the match mentioned, but the system fails to boot. 
>> > > > > Instead, it
>> > > > > drops to a BusyBox shell. It seems to be a file system issue.
>> > > > So you have applied 173c7803592065d27bf2e60d50e08e197a0efa83 and it
>> > > > causes a regression for you?
>> > > >
>> > > > As I understand it, before applying 173c780359206 you where capable of
>> > > > booting the PVH Dom0, albeit with non-working USB?
>> > > >
>> > > > And after applying 173c780359206 you are no longer able to boot?
>> > > Right, after applying 173c780359206 the system fails to boot (it drops to
>> > > the BusyBox shell).
>> > > > > Following is a sequence of the boot log regarding the file system 
>> > > > > issue.
>> > > > At least part of the issue seems to be that the IO-APIC information
>> > > > provided to Dom0 is wrong (from the attached log):
>> > > >
>> > > > [0.00] IOAPIC[0]: apic_id 2, version 152, address 0xfec0, 
>> > > > GSI 
> 0-
>> > 0
>> > > > [0.00] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
>> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 2
>> > > > [0.00] Failed to find ioapic for gsi : 2
>> > > > [0.00] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high 
> level)
>> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 9
>> > > > [0.00] Failed to find ioapic for gsi : 9
>> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 1
>> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 2
>> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 3
>> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 4
>> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 5
>> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 6
>> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 7
>> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 8
>> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 9
>> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 10
>> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 11
>> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 12
>> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 13
>> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 14
>> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 15
>> > > >
>> > > > Can you try to boot with just the staging branch (current commit is
>> > > > 008a8fb249b9d433) and see how that goes?
>> > > >
>> > > > Thanks, Roger.
>> > > >
>> > > I recompiled Xen using the staging branch, commit 008a8fb249b9d433 and
>> > the
>> > > system boots,
>> > 
>> > OK, so your issues where not caused by 173c780359206 then?
>> > 
>> > 008a8fb249b9d433 already contains 173c780359206 because it was
>> > committed earlier. In any case it's good to know you are able to boot
>> > (albeit with issues) using the current staging branch.
>> > 
>> > > however the USB problem persists. I was able to log in using the serial 
> port
>> > > and after executing
>> > 
>> > Yes, the fixes for this issue have not been committed yet, see:
>> > 
>> > https://lists.xenproject.org/archives/html/xen-devel/2018- 
>> > 08/msg00528.html
>> > 
>> > If you want you can give this branch a try, it should hopefully solve
>> > your USB issues.
>> > 
>> > > "xl list -l" the memory decrease problem appeared.
>> > 
>> > Yup, I will look into this now in order to find some kind of
>> > workaround.
>> > 
>> > > I attached the boot log. Following are some lines extracted from the log,
>> > > regarding the USB
>> > >
>> > > devices problem:
>> > >
>> > > [5.864084] usb 1-1: device descriptor read/64, error -71
>> > >
>> > > [7.564025] usb 1-1: new full-speed USB device number 4 using xhci_hcd
>> > > [7.571347] usb 1-1: Device not responding to setup address.
>> > >
>> > > [8.008031] usb 1-1: device not accepting address 4, error -71
>> > >
>> > > [8.609623] usb 1-1: device not accepting address 5, error -71
>> > >
>> > >
>> > > At the beginning of the log, there is a message regarding
>> > > iommu_inclusive_mapping:
>> > >
>> > >
>> > > (XEN) [VT-D]found ACPI_DMAR_RMRR:
>> > > (XEN) [VT-D]  RMRR address range 3e2e..3e2f not in reserved
>> > memory;
>> > > need "iommu_inclusive_mapping=1"?
>> > > (XEN) 

Re: [Xen-devel] [PATCH v21 2/2] tools/libxenctrl: use new xenforeignmemory API to seed grant table

2018-08-08 Thread Andrew Cooper
On 08/08/18 10:00, Paul Durrant wrote:
> A previous patch added support for priv-mapping guest resources directly
> (rather than having to foreign-map, which requires P2M modification for
> HVM guests).
>
> This patch makes use of the new API to seed the guest grant table unless
> the underlying infrastructure (i.e. privcmd) doesn't support it, in which
> case the old scheme is used.
>
> NOTE: The call to xc_dom_gnttab_hvm_seed() in hvm_build_set_params() was
>   actually unnecessary, as the grant table has already been seeded
>   by a prior call to xc_dom_gnttab_init() made by libxl__build_dom().
>
> Signed-off-by: Paul Durrant 
> Acked-by: Marek Marczykowski-Górecki 
> Reviewed-by: Roger Pau Monné 
> Acked-by: Wei Liu 

Some minor style issues, all of which can fixed on commit (probably). 
Everything else looks fine.

> diff --git a/tools/libxc/include/xc_dom.h b/tools/libxc/include/xc_dom.h
> index 8a66889c68..a8a0c0da66 100644
> --- a/tools/libxc/include/xc_dom.h
> +++ b/tools/libxc/include/xc_dom.h
> @@ -337,12 +337,8 @@ void *xc_dom_boot_domU_map(struct xc_dom_image *dom, 
> xen_pfn_t pfn,
>  int xc_dom_boot_image(struct xc_dom_image *dom);
>  int xc_dom_compat_check(struct xc_dom_image *dom);
>  int xc_dom_gnttab_init(struct xc_dom_image *dom);
> -int xc_dom_gnttab_hvm_seed(xc_interface *xch, uint32_t domid,
> -   xen_pfn_t console_gmfn,
> -   xen_pfn_t xenstore_gmfn,
> -   uint32_t console_domid,
> -   uint32_t xenstore_domid);
> -int xc_dom_gnttab_seed(xc_interface *xch, uint32_t domid,
> +int xc_dom_gnttab_seed(xc_interface *xch, uint32_t guest_domid,
> +   bool is_hvm,
> xen_pfn_t console_gmfn,
> xen_pfn_t xenstore_gmfn,

As you're rewriting most of the functionality, can we switch to gfn to
correct the terminology?

> uint32_t console_domid,
> diff --git a/tools/libxc/xc_dom_boot.c b/tools/libxc/xc_dom_boot.c
> index 2e5681dc5d..8307ebeaf6 100644
> --- a/tools/libxc/xc_dom_boot.c
> +++ b/tools/libxc/xc_dom_boot.c
> @@ -256,11 +256,29 @@ static xen_pfn_t xc_dom_gnttab_setup(xc_interface *xch, 
> uint32_t domid)
>  return gmfn;
>  }
>  
> -int xc_dom_gnttab_seed(xc_interface *xch, uint32_t domid,
> -   xen_pfn_t console_gmfn,
> -   xen_pfn_t xenstore_gmfn,
> -   uint32_t console_domid,
> -   uint32_t xenstore_domid)
> +static void xc_dom_set_gnttab_entry(xc_interface *xch,
> +grant_entry_v1_t *gnttab,
> +unsigned int idx,
> +uint32_t guest_domid,
> +uint32_t backend_domid,
> +xen_pfn_t backend_gmfn)

gfn

> +{
> +if ( guest_domid == backend_domid || backend_gmfn == -1)

Space at the end.

> +return;
> +
> +xc_dom_printf(xch, "%s: [%u] -> 0x%"PRI_xen_pfn,
> +  __FUNCTION__, idx, backend_gmfn);

__func__ rather than __FUNCTION__.  Also, "[%u] => d%d 0x"PRI_xen_pfn
would be more helpful for diagnostics.  (I do realise that backend
domid's were retrofitted without adjusting the printf().)

> +
> +gnttab[idx].flags = GTF_permit_access;
> +gnttab[idx].domid = backend_domid;
> +gnttab[idx].frame = backend_gmfn;
> +}
> +
> +static int compat_gnttab_seed(xc_interface *xch, uint32_t domid,

compat_gnttab_pv_seed() ?

> +  xen_pfn_t console_gmfn,
> +  xen_pfn_t xenstore_gmfn,

gfn

> +  uint32_t console_domid,
> +  uint32_t xenstore_domid)
>  {
>  
>  xen_pfn_t gnttab_gmfn;
> @@ -313,11 +323,11 @@ int xc_dom_gnttab_seed(xc_interface *xch, uint32_t 
> domid,
>  return 0;
>  }
>  
> -int xc_dom_gnttab_hvm_seed(xc_interface *xch, uint32_t domid,
> -   xen_pfn_t console_gpfn,
> -   xen_pfn_t xenstore_gpfn,
> -   uint32_t console_domid,
> -   uint32_t xenstore_domid)
> +static int compat_gnttab_hvm_seed(xc_interface *xch, uint32_t domid,
> +  xen_pfn_t console_gpfn,
> +  xen_pfn_t xenstore_gpfn,

gfn.

> +  uint32_t console_domid,
> +  uint32_t xenstore_domid)
>  {
>  int rc;
>  xen_pfn_t scratch_gpfn;
> @@ -356,7 +366,7 @@ int xc_dom_gnttab_hvm_seed(xc_interface *xch, uint32_t 
> domid,
>  return -1;
>  }
>  
> -rc = xc_dom_gnttab_seed(xch, domid,
> +rc = compat_gnttab_seed(xch, domid,
>  console_gpfn, xenstore_gpfn,
>  console_domid, xenstore_domid);
>  if (rc != 0)
> @@ -381,18 +391,56 @@ 

Re: [Xen-devel] x86 Community Call - Wed Aug 15, 14:00 - 15:00 UTC - Call for agenda items

2018-08-08 Thread Lars Kurth
> I'd like to ask about Xen's memory scrubbing, to better understand the 
> changes made to it in recent versions of Xen (4.10+) and the community 
> understanding of requirements on it.
That looks good. Will add
Lars

From: Christopher Clark 
Date: Wednesday, 8 August 2018 at 01:28
To: Lars Kurth 
Cc: xen-devel , "committ...@xenproject.org" 
, Tamas K Lengyel , 
"intel-...@intel.com" , "daniel.ki...@oracle.com" 
, Roger Monne , Rich Persaud 
, Brian Woods , Stefano Stabellini 
, Julien Grall , Juergen Gross 
, Paul Durrant , "Ji, John" 
, "Natarajan, Janakarajan" , 
"edgar.igles...@xilinx.com" , 
"davorin.mi...@aggios.com" , "robin.randh...@arm.com" 
, Artem Mygaiev , Matt Spencer 
, "anastassios.na...@onapp.com" 
, Stewart Hildebrand 
, "vfac...@de.adit-jv.com" 
, Volodymyr Babchuk , 
"mirela.simono...@aggios.com" , Jarvis Roach 

Subject: Re: x86 Community Call - Wed Aug 15, 14:00 - 15:00 UTC - Call for 
agenda items

On Tue, Aug 7, 2018 at 2:26 AM, Lars Kurth 
mailto:lars.ku...@citrix.com>> wrote:
Dear community members,

please send me agenda items for next week’s community call by this Friday.

If there is time available on the call, I'd like to ask about Xen's memory 
scrubbing, to better understand the changes made to it in recent versions of 
Xen (4.10+) and the community understanding of requirements on it.

thanks,

Christopher

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 4/4] x86/iommu: add reserved dom0-iommu option to map reserved memory ranges

2018-08-08 Thread Jan Beulich
>>> On 08.08.18 at 11:53,  wrote:
> I've lost a chunk here during one of the rebases, so the following
> diff should be added to the patch in order to create maps if the iommu
> page tables are shared:
> 
> diff --git a/xen/drivers/passthrough/x86/iommu.c 
> b/xen/drivers/passthrough/x86/iommu.c
> index 6aec43ed1a..6271d8b671 100644
> --- a/xen/drivers/passthrough/x86/iommu.c
> +++ b/xen/drivers/passthrough/x86/iommu.c
> @@ -209,7 +209,13 @@ void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
>  if ( !hwdom_iommu_map(d, pfn, max_pfn) )
>  continue;
>  
> -rc = iommu_map_page(d, pfn, pfn, IOMMUF_readable|IOMMUF_writable);
> +if ( iommu_use_hap_pt(d) )
> +{
> +ASSERT(is_hvm_domain(d));
> +rc = set_identity_p2m_entry(d, pfn, p2m_access_rw, 0);
> +}
> +else
> +rc = iommu_map_page(d, pfn, pfn, 
> IOMMUF_readable|IOMMUF_writable);
>  if ( rc )
>  printk(XENLOG_WARNING " d%d: IOMMU mapping failed: %d\n",
> d->domain_id, rc);
> 
> I can resend the series in order to ease review.

This would indeed be nice, thanks.

Jan




___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries

2018-08-08 Thread Jan Beulich
>>> On 08.08.18 at 10:25,  wrote:
> On Tue, Aug 07, 2018 at 10:45:32AM -0600, Tamas K Lengyel wrote:
>> On Tue, Aug 7, 2018 at 10:04 AM Tamas K Lengyel
>>  wrote:
>> (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow
>> (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault
>> (XEN) [VT-D]DMAR:[DMA Read] Request device [:00:02.0] fault addr
>> 428f926000, iommu reg = 82c000a0c000
>> (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
>> (XEN) print_vtd_entries: iommu #0 dev :00:02.0 gmfn 428f926
>> (XEN) root_entry[00] = 43aaae001
>> (XEN) context[10] = 2_43cf92001
>> (XEN) l4[000] = 9c043cf91107
>> (XEN) l3[10a] = 8000
>> (XEN) l3[10a] not present
>> 
>> The fault is repeated a million times per second and the system is
>> pretty much stalled.
> 
> As Jan says, this page is outside of any range in the memory map. I
> wonder however what's in there.
> 
> I think (also seeing the PV issues) you should bring this up with the
> driver maintainers, it might actually be a bug that the driver is
> trying to access such address.

Right, especially considering that the address does not appear to be
invariant, I suspect the driver sets up some I/O from (presumably)
uninitialized data. This goes unnoticed until DMA translation comes
into play. Tamas - did you try enabling DMA translation in Linux
when not running on top of Xen? Depending on the
CONFIG_INTEL_IOMMU_DEFAULT_ON setting this may not be the
default.

> In the meantime, you can try to add to the command line:
> 
> rmrr=0x428f926=0:0:2.0
> 
> In order to force an iommu mapping of this address.

I suspect this won't help much.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 4/4] x86/iommu: add reserved dom0-iommu option to map reserved memory ranges

2018-08-08 Thread Roger Pau Monné
On Tue, Aug 07, 2018 at 04:02:43PM +0200, Roger Pau Monne wrote:
> @@ -149,36 +204,9 @@ void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
>  for ( i = 0; i < top; i++ )
>  {
>  unsigned long pfn = pdx_to_pfn(i);
> -bool map;
>  int rc;
>  
> -/*
> - * Set up 1:1 mapping for dom0. Default to include only
> - * conventional RAM areas and let RMRRs include needed reserved
> - * regions. When set, the inclusive mapping additionally maps in
> - * every pfn up to 4GB except those that fall in unusable ranges.
> - */
> -if ( pfn > max_pfn && !mfn_valid(_mfn(pfn)) )
> -continue;
> -
> -if ( iommu_dom0_inclusive && pfn <= max_pfn )
> -map = !page_is_ram_type(pfn, RAM_TYPE_UNUSABLE);
> -else
> -map = page_is_ram_type(pfn, RAM_TYPE_CONVENTIONAL);
> -
> -if ( !map )
> -continue;
> -
> -/* Exclude Xen bits */
> -if ( xen_in_range(pfn) )
> -continue;
> -
> -/*
> - * If dom0-strict mode is enabled then exclude conventional RAM
> - * and let the common code map dom0's pages.
> - */
> -if ( iommu_dom0_strict &&
> - page_is_ram_type(pfn, RAM_TYPE_CONVENTIONAL) )
> +if ( !hwdom_iommu_map(d, pfn, max_pfn) )
>  continue;
>  
>  rc = iommu_map_page(d, pfn, pfn, IOMMUF_readable|IOMMUF_writable);

I've lost a chunk here during one of the rebases, so the following
diff should be added to the patch in order to create maps if the iommu
page tables are shared:

diff --git a/xen/drivers/passthrough/x86/iommu.c 
b/xen/drivers/passthrough/x86/iommu.c
index 6aec43ed1a..6271d8b671 100644
--- a/xen/drivers/passthrough/x86/iommu.c
+++ b/xen/drivers/passthrough/x86/iommu.c
@@ -209,7 +209,13 @@ void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
 if ( !hwdom_iommu_map(d, pfn, max_pfn) )
 continue;
 
-rc = iommu_map_page(d, pfn, pfn, IOMMUF_readable|IOMMUF_writable);
+if ( iommu_use_hap_pt(d) )
+{
+ASSERT(is_hvm_domain(d));
+rc = set_identity_p2m_entry(d, pfn, p2m_access_rw, 0);
+}
+else
+rc = iommu_map_page(d, pfn, pfn, IOMMUF_readable|IOMMUF_writable);
 if ( rc )
 printk(XENLOG_WARNING " d%d: IOMMU mapping failed: %d\n",
d->domain_id, rc);

I can resend the series in order to ease review.

Roger.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] PVH dom0 creation fails - the system freezes

2018-08-08 Thread Roger Pau Monné
On Wed, Aug 08, 2018 at 11:54:40AM +0300, berca...@amazon.com wrote:
> On 08/08/2018 11:51 AM, Roger Pau Monné wrote:
> > On Wed, Aug 08, 2018 at 09:43:39AM +0100, Paul Durrant wrote:
> > > > -Original Message-
> > > > From: Roger Pau Monne
> > > > Sent: 08 August 2018 09:08
> > > > To: berca...@amazon.com
> > > > Cc: Paul Durrant ; xen-devel  > > > de...@lists.xenproject.org>; David Woodhouse ;
> > > > Jan Beulich ; Belgun, Adrian 
> > > > Subject: Re: [Xen-devel] PVH dom0 creation fails - the system freezes
> > > > 
> > > > On Wed, Aug 08, 2018 at 10:46:40AM +0300, berca...@amazon.com wrote:
> > > > > On 08/02/2018 04:55 PM, Roger Pau Monné wrote:
> > > > > > Please try to avoid top posting.
> > > > > > 
> > > > > > On Thu, Aug 02, 2018 at 11:36:26AM +, Bercaru, Gabriel wrote:
> > > > > > > I applied the match mentioned, but the system fails to boot. 
> > > > > > > Instead, it
> > > > > > > drops to a BusyBox shell. It seems to be a file system issue.
> > > > > > So you have applied 173c7803592065d27bf2e60d50e08e197a0efa83 and it
> > > > > > causes a regression for you?
> > > > > > 
> > > > > > As I understand it, before applying 173c780359206 you where capable 
> > > > > > of
> > > > > > booting the PVH Dom0, albeit with non-working USB?
> > > > > > 
> > > > > > And after applying 173c780359206 you are no longer able to boot?
> > > > > Right, after applying 173c780359206 the system fails to boot (it 
> > > > > drops to
> > > > > the BusyBox shell).
> > > > > > > Following is a sequence of the boot log regarding the file system 
> > > > > > > issue.
> > > > > > At least part of the issue seems to be that the IO-APIC information
> > > > > > provided to Dom0 is wrong (from the attached log):
> > > > > > 
> > > > > > [0.00] IOAPIC[0]: apic_id 2, version 152, address 
> > > > > > 0xfec0, GSI 0-
> > > > 0
> > > > > > [0.00] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl 
> > > > > > dfl)
> > > > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 2
> > > > > > [0.00] Failed to find ioapic for gsi : 2
> > > > > > [0.00] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high 
> > > > > > level)
> > > > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 9
> > > > > > [0.00] Failed to find ioapic for gsi : 9
> > > > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 1
> > > > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 2
> > > > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 3
> > > > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 4
> > > > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 5
> > > > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 6
> > > > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 7
> > > > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 8
> > > > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 9
> > > > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 10
> > > > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 11
> > > > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 12
> > > > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 13
> > > > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 14
> > > > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 15
> > > > > > 
> > > > > > Can you try to boot with just the staging branch (current commit is
> > > > > > 008a8fb249b9d433) and see how that goes?
> > > > > > 
> > > > > > Thanks, Roger.
> > > > > > 
> > > > > I recompiled Xen using the staging branch, commit 008a8fb249b9d433 and
> > > > the
> > > > > system boots,
> > > > OK, so your issues where not caused by 173c780359206 then?
> > > > 
> > > > 008a8fb249b9d433 already contains 173c780359206 because it was
> > > > committed earlier. In any case it's good to know you are able to boot
> > > > (albeit with issues) using the current staging branch.
> > > > 
> > > > > however the USB problem persists. I was able to log in using the 
> > > > > serial port
> > > > > and after executing
> > > > Yes, the fixes for this issue have not been committed yet, see:
> > > > 
> > > > https://lists.xenproject.org/archives/html/xen-devel/2018-
> > > > 08/msg00528.html
> > > > 
> > > > If you want you can give this branch a try, it should hopefully solve
> > > > your USB issues.
> > > > 
> > > > > "xl list -l" the memory decrease problem appeared.
> > > > Yup, I will look into this now in order to find some kind of
> > > > workaround.
> > > > 
> > > > > I attached the boot log. Following are some lines extracted from the 
> > > > > log,
> > > > > regarding the USB
> > > > > 
> > > > > devices problem:
> > > > > 
> > > > > [    5.864084] usb 1-1: device descriptor read/64, error -71
> > > > > 
> > > > > [    7.564025] usb 1-1: new full-speed USB device number 4 using 
> > > > > xhci_hcd
> > > > > [    7.571347] usb 1-1: Device not responding to setup address.
> > > > > 
> > > > > [    8.008031] usb 

Re: [Xen-devel] [PATCH 6/9] x86: move memory_type_changed to mm.c

2018-08-08 Thread Jan Beulich
>>> On 07.08.18 at 18:19,  wrote:
> On Tue, Aug 07, 2018 at 05:33:35AM -0600, Jan Beulich wrote:
>> >>> On 07.08.18 at 12:00,  wrote:
>> > This function is common to both PV and HVM. Move it to x86 common
>> > code.
>> 
>> I'm afraid that's the wrong way round: p2m_memory_type_changed()
>> is HVM (in fact EPT) specific. The only uses of the function that aren't
>> already HVM-specific are from domctl.c and from iommu_construct(),
>> yet I doubt the flush_all(FLUSH_CACHE) has any relevance there in
>> the PV case.
> 
> Yeah, I got the impression that flushing is relevant to PV from reading
> existing code.

Well, I would have thought so as well, but the few places where the
function is possibly used on a PV domain all don't look as if the cache
flush could matter there (quite possibly it could even be avoided in
those specific cases for HVM, at the very least in iommu_construct()).

> If it is HVM only I suggest we add hvm prefix to it and only call it
> when the domain in question is hvm. This follows what we already do in
> other places.
> 
>if ( is_hvm_domain(d) )
>hvm_memory_type_change(...)

Yeah, that would probably be fine if we indeed settle on the function
being meaningless for PV.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v21 0/2] guest resource mapping (reprise)

2018-08-08 Thread Paul Durrant
These patches are the patches from my original resource mapping series
that did not make it into 4.11.

Paul Durrant (2):
  common: add a new mappable resource type: XENMEM_resource_grant_table
  tools/libxenctrl: use new xenforeignmemory API to seed grant table

 tools/libxc/include/xc_dom.h|   8 +--
 tools/libxc/xc_dom_boot.c   | 114 +---
 tools/libxc/xc_sr_restore_x86_hvm.c |  10 ++--
 tools/libxc/xc_sr_restore_x86_pv.c  |   2 +-
 tools/libxl/libxl_dom.c |   1 -
 tools/python/xen/lowlevel/xc/xc.c   |   6 +-
 xen/common/grant_table.c|  95 ++
 xen/common/memory.c |  55 -
 xen/include/public/memory.h |   6 ++
 xen/include/xen/grant_table.h   |   4 ++
 10 files changed, 238 insertions(+), 63 deletions(-)

-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v21 1/2] common: add a new mappable resource type: XENMEM_resource_grant_table

2018-08-08 Thread Paul Durrant
This patch allows grant table frames to be mapped using the
XENMEM_acquire_resource memory op.

NOTE: This patch expands the on-stack mfn_list array in acquire_resource()
  but it is still small enough to remain on-stack.

Signed-off-by: Paul Durrant 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: Wei Liu 

v21:
 - Prevent PVH/HVM tools domains from mapping any non-caller-owned resource
   unless the tools domain is also the hardware domain.
 - Grow the grant table appropriately whether it is a shared frame or
   a status frame that is being mapped.
 - Fix comment style breakage in memory.h.
 - Move implicit version setting to gnttab_get_shared_frame().

v19:
 - Add test to prevent PVH/HVM tools domains mapping grant status frames
   this way as the mapping infrastructure in Xen does not yet implement the
   necessary reference counting to make this safe.
 - Make sure grant table version is set before any attempt to grow the table.

v18:
 - Non-trivial re-base of grant table code.
 - Dropped Jan's R-b because of the grant table changes.

v13:
 - Re-work the internals to avoid using the XENMAPIDX_grant_table_status
   hack.

v12:
 - Dropped limit checks as requested by Jan.

v10:
 - Addressed comments from Jan.

v8:
 - The functionality was originally incorporated into the earlier patch
   "x86/mm: add HYPERVISOR_memory_op to acquire guest resources".
---
 xen/common/grant_table.c  | 95 +--
 xen/common/memory.c   | 55 -
 xen/include/public/memory.h   |  6 +++
 xen/include/xen/grant_table.h |  4 ++
 4 files changed, 146 insertions(+), 14 deletions(-)

diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index d9ec711c73..8e623ea08e 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -358,6 +358,12 @@ static inline unsigned int grant_to_status_frames(unsigned 
int grant_frames)
 return DIV_ROUND_UP(grant_frames * GRANT_PER_PAGE, GRANT_STATUS_PER_PAGE);
 }
 
+/* Number of grant table entries. Caller must hold d's gr. table lock.*/
+static inline unsigned int status_to_grant_frames(unsigned int status_frames)
+{
+return DIV_ROUND_UP(status_frames * GRANT_STATUS_PER_PAGE, GRANT_PER_PAGE);
+}
+
 /* Check if the page has been paged out, or needs unsharing.
If rc == GNTST_okay, *page contains the page struct with a ref taken.
Caller must do put_page(*page).
@@ -3860,6 +3866,47 @@ int mem_sharing_gref_to_gfn(struct grant_table *gt, 
grant_ref_t ref,
 }
 #endif
 
+/* caller must hold write lock */
+static int gnttab_get_status_frame_mfn(struct domain *d,
+   unsigned long idx, mfn_t *mfn)
+{
+struct grant_table *gt = d->grant_table;
+
+ASSERT(gt->gt_version == 2);
+
+if ( idx >= nr_status_frames(gt) )
+{
+unsigned long nr = status_to_grant_frames(idx + 1);
+
+if ( nr <= gt->max_grant_frames )
+gnttab_grow_table(d, nr);
+
+if ( idx >= nr_status_frames(gt) )
+return -EINVAL;
+}
+
+*mfn = _mfn(virt_to_mfn(gt->status[idx]));
+return 0;
+}
+
+/* caller must hold write lock */
+static int gnttab_get_shared_frame_mfn(struct domain *d,
+   unsigned long idx, mfn_t *mfn)
+{
+struct grant_table *gt = d->grant_table;
+
+ASSERT(gt->gt_version != 0);
+
+if ( (idx >= nr_grant_frames(gt)) && (idx < gt->max_grant_frames) )
+gnttab_grow_table(d, idx + 1);
+
+if ( idx >= nr_grant_frames(gt) )
+return -EINVAL;
+
+*mfn = _mfn(virt_to_mfn(gt->shared_raw[idx]));
+return 0;
+}
+
 int gnttab_map_frame(struct domain *d, unsigned long idx, gfn_t gfn,
  mfn_t *mfn)
 {
@@ -3877,21 +3924,11 @@ int gnttab_map_frame(struct domain *d, unsigned long 
idx, gfn_t gfn,
 {
 idx &= ~XENMAPIDX_grant_table_status;
 status = true;
-if ( idx < nr_status_frames(gt) )
-*mfn = _mfn(virt_to_mfn(gt->status[idx]));
-else
-rc = -EINVAL;
-}
-else
-{
-if ( (idx >= nr_grant_frames(gt)) && (idx < gt->max_grant_frames) )
-gnttab_grow_table(d, idx + 1);
 
-if ( idx < nr_grant_frames(gt) )
-*mfn = _mfn(virt_to_mfn(gt->shared_raw[idx]));
-else
-rc = -EINVAL;
+rc = gnttab_get_status_frame_mfn(d, idx, mfn);
 }
+else
+rc = gnttab_get_shared_frame_mfn(d, idx, mfn);
 
 if ( !rc && paging_mode_translate(d) &&
  !gfn_eq(gnttab_get_frame_gfn(gt, status, idx), INVALID_GFN) )
@@ -3906,6 +3943,38 @@ int gnttab_map_frame(struct domain *d, unsigned long 
idx, gfn_t gfn,
 return rc;
 }
 
+int gnttab_get_shared_frame(struct domain *d, unsigned long idx,
+mfn_t *mfn)
+{
+struct grant_table *gt = d->grant_table;
+int rc;
+
+grant_write_lock(gt);
+
+

[Xen-devel] [PATCH v21 2/2] tools/libxenctrl: use new xenforeignmemory API to seed grant table

2018-08-08 Thread Paul Durrant
A previous patch added support for priv-mapping guest resources directly
(rather than having to foreign-map, which requires P2M modification for
HVM guests).

This patch makes use of the new API to seed the guest grant table unless
the underlying infrastructure (i.e. privcmd) doesn't support it, in which
case the old scheme is used.

NOTE: The call to xc_dom_gnttab_hvm_seed() in hvm_build_set_params() was
  actually unnecessary, as the grant table has already been seeded
  by a prior call to xc_dom_gnttab_init() made by libxl__build_dom().

Signed-off-by: Paul Durrant 
Acked-by: Marek Marczykowski-Górecki 
Reviewed-by: Roger Pau Monné 
Acked-by: Wei Liu 
---
Cc: Ian Jackson 

v18:
 - Trivial re-base.

v13:
 - Re-base.

v10:
 - Use new id constant for grant table.

v4:
 - Minor cosmetic fix suggested by Roger.

v3:
 - Introduced xc_dom_set_gnttab_entry() to avoid duplicated code.
---
 tools/libxc/include/xc_dom.h|   8 +--
 tools/libxc/xc_dom_boot.c   | 114 +---
 tools/libxc/xc_sr_restore_x86_hvm.c |  10 ++--
 tools/libxc/xc_sr_restore_x86_pv.c  |   2 +-
 tools/libxl/libxl_dom.c |   1 -
 tools/python/xen/lowlevel/xc/xc.c   |   6 +-
 6 files changed, 92 insertions(+), 49 deletions(-)

diff --git a/tools/libxc/include/xc_dom.h b/tools/libxc/include/xc_dom.h
index 8a66889c68..a8a0c0da66 100644
--- a/tools/libxc/include/xc_dom.h
+++ b/tools/libxc/include/xc_dom.h
@@ -337,12 +337,8 @@ void *xc_dom_boot_domU_map(struct xc_dom_image *dom, 
xen_pfn_t pfn,
 int xc_dom_boot_image(struct xc_dom_image *dom);
 int xc_dom_compat_check(struct xc_dom_image *dom);
 int xc_dom_gnttab_init(struct xc_dom_image *dom);
-int xc_dom_gnttab_hvm_seed(xc_interface *xch, uint32_t domid,
-   xen_pfn_t console_gmfn,
-   xen_pfn_t xenstore_gmfn,
-   uint32_t console_domid,
-   uint32_t xenstore_domid);
-int xc_dom_gnttab_seed(xc_interface *xch, uint32_t domid,
+int xc_dom_gnttab_seed(xc_interface *xch, uint32_t guest_domid,
+   bool is_hvm,
xen_pfn_t console_gmfn,
xen_pfn_t xenstore_gmfn,
uint32_t console_domid,
diff --git a/tools/libxc/xc_dom_boot.c b/tools/libxc/xc_dom_boot.c
index 2e5681dc5d..8307ebeaf6 100644
--- a/tools/libxc/xc_dom_boot.c
+++ b/tools/libxc/xc_dom_boot.c
@@ -256,11 +256,29 @@ static xen_pfn_t xc_dom_gnttab_setup(xc_interface *xch, 
uint32_t domid)
 return gmfn;
 }
 
-int xc_dom_gnttab_seed(xc_interface *xch, uint32_t domid,
-   xen_pfn_t console_gmfn,
-   xen_pfn_t xenstore_gmfn,
-   uint32_t console_domid,
-   uint32_t xenstore_domid)
+static void xc_dom_set_gnttab_entry(xc_interface *xch,
+grant_entry_v1_t *gnttab,
+unsigned int idx,
+uint32_t guest_domid,
+uint32_t backend_domid,
+xen_pfn_t backend_gmfn)
+{
+if ( guest_domid == backend_domid || backend_gmfn == -1)
+return;
+
+xc_dom_printf(xch, "%s: [%u] -> 0x%"PRI_xen_pfn,
+  __FUNCTION__, idx, backend_gmfn);
+
+gnttab[idx].flags = GTF_permit_access;
+gnttab[idx].domid = backend_domid;
+gnttab[idx].frame = backend_gmfn;
+}
+
+static int compat_gnttab_seed(xc_interface *xch, uint32_t domid,
+  xen_pfn_t console_gmfn,
+  xen_pfn_t xenstore_gmfn,
+  uint32_t console_domid,
+  uint32_t xenstore_domid)
 {
 
 xen_pfn_t gnttab_gmfn;
@@ -284,18 +302,10 @@ int xc_dom_gnttab_seed(xc_interface *xch, uint32_t domid,
 return -1;
 }
 
-if ( domid != console_domid  && console_gmfn != -1)
-{
-gnttab[GNTTAB_RESERVED_CONSOLE].flags = GTF_permit_access;
-gnttab[GNTTAB_RESERVED_CONSOLE].domid = console_domid;
-gnttab[GNTTAB_RESERVED_CONSOLE].frame = console_gmfn;
-}
-if ( domid != xenstore_domid && xenstore_gmfn != -1)
-{
-gnttab[GNTTAB_RESERVED_XENSTORE].flags = GTF_permit_access;
-gnttab[GNTTAB_RESERVED_XENSTORE].domid = xenstore_domid;
-gnttab[GNTTAB_RESERVED_XENSTORE].frame = xenstore_gmfn;
-}
+xc_dom_set_gnttab_entry(xch, gnttab, GNTTAB_RESERVED_CONSOLE,
+domid, console_domid, console_gmfn);
+xc_dom_set_gnttab_entry(xch, gnttab, GNTTAB_RESERVED_XENSTORE,
+domid, xenstore_domid, xenstore_gmfn);
 
 if ( munmap(gnttab, PAGE_SIZE) == -1 )
 {
@@ -313,11 +323,11 @@ int xc_dom_gnttab_seed(xc_interface *xch, uint32_t domid,
 return 0;
 }
 
-int xc_dom_gnttab_hvm_seed(xc_interface *xch, uint32_t domid,
-   xen_pfn_t console_gpfn,
-

Re: [Xen-devel] PVH dom0 creation fails - the system freezes

2018-08-08 Thread bercarug

On 08/08/2018 11:51 AM, Roger Pau Monné wrote:

On Wed, Aug 08, 2018 at 09:43:39AM +0100, Paul Durrant wrote:

-Original Message-
From: Roger Pau Monne
Sent: 08 August 2018 09:08
To: berca...@amazon.com
Cc: Paul Durrant ; xen-devel ; David Woodhouse ;
Jan Beulich ; Belgun, Adrian 
Subject: Re: [Xen-devel] PVH dom0 creation fails - the system freezes

On Wed, Aug 08, 2018 at 10:46:40AM +0300, berca...@amazon.com wrote:

On 08/02/2018 04:55 PM, Roger Pau Monné wrote:

Please try to avoid top posting.

On Thu, Aug 02, 2018 at 11:36:26AM +, Bercaru, Gabriel wrote:

I applied the match mentioned, but the system fails to boot. Instead, it
drops to a BusyBox shell. It seems to be a file system issue.

So you have applied 173c7803592065d27bf2e60d50e08e197a0efa83 and it
causes a regression for you?

As I understand it, before applying 173c780359206 you where capable of
booting the PVH Dom0, albeit with non-working USB?

And after applying 173c780359206 you are no longer able to boot?

Right, after applying 173c780359206 the system fails to boot (it drops to
the BusyBox shell).

Following is a sequence of the boot log regarding the file system issue.

At least part of the issue seems to be that the IO-APIC information
provided to Dom0 is wrong (from the attached log):

[0.00] IOAPIC[0]: apic_id 2, version 152, address 0xfec0, GSI 0-

0

[0.00] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[0.00] ERROR: Unable to locate IOAPIC for GSI 2
[0.00] Failed to find ioapic for gsi : 2
[0.00] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[0.00] ERROR: Unable to locate IOAPIC for GSI 9
[0.00] Failed to find ioapic for gsi : 9
[0.00] ERROR: Unable to locate IOAPIC for GSI 1
[0.00] ERROR: Unable to locate IOAPIC for GSI 2
[0.00] ERROR: Unable to locate IOAPIC for GSI 3
[0.00] ERROR: Unable to locate IOAPIC for GSI 4
[0.00] ERROR: Unable to locate IOAPIC for GSI 5
[0.00] ERROR: Unable to locate IOAPIC for GSI 6
[0.00] ERROR: Unable to locate IOAPIC for GSI 7
[0.00] ERROR: Unable to locate IOAPIC for GSI 8
[0.00] ERROR: Unable to locate IOAPIC for GSI 9
[0.00] ERROR: Unable to locate IOAPIC for GSI 10
[0.00] ERROR: Unable to locate IOAPIC for GSI 11
[0.00] ERROR: Unable to locate IOAPIC for GSI 12
[0.00] ERROR: Unable to locate IOAPIC for GSI 13
[0.00] ERROR: Unable to locate IOAPIC for GSI 14
[0.00] ERROR: Unable to locate IOAPIC for GSI 15

Can you try to boot with just the staging branch (current commit is
008a8fb249b9d433) and see how that goes?

Thanks, Roger.


I recompiled Xen using the staging branch, commit 008a8fb249b9d433 and

the

system boots,

OK, so your issues where not caused by 173c780359206 then?

008a8fb249b9d433 already contains 173c780359206 because it was
committed earlier. In any case it's good to know you are able to boot
(albeit with issues) using the current staging branch.


however the USB problem persists. I was able to log in using the serial port
and after executing

Yes, the fixes for this issue have not been committed yet, see:

https://lists.xenproject.org/archives/html/xen-devel/2018-
08/msg00528.html

If you want you can give this branch a try, it should hopefully solve
your USB issues.


"xl list -l" the memory decrease problem appeared.

Yup, I will look into this now in order to find some kind of
workaround.


I attached the boot log. Following are some lines extracted from the log,
regarding the USB

devices problem:

[    5.864084] usb 1-1: device descriptor read/64, error -71

[    7.564025] usb 1-1: new full-speed USB device number 4 using xhci_hcd
[    7.571347] usb 1-1: Device not responding to setup address.

[    8.008031] usb 1-1: device not accepting address 4, error -71

[    8.609623] usb 1-1: device not accepting address 5, error -71


At the beginning of the log, there is a message regarding
iommu_inclusive_mapping:


(XEN) [VT-D]found ACPI_DMAR_RMRR:
(XEN) [VT-D]  RMRR address range 3e2e..3e2f not in reserved

memory;

need "iommu_inclusive_mapping=1"?
(XEN) [VT-D] endpoint: :00:14.0


I mention that I tried to boot again using this command line option, but
this message and the

USB messages persist.

Does USB work despite of the warning message?

No, it does not.



Yes, iommu_inclusive_mapping doesn't work for PVH, that's what my
patch series is trying to address. The error is caused by
missing/wrong RMRR regions in the ACPI tables.


Looks like this warning is suggesting that there is an RMRR that falls outside 
of an E820 reserved region. For PV I guess that 'inclusive' will fix this, but 
for PVH 'reserved' isn't going to fix it. I hope that the range at least falls 
in a hole, so maybe we also need a dom0_iommu=holes option too? Ugly, but maybe 
necessary.

I wanted to avoid adding such option because I think it's going to
interact quite badly 

Re: [Xen-devel] PVH dom0 creation fails - the system freezes

2018-08-08 Thread Roger Pau Monné
On Wed, Aug 08, 2018 at 09:43:39AM +0100, Paul Durrant wrote:
> > -Original Message-
> > From: Roger Pau Monne
> > Sent: 08 August 2018 09:08
> > To: berca...@amazon.com
> > Cc: Paul Durrant ; xen-devel  > de...@lists.xenproject.org>; David Woodhouse ;
> > Jan Beulich ; Belgun, Adrian 
> > Subject: Re: [Xen-devel] PVH dom0 creation fails - the system freezes
> > 
> > On Wed, Aug 08, 2018 at 10:46:40AM +0300, berca...@amazon.com wrote:
> > > On 08/02/2018 04:55 PM, Roger Pau Monné wrote:
> > > > Please try to avoid top posting.
> > > >
> > > > On Thu, Aug 02, 2018 at 11:36:26AM +, Bercaru, Gabriel wrote:
> > > > > I applied the match mentioned, but the system fails to boot. Instead, 
> > > > > it
> > > > > drops to a BusyBox shell. It seems to be a file system issue.
> > > > So you have applied 173c7803592065d27bf2e60d50e08e197a0efa83 and it
> > > > causes a regression for you?
> > > >
> > > > As I understand it, before applying 173c780359206 you where capable of
> > > > booting the PVH Dom0, albeit with non-working USB?
> > > >
> > > > And after applying 173c780359206 you are no longer able to boot?
> > > Right, after applying 173c780359206 the system fails to boot (it drops to
> > > the BusyBox shell).
> > > > > Following is a sequence of the boot log regarding the file system 
> > > > > issue.
> > > > At least part of the issue seems to be that the IO-APIC information
> > > > provided to Dom0 is wrong (from the attached log):
> > > >
> > > > [0.00] IOAPIC[0]: apic_id 2, version 152, address 0xfec0, 
> > > > GSI 0-
> > 0
> > > > [0.00] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 2
> > > > [0.00] Failed to find ioapic for gsi : 2
> > > > [0.00] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high 
> > > > level)
> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 9
> > > > [0.00] Failed to find ioapic for gsi : 9
> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 1
> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 2
> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 3
> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 4
> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 5
> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 6
> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 7
> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 8
> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 9
> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 10
> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 11
> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 12
> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 13
> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 14
> > > > [0.00] ERROR: Unable to locate IOAPIC for GSI 15
> > > >
> > > > Can you try to boot with just the staging branch (current commit is
> > > > 008a8fb249b9d433) and see how that goes?
> > > >
> > > > Thanks, Roger.
> > > >
> > > I recompiled Xen using the staging branch, commit 008a8fb249b9d433 and
> > the
> > > system boots,
> > 
> > OK, so your issues where not caused by 173c780359206 then?
> > 
> > 008a8fb249b9d433 already contains 173c780359206 because it was
> > committed earlier. In any case it's good to know you are able to boot
> > (albeit with issues) using the current staging branch.
> > 
> > > however the USB problem persists. I was able to log in using the serial 
> > > port
> > > and after executing
> > 
> > Yes, the fixes for this issue have not been committed yet, see:
> > 
> > https://lists.xenproject.org/archives/html/xen-devel/2018-
> > 08/msg00528.html
> > 
> > If you want you can give this branch a try, it should hopefully solve
> > your USB issues.
> > 
> > > "xl list -l" the memory decrease problem appeared.
> > 
> > Yup, I will look into this now in order to find some kind of
> > workaround.
> > 
> > > I attached the boot log. Following are some lines extracted from the log,
> > > regarding the USB
> > >
> > > devices problem:
> > >
> > > [    5.864084] usb 1-1: device descriptor read/64, error -71
> > >
> > > [    7.564025] usb 1-1: new full-speed USB device number 4 using xhci_hcd
> > > [    7.571347] usb 1-1: Device not responding to setup address.
> > >
> > > [    8.008031] usb 1-1: device not accepting address 4, error -71
> > >
> > > [    8.609623] usb 1-1: device not accepting address 5, error -71
> > >
> > >
> > > At the beginning of the log, there is a message regarding
> > > iommu_inclusive_mapping:
> > >
> > >
> > > (XEN) [VT-D]found ACPI_DMAR_RMRR:
> > > (XEN) [VT-D]  RMRR address range 3e2e..3e2f not in reserved
> > memory;
> > > need "iommu_inclusive_mapping=1"?
> > > (XEN) [VT-D] endpoint: :00:14.0
> > >
> > >
> > > I mention that I tried to boot again using this command line option, but
> > > this message and 

Re: [Xen-devel] PVH dom0 creation fails - the system freezes

2018-08-08 Thread Paul Durrant
> -Original Message-
> From: Roger Pau Monne
> Sent: 08 August 2018 09:08
> To: berca...@amazon.com
> Cc: Paul Durrant ; xen-devel  de...@lists.xenproject.org>; David Woodhouse ;
> Jan Beulich ; Belgun, Adrian 
> Subject: Re: [Xen-devel] PVH dom0 creation fails - the system freezes
> 
> On Wed, Aug 08, 2018 at 10:46:40AM +0300, berca...@amazon.com wrote:
> > On 08/02/2018 04:55 PM, Roger Pau Monné wrote:
> > > Please try to avoid top posting.
> > >
> > > On Thu, Aug 02, 2018 at 11:36:26AM +, Bercaru, Gabriel wrote:
> > > > I applied the match mentioned, but the system fails to boot. Instead, it
> > > > drops to a BusyBox shell. It seems to be a file system issue.
> > > So you have applied 173c7803592065d27bf2e60d50e08e197a0efa83 and it
> > > causes a regression for you?
> > >
> > > As I understand it, before applying 173c780359206 you where capable of
> > > booting the PVH Dom0, albeit with non-working USB?
> > >
> > > And after applying 173c780359206 you are no longer able to boot?
> > Right, after applying 173c780359206 the system fails to boot (it drops to
> > the BusyBox shell).
> > > > Following is a sequence of the boot log regarding the file system issue.
> > > At least part of the issue seems to be that the IO-APIC information
> > > provided to Dom0 is wrong (from the attached log):
> > >
> > > [0.00] IOAPIC[0]: apic_id 2, version 152, address 0xfec0, GSI 
> > > 0-
> 0
> > > [0.00] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
> > > [0.00] ERROR: Unable to locate IOAPIC for GSI 2
> > > [0.00] Failed to find ioapic for gsi : 2
> > > [0.00] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
> > > [0.00] ERROR: Unable to locate IOAPIC for GSI 9
> > > [0.00] Failed to find ioapic for gsi : 9
> > > [0.00] ERROR: Unable to locate IOAPIC for GSI 1
> > > [0.00] ERROR: Unable to locate IOAPIC for GSI 2
> > > [0.00] ERROR: Unable to locate IOAPIC for GSI 3
> > > [0.00] ERROR: Unable to locate IOAPIC for GSI 4
> > > [0.00] ERROR: Unable to locate IOAPIC for GSI 5
> > > [0.00] ERROR: Unable to locate IOAPIC for GSI 6
> > > [0.00] ERROR: Unable to locate IOAPIC for GSI 7
> > > [0.00] ERROR: Unable to locate IOAPIC for GSI 8
> > > [0.00] ERROR: Unable to locate IOAPIC for GSI 9
> > > [0.00] ERROR: Unable to locate IOAPIC for GSI 10
> > > [0.00] ERROR: Unable to locate IOAPIC for GSI 11
> > > [0.00] ERROR: Unable to locate IOAPIC for GSI 12
> > > [0.00] ERROR: Unable to locate IOAPIC for GSI 13
> > > [0.00] ERROR: Unable to locate IOAPIC for GSI 14
> > > [0.00] ERROR: Unable to locate IOAPIC for GSI 15
> > >
> > > Can you try to boot with just the staging branch (current commit is
> > > 008a8fb249b9d433) and see how that goes?
> > >
> > > Thanks, Roger.
> > >
> > I recompiled Xen using the staging branch, commit 008a8fb249b9d433 and
> the
> > system boots,
> 
> OK, so your issues where not caused by 173c780359206 then?
> 
> 008a8fb249b9d433 already contains 173c780359206 because it was
> committed earlier. In any case it's good to know you are able to boot
> (albeit with issues) using the current staging branch.
> 
> > however the USB problem persists. I was able to log in using the serial port
> > and after executing
> 
> Yes, the fixes for this issue have not been committed yet, see:
> 
> https://lists.xenproject.org/archives/html/xen-devel/2018-
> 08/msg00528.html
> 
> If you want you can give this branch a try, it should hopefully solve
> your USB issues.
> 
> > "xl list -l" the memory decrease problem appeared.
> 
> Yup, I will look into this now in order to find some kind of
> workaround.
> 
> > I attached the boot log. Following are some lines extracted from the log,
> > regarding the USB
> >
> > devices problem:
> >
> > [    5.864084] usb 1-1: device descriptor read/64, error -71
> >
> > [    7.564025] usb 1-1: new full-speed USB device number 4 using xhci_hcd
> > [    7.571347] usb 1-1: Device not responding to setup address.
> >
> > [    8.008031] usb 1-1: device not accepting address 4, error -71
> >
> > [    8.609623] usb 1-1: device not accepting address 5, error -71
> >
> >
> > At the beginning of the log, there is a message regarding
> > iommu_inclusive_mapping:
> >
> >
> > (XEN) [VT-D]found ACPI_DMAR_RMRR:
> > (XEN) [VT-D]  RMRR address range 3e2e..3e2f not in reserved
> memory;
> > need "iommu_inclusive_mapping=1"?
> > (XEN) [VT-D] endpoint: :00:14.0
> >
> >
> > I mention that I tried to boot again using this command line option, but
> > this message and the
> >
> > USB messages persist.
> 
> Yes, iommu_inclusive_mapping doesn't work for PVH, that's what my
> patch series is trying to address. The error is caused by
> missing/wrong RMRR regions in the ACPI tables.
> 

Looks like this warning is suggesting that there is an RMRR that falls outside 
of an E820 reserved region. 

Re: [Xen-devel] PVH dom0 creation fails - the system freezes

2018-08-08 Thread bercarug

On 08/08/2018 11:08 AM, Roger Pau Monné wrote:

On Wed, Aug 08, 2018 at 10:46:40AM +0300, berca...@amazon.com wrote:

On 08/02/2018 04:55 PM, Roger Pau Monné wrote:

Please try to avoid top posting.

On Thu, Aug 02, 2018 at 11:36:26AM +, Bercaru, Gabriel wrote:

I applied the match mentioned, but the system fails to boot. Instead, it
drops to a BusyBox shell. It seems to be a file system issue.

So you have applied 173c7803592065d27bf2e60d50e08e197a0efa83 and it
causes a regression for you?

As I understand it, before applying 173c780359206 you where capable of
booting the PVH Dom0, albeit with non-working USB?

And after applying 173c780359206 you are no longer able to boot?

Right, after applying 173c780359206 the system fails to boot (it drops to
the BusyBox shell).

Following is a sequence of the boot log regarding the file system issue.

At least part of the issue seems to be that the IO-APIC information
provided to Dom0 is wrong (from the attached log):

[0.00] IOAPIC[0]: apic_id 2, version 152, address 0xfec0, GSI 0-0
[0.00] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[0.00] ERROR: Unable to locate IOAPIC for GSI 2
[0.00] Failed to find ioapic for gsi : 2
[0.00] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[0.00] ERROR: Unable to locate IOAPIC for GSI 9
[0.00] Failed to find ioapic for gsi : 9
[0.00] ERROR: Unable to locate IOAPIC for GSI 1
[0.00] ERROR: Unable to locate IOAPIC for GSI 2
[0.00] ERROR: Unable to locate IOAPIC for GSI 3
[0.00] ERROR: Unable to locate IOAPIC for GSI 4
[0.00] ERROR: Unable to locate IOAPIC for GSI 5
[0.00] ERROR: Unable to locate IOAPIC for GSI 6
[0.00] ERROR: Unable to locate IOAPIC for GSI 7
[0.00] ERROR: Unable to locate IOAPIC for GSI 8
[0.00] ERROR: Unable to locate IOAPIC for GSI 9
[0.00] ERROR: Unable to locate IOAPIC for GSI 10
[0.00] ERROR: Unable to locate IOAPIC for GSI 11
[0.00] ERROR: Unable to locate IOAPIC for GSI 12
[0.00] ERROR: Unable to locate IOAPIC for GSI 13
[0.00] ERROR: Unable to locate IOAPIC for GSI 14
[0.00] ERROR: Unable to locate IOAPIC for GSI 15

Can you try to boot with just the staging branch (current commit is
008a8fb249b9d433) and see how that goes?

Thanks, Roger.


I recompiled Xen using the staging branch, commit 008a8fb249b9d433 and the
system boots,

OK, so your issues where not caused by 173c780359206 then?

008a8fb249b9d433 already contains 173c780359206 because it was
committed earlier. In any case it's good to know you are able to boot
(albeit with issues) using the current staging branch.


however the USB problem persists. I was able to log in using the serial port
and after executing

Yes, the fixes for this issue have not been committed yet, see:

https://lists.xenproject.org/archives/html/xen-devel/2018-08/msg00528.html

If you want you can give this branch a try, it should hopefully solve
your USB issues.


"xl list -l" the memory decrease problem appeared.

Yup, I will look into this now in order to find some kind of
workaround.


I attached the boot log. Following are some lines extracted from the log,
regarding the USB

devices problem:

[    5.864084] usb 1-1: device descriptor read/64, error -71

[    7.564025] usb 1-1: new full-speed USB device number 4 using xhci_hcd
[    7.571347] usb 1-1: Device not responding to setup address.

[    8.008031] usb 1-1: device not accepting address 4, error -71

[    8.609623] usb 1-1: device not accepting address 5, error -71


At the beginning of the log, there is a message regarding
iommu_inclusive_mapping:


(XEN) [VT-D]found ACPI_DMAR_RMRR:
(XEN) [VT-D]  RMRR address range 3e2e..3e2f not in reserved memory;
need "iommu_inclusive_mapping=1"?
(XEN) [VT-D] endpoint: :00:14.0


I mention that I tried to boot again using this command line option, but
this message and the

USB messages persist.

Yes, iommu_inclusive_mapping doesn't work for PVH, that's what my
patch series is trying to address. The error is caused by
missing/wrong RMRR regions in the ACPI tables.

Thanks, Roger.


I tried compiling from the branch mentioned. I changed the command line by

adding "dom0-iommu=reserved", but I get the same error messages about USB

during boot.


Gabriel




Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar 
Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in 
Romania. Registration number J22/2621/2005.
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 2/4] xen/blkfront: cleanup stale persistent grants

2018-08-08 Thread Roger Pau Monné
On Tue, Aug 07, 2018 at 05:56:38PM +0200, Juergen Gross wrote:
> On 07/08/18 16:14, Roger Pau Monné wrote:
> > On Tue, Aug 07, 2018 at 08:31:31AM +0200, Juergen Gross wrote:
> >> On 06/08/18 18:16, Roger Pau Monné wrote:
> >>> On Mon, Aug 06, 2018 at 01:34:01PM +0200, Juergen Gross wrote:
>  Add a periodic cleanup function to remove old persistent grants which
>  are no longer in use on the backend side. This avoids starvation in
>  case there are lots of persistent grants for a device which no longer
>  is involved in I/O business.
> 
>  Signed-off-by: Juergen Gross 
>  ---
>   drivers/block/xen-blkfront.c | 99 
>  ++--
>   1 file changed, 95 insertions(+), 4 deletions(-)
> 
>  diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
>  index b5cedccb5d7d..19feb8835fc4 100644
>  --- a/drivers/block/xen-blkfront.c
>  +++ b/drivers/block/xen-blkfront.c
>  @@ -46,6 +46,7 @@
>   #include 
>   #include 
>   #include 
>  +#include 
>   
>   #include 
>   #include 
>  @@ -121,6 +122,9 @@ static inline struct blkif_req *blkif_req(struct 
>  request *rq)
>   
>   static DEFINE_MUTEX(blkfront_mutex);
>   static const struct block_device_operations xlvbd_block_fops;
>  +static struct delayed_work blkfront_work;
>  +static LIST_HEAD(info_list);
>  +static bool blkfront_work_active;
>   
>   /*
>    * Maximum number of segments in indirect requests, the actual value 
>  used by
>  @@ -216,6 +220,7 @@ struct blkfront_info
>   /* Save uncomplete reqs and bios for migration. */
>   struct list_head requests;
>   struct bio_list bio_list;
>  +struct list_head info_list;
>   };
>   
>   static unsigned int nr_minors;
>  @@ -1764,6 +1769,12 @@ static int write_per_ring_nodes(struct 
>  xenbus_transaction xbt,
>   return err;
>   }
>   
>  +static void free_info(struct blkfront_info *info)
>  +{
>  +list_del(>info_list);
>  +kfree(info);
>  +}
>  +
>   /* Common code used when first setting up, and when resuming. */
>   static int talk_to_blkback(struct xenbus_device *dev,
>  struct blkfront_info *info)
>  @@ -1885,7 +1896,10 @@ static int talk_to_blkback(struct xenbus_device 
>  *dev,
>    destroy_blkring:
>   blkif_free(info, 0);
>   
>  -kfree(info);
>  +mutex_lock(_mutex);
>  +free_info(info);
>  +mutex_unlock(_mutex);
>  +
>   dev_set_drvdata(>dev, NULL);
>   
>   return err;
>  @@ -1996,6 +2010,10 @@ static int blkfront_probe(struct xenbus_device 
>  *dev,
>   info->handle = simple_strtoul(strrchr(dev->nodename, '/')+1, 
>  NULL, 0);
>   dev_set_drvdata(>dev, info);
>   
>  +mutex_lock(_mutex);
>  +list_add(>info_list, _list);
>  +mutex_unlock(_mutex);
>  +
>   return 0;
>   }
>   
>  @@ -2306,6 +2324,15 @@ static void 
>  blkfront_gather_backend_features(struct blkfront_info *info)
>   if (indirect_segments <= BLKIF_MAX_SEGMENTS_PER_REQUEST)
>   indirect_segments = 0;
>   info->max_indirect_segments = indirect_segments;
>  +
>  +if (info->feature_persistent) {
>  +mutex_lock(_mutex);
>  +if (!blkfront_work_active) {
>  +blkfront_work_active = true;
>  +schedule_delayed_work(_work, HZ * 10);
> >>>
> >>> Does it make sense to provide a module parameter to rune the schedule
> >>> of the cleanup routine?
> >>
> >> I don't think this is something anyone would like to tune.
> >>
> >> In case you think it should be tunable I can add a parameter, of course.
> > 
> > We can always add it later if required. I'm fine as-is now.
> > 
> >>>
>  +}
>  +mutex_unlock(_mutex);
> >>>
> >>> Is it really necessary to have the blkfront_work_active boolean? What
> >>> happens if you queue the same delayed work more than once?
> >>
> >> In case there is already work queued later calls of
> >> schedule_delayed_work() will be ignored.
> >>
> >> So yes, I can drop the global boolean (I still need a local flag in
> >> blkfront_delay_work() for controlling the need to call
> >> schedule_delayed_work() again).
> > 
> > Can't you just call schedule_delayed_work if info->feature_persistent
> > is set, even if that means calling it multiple times if multiple
> > blkfront instances are using persistent grants?
> 
> I don't like that. With mq we have a high chance for multiple instances
> to use persistent grants and a local bool is much cheaper than unneeded
> calls of 

[Xen-devel] [distros-debian-squeeze test] 75053: tolerable FAIL

2018-08-08 Thread Platform Team regression test user
flight 75053 distros-debian-squeeze real [real]
http://osstest.xensource.com/osstest/logs/75053/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-i386-squeeze-netboot-pygrub 10 debian-di-install fail like 
75033
 test-amd64-amd64-amd64-squeeze-netboot-pygrub 10 debian-di-install fail like 
75033
 test-amd64-i386-i386-squeeze-netboot-pygrub 10 debian-di-install fail like 
75033
 test-amd64-i386-amd64-squeeze-netboot-pygrub 10 debian-di-install fail like 
75033

baseline version:
 flight   75033

jobs:
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-amd64-squeeze-netboot-pygrubfail
 test-amd64-i386-amd64-squeeze-netboot-pygrub fail
 test-amd64-amd64-i386-squeeze-netboot-pygrub fail
 test-amd64-i386-i386-squeeze-netboot-pygrub  fail



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xensource.com/osstest/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries

2018-08-08 Thread Roger Pau Monné
On Tue, Aug 07, 2018 at 10:45:32AM -0600, Tamas K Lengyel wrote:
> On Tue, Aug 7, 2018 at 10:04 AM Tamas K Lengyel
>  wrote:
> (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow
> (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault
> (XEN) [VT-D]DMAR:[DMA Read] Request device [:00:02.0] fault addr
> 428f926000, iommu reg = 82c000a0c000
> (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
> (XEN) print_vtd_entries: iommu #0 dev :00:02.0 gmfn 428f926
> (XEN) root_entry[00] = 43aaae001
> (XEN) context[10] = 2_43cf92001
> (XEN) l4[000] = 9c043cf91107
> (XEN) l3[10a] = 8000
> (XEN) l3[10a] not present
> 
> The fault is repeated a million times per second and the system is
> pretty much stalled.

As Jan says, this page is outside of any range in the memory map. I
wonder however what's in there.

I think (also seeing the PV issues) you should bring this up with the
driver maintainers, it might actually be a bug that the driver is
trying to access such address.

In the meantime, you can try to add to the command line:

rmrr=0x428f926=0:0:2.0

In order to force an iommu mapping of this address.

Thanks, Roger.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] PVH dom0 creation fails - the system freezes

2018-08-08 Thread Roger Pau Monné
On Wed, Aug 08, 2018 at 10:46:40AM +0300, berca...@amazon.com wrote:
> On 08/02/2018 04:55 PM, Roger Pau Monné wrote:
> > Please try to avoid top posting.
> > 
> > On Thu, Aug 02, 2018 at 11:36:26AM +, Bercaru, Gabriel wrote:
> > > I applied the match mentioned, but the system fails to boot. Instead, it
> > > drops to a BusyBox shell. It seems to be a file system issue.
> > So you have applied 173c7803592065d27bf2e60d50e08e197a0efa83 and it
> > causes a regression for you?
> > 
> > As I understand it, before applying 173c780359206 you where capable of
> > booting the PVH Dom0, albeit with non-working USB?
> > 
> > And after applying 173c780359206 you are no longer able to boot?
> Right, after applying 173c780359206 the system fails to boot (it drops to
> the BusyBox shell).
> > > Following is a sequence of the boot log regarding the file system issue.
> > At least part of the issue seems to be that the IO-APIC information
> > provided to Dom0 is wrong (from the attached log):
> > 
> > [0.00] IOAPIC[0]: apic_id 2, version 152, address 0xfec0, GSI 
> > 0-0
> > [0.00] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
> > [0.00] ERROR: Unable to locate IOAPIC for GSI 2
> > [0.00] Failed to find ioapic for gsi : 2
> > [0.00] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
> > [0.00] ERROR: Unable to locate IOAPIC for GSI 9
> > [0.00] Failed to find ioapic for gsi : 9
> > [0.00] ERROR: Unable to locate IOAPIC for GSI 1
> > [0.00] ERROR: Unable to locate IOAPIC for GSI 2
> > [0.00] ERROR: Unable to locate IOAPIC for GSI 3
> > [0.00] ERROR: Unable to locate IOAPIC for GSI 4
> > [0.00] ERROR: Unable to locate IOAPIC for GSI 5
> > [0.00] ERROR: Unable to locate IOAPIC for GSI 6
> > [0.00] ERROR: Unable to locate IOAPIC for GSI 7
> > [0.00] ERROR: Unable to locate IOAPIC for GSI 8
> > [0.00] ERROR: Unable to locate IOAPIC for GSI 9
> > [0.00] ERROR: Unable to locate IOAPIC for GSI 10
> > [0.00] ERROR: Unable to locate IOAPIC for GSI 11
> > [0.00] ERROR: Unable to locate IOAPIC for GSI 12
> > [0.00] ERROR: Unable to locate IOAPIC for GSI 13
> > [0.00] ERROR: Unable to locate IOAPIC for GSI 14
> > [0.00] ERROR: Unable to locate IOAPIC for GSI 15
> > 
> > Can you try to boot with just the staging branch (current commit is
> > 008a8fb249b9d433) and see how that goes?
> > 
> > Thanks, Roger.
> > 
> I recompiled Xen using the staging branch, commit 008a8fb249b9d433 and the
> system boots,

OK, so your issues where not caused by 173c780359206 then?

008a8fb249b9d433 already contains 173c780359206 because it was
committed earlier. In any case it's good to know you are able to boot
(albeit with issues) using the current staging branch.

> however the USB problem persists. I was able to log in using the serial port
> and after executing

Yes, the fixes for this issue have not been committed yet, see:

https://lists.xenproject.org/archives/html/xen-devel/2018-08/msg00528.html

If you want you can give this branch a try, it should hopefully solve
your USB issues.

> "xl list -l" the memory decrease problem appeared.

Yup, I will look into this now in order to find some kind of
workaround.

> I attached the boot log. Following are some lines extracted from the log,
> regarding the USB
> 
> devices problem:
> 
> [    5.864084] usb 1-1: device descriptor read/64, error -71
> 
> [    7.564025] usb 1-1: new full-speed USB device number 4 using xhci_hcd
> [    7.571347] usb 1-1: Device not responding to setup address.
> 
> [    8.008031] usb 1-1: device not accepting address 4, error -71
> 
> [    8.609623] usb 1-1: device not accepting address 5, error -71
> 
> 
> At the beginning of the log, there is a message regarding
> iommu_inclusive_mapping:
> 
> 
> (XEN) [VT-D]found ACPI_DMAR_RMRR:
> (XEN) [VT-D]  RMRR address range 3e2e..3e2f not in reserved memory;
> need "iommu_inclusive_mapping=1"?
> (XEN) [VT-D] endpoint: :00:14.0
> 
> 
> I mention that I tried to boot again using this command line option, but
> this message and the
> 
> USB messages persist.

Yes, iommu_inclusive_mapping doesn't work for PVH, that's what my
patch series is trying to address. The error is caused by
missing/wrong RMRR regions in the ACPI tables.

Thanks, Roger.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

  1   2   >