TL;DR: There are issues, but IMHO switching can be justified. On Thu, 2015-09-03 at 09:59 +0100, Ian Campbell wrote: > TL;DR: There are issues which need fixing first... > > On Fri, 2015-08-21 at 17:24 +0000, osstest service owner wrote: > > flight 60785 linux-4.1 real [real] > > http://logs.test-lab.xenproject.org/osstest/logs/60785/ > > > > Failures :-/ but no regressions. > > > > Tests which are failing intermittently (not blocking): > > test-amd64-i386-xl-qemut-debianhvm-amd64-xsm 19 guest > > -start/debianhvm.repeat fail in 60746 pass in 60785 > > test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 16 guest > > -start/debianhvm.repeat fail pass in 60746 > > test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 13 guest > > -localmigrate fail pass in 60746 > > I've been considering updating osstest to switch from the current baseline > kernel (the one used for all non linux-* tests) from 3.14 to 4.1. > > The last pass on 3.14 was 60666[0] a little while ago on 13/8 (due to a > known issue[1]). As well as this pass of 4.1 today I therefore also > considered 60654[2] on 12/8. Two reports (like this one) are attached > comparing 3.16:60666 against 4.1:60654 and 4.1:60795 respectively.
Some time has passed and the issues with migration v2 are now fixed. The most recent pass of 3.14 is (still, sigh) 60666[0]. The most recent fail was 62085[1] which suffers from the same known issue WRT gntdev (which breaks qdisk based tests for qcow2 and vhd) The latest 4.1 pass was 62053[2] from Friday. Attached are two reports of 4.1:62053 vs 3.14:60666 and 3.14:62085. The failures are: vs. 60666 only: test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 13 guest-localmigrate fail REGR. vs. 60666 vs. 60666 and 62085: test-amd64-amd64-xl-qemuu-ovmf-amd64 19 guest-start/debianhvm.repeat fail REGR. vs. 60666 For the first considering the history: http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm/linux-3.14.html http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm/linux-4.1.html It seems pretty obvious that this is host specific, and in fact 60666 failed, just as guest-localmigrate.2 not guest-localmigrate like 62053 did. For the second the history: http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-amd64-xl-qemuu-ovmf-amd64/linux-3.14.html http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-amd64-xl-qemuu-ovmf-amd64/linux-4.1.html Suggests the issue is pretty intermittent. FWIW the push associated with 62085 justified this failure using 61947[3] where it passed with the exact same kernel version. So, I think it could be justified to switch our default kernel version to linux-4.1. Probably after 4.6 is released though. Ian. [0] http://lists.xen.org/archives/html/xen-devel/2015-09/msg00304.html [1] http://lists.xen.org/archives/html/xen-devel/2015-09/msg02441.html [2] http://lists.xen.org/archives/html/xen-devel/2015-09/msg02364.html [3] http://lists.xen.org/archives/html/xen-devel/2015-09/msg01921.html
62053: regressions - FAIL flight 62053 linux-4.1 real [real] http://logs.test-lab.xenproject.org/osstest/logs/62053/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 13 guest-localmigrate fail REGR. vs. 60666 test-amd64-amd64-xl-qemuu-ovmf-amd64 19 guest-start/debianhvm.repeat fail REGR. vs. 60666 Regressions which are regarded as allowable (not blocking): test-armhf-armhf-xl-rtds 11 guest-start fail blocked in 60666 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 13 guest-localmigrate fail blocked in 60666 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail like 60666 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail like 60666 Tests which did not succeed, but are not blocking: test-amd64-amd64-xl-pvh-intel 14 guest-saverestore fail never pass test-armhf-armhf-xl-qcow2 9 debian-di-install fail never pass test-armhf-armhf-libvirt-qcow2 9 debian-di-install fail never pass test-armhf-armhf-xl-vhd 9 debian-di-install fail never pass test-armhf-armhf-libvirt-vhd 9 debian-di-install fail never pass test-armhf-armhf-xl-raw 9 debian-di-install fail never pass test-amd64-amd64-xl-pvh-amd 11 guest-start fail never pass test-armhf-armhf-libvirt-raw 9 debian-di-install fail never pass test-armhf-armhf-libvirt-xsm 12 migrate-support-check fail never pass test-armhf-armhf-libvirt-xsm 14 guest-saverestore fail never pass test-amd64-amd64-libvirt-xsm 12 migrate-support-check fail never pass test-amd64-amd64-libvirt-pair 21 guest-migrate/src_host/dst_host fail never pass test-amd64-amd64-libvirt 12 migrate-support-check fail never pass test-armhf-armhf-libvirt 14 guest-saverestore fail never pass test-armhf-armhf-libvirt 12 migrate-support-check fail never pass test-amd64-i386-libvirt-xsm 12 migrate-support-check fail never pass test-amd64-i386-libvirt 12 migrate-support-check fail never pass test-amd64-i386-libvirt-pair 21 guest-migrate/src_host/dst_host fail never pass test-armhf-armhf-xl-xsm 13 saverestore-support-check fail never pass test-armhf-armhf-xl-xsm 12 migrate-support-check fail never pass test-armhf-armhf-xl-arndale 12 migrate-support-check fail never pass test-armhf-armhf-xl-arndale 13 saverestore-support-check fail never pass test-armhf-armhf-xl 12 migrate-support-check fail never pass test-armhf-armhf-xl 13 saverestore-support-check fail never pass test-armhf-armhf-xl-multivcpu 13 saverestore-support-check fail never pass test-armhf-armhf-xl-multivcpu 12 migrate-support-check fail never pass test-amd64-amd64-libvirt-qcow2 11 migrate-support-check fail never pass test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check fail never pass test-amd64-amd64-libvirt-raw 11 migrate-support-check fail never pass test-amd64-i386-libvirt-vhd 11 migrate-support-check fail never pass test-amd64-amd64-libvirt-vhd 11 migrate-support-check fail never pass test-amd64-i386-libvirt-raw 11 migrate-support-check fail never pass test-armhf-armhf-xl-credit2 13 saverestore-support-check fail never pass test-armhf-armhf-xl-credit2 12 migrate-support-check fail never pass test-armhf-armhf-xl-cubietruck 12 migrate-support-check fail never pass test-armhf-armhf-xl-cubietruck 13 saverestore-support-check fail never pass test-amd64-i386-libvirt-qcow2 11 migrate-support-check fail never pass test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check fail never pass test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail never pass test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail never pass baseline version: flight 60666 jobs: build-amd64-xsm pass build-armhf-xsm pass build-i386-xsm pass build-amd64 pass build-armhf pass build-i386 pass build-amd64-libvirt pass build-armhf-libvirt pass build-i386-libvirt pass build-amd64-pvops pass build-armhf-pvops pass build-i386-pvops pass build-amd64-rumpuserxen pass build-i386-rumpuserxen pass test-amd64-amd64-xl pass test-armhf-armhf-xl pass test-amd64-i386-xl pass test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm pass test-amd64-i386-xl-qemut-debianhvm-amd64-xsm pass test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm pass test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm pass test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm pass test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm pass test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm fail test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm fail test-amd64-amd64-libvirt-xsm pass test-armhf-armhf-libvirt-xsm fail test-amd64-i386-libvirt-xsm pass test-amd64-amd64-xl-xsm pass test-armhf-armhf-xl-xsm pass test-amd64-i386-xl-xsm pass test-amd64-amd64-xl-pvh-amd fail test-amd64-i386-qemut-rhel6hvm-amd pass test-amd64-i386-qemuu-rhel6hvm-amd pass test-amd64-amd64-xl-qemut-debianhvm-amd64 pass test-amd64-i386-xl-qemut-debianhvm-amd64 pass test-amd64-amd64-xl-qemuu-debianhvm-amd64 pass test-amd64-i386-xl-qemuu-debianhvm-amd64 pass test-amd64-i386-freebsd10-amd64 pass test-amd64-amd64-xl-qemuu-ovmf-amd64 fail test-amd64-i386-xl-qemuu-ovmf-amd64 pass test-amd64-amd64-rumpuserxen-amd64 pass test-amd64-amd64-xl-qemut-win7-amd64 fail test-amd64-i386-xl-qemut-win7-amd64 fail test-amd64-amd64-xl-qemuu-win7-amd64 fail test-amd64-i386-xl-qemuu-win7-amd64 fail test-armhf-armhf-xl-arndale pass test-amd64-amd64-xl-credit2 pass test-armhf-armhf-xl-credit2 pass test-armhf-armhf-xl-cubietruck pass test-amd64-i386-freebsd10-i386 pass test-amd64-i386-rumpuserxen-i386 pass test-amd64-amd64-xl-pvh-intel fail test-amd64-i386-qemut-rhel6hvm-intel pass test-amd64-i386-qemuu-rhel6hvm-intel pass test-amd64-amd64-libvirt pass test-armhf-armhf-libvirt fail test-amd64-i386-libvirt pass test-amd64-amd64-xl-multivcpu pass test-armhf-armhf-xl-multivcpu pass test-amd64-amd64-pair pass test-amd64-i386-pair pass test-amd64-amd64-libvirt-pair fail test-amd64-i386-libvirt-pair fail test-amd64-amd64-amd64-pvgrub pass test-amd64-amd64-i386-pvgrub pass test-amd64-amd64-pygrub pass test-amd64-amd64-libvirt-qcow2 pass test-armhf-armhf-libvirt-qcow2 fail test-amd64-i386-libvirt-qcow2 pass test-amd64-amd64-xl-qcow2 pass test-armhf-armhf-xl-qcow2 fail test-amd64-i386-xl-qcow2 pass test-amd64-amd64-libvirt-raw pass test-armhf-armhf-libvirt-raw fail test-amd64-i386-libvirt-raw pass test-amd64-amd64-xl-raw pass test-armhf-armhf-xl-raw fail test-amd64-i386-xl-raw pass test-amd64-amd64-xl-rtds pass test-armhf-armhf-xl-rtds fail test-amd64-i386-xl-qemut-winxpsp3-vcpus1 pass test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 pass test-amd64-amd64-libvirt-vhd pass test-armhf-armhf-libvirt-vhd fail test-amd64-i386-libvirt-vhd pass test-amd64-amd64-xl-vhd pass test-armhf-armhf-xl-vhd fail test-amd64-i386-xl-vhd pass test-amd64-amd64-xl-qemut-winxpsp3 pass test-amd64-i386-xl-qemut-winxpsp3 pass test-amd64-amd64-xl-qemuu-winxpsp3 pass test-amd64-i386-xl-qemuu-winxpsp3 pass ------------------------------------------------------------ sg-report-flight on osstest.test-lab.xenproject.org logs: /home/logs/logs images: /home/logs/images Logs, config files, etc. are available at http://logs.test-lab.xenproject.org/osstest/logs Explanation of these reports, and of osstest in general, is at http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master Test harness code can be found at http://xenbits.xen.org/gitweb?p=osstest.git;a=summary
62053: regressions - FAIL flight 62053 linux-4.1 real [real] http://logs.test-lab.xenproject.org/osstest/logs/62053/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-xl-qemuu-ovmf-amd64 19 guest-start/debianhvm.repeat fail REGR. vs. 62085 Regressions which are regarded as allowable (not blocking): test-armhf-armhf-xl-rtds 11 guest-start fail blocked in 62085 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 13 guest-localmigrate fail blocked in 62085 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 13 guest-localmigrate fail like 62085 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail like 62085 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail like 62085 Tests which did not succeed, but are not blocking: test-amd64-amd64-xl-pvh-intel 14 guest-saverestore fail never pass test-armhf-armhf-xl-qcow2 9 debian-di-install fail never pass test-armhf-armhf-libvirt-qcow2 9 debian-di-install fail never pass test-armhf-armhf-xl-vhd 9 debian-di-install fail never pass test-armhf-armhf-libvirt-vhd 9 debian-di-install fail never pass test-armhf-armhf-xl-raw 9 debian-di-install fail never pass test-amd64-amd64-xl-pvh-amd 11 guest-start fail never pass test-armhf-armhf-libvirt-raw 9 debian-di-install fail never pass test-armhf-armhf-libvirt-xsm 12 migrate-support-check fail never pass test-armhf-armhf-libvirt-xsm 14 guest-saverestore fail never pass test-amd64-amd64-libvirt-xsm 12 migrate-support-check fail never pass test-amd64-amd64-libvirt-pair 21 guest-migrate/src_host/dst_host fail never pass test-amd64-amd64-libvirt 12 migrate-support-check fail never pass test-armhf-armhf-libvirt 14 guest-saverestore fail never pass test-armhf-armhf-libvirt 12 migrate-support-check fail never pass test-amd64-i386-libvirt-xsm 12 migrate-support-check fail never pass test-amd64-i386-libvirt 12 migrate-support-check fail never pass test-amd64-i386-libvirt-pair 21 guest-migrate/src_host/dst_host fail never pass test-armhf-armhf-xl-xsm 13 saverestore-support-check fail never pass test-armhf-armhf-xl-xsm 12 migrate-support-check fail never pass test-armhf-armhf-xl-arndale 12 migrate-support-check fail never pass test-armhf-armhf-xl-arndale 13 saverestore-support-check fail never pass test-armhf-armhf-xl 12 migrate-support-check fail never pass test-armhf-armhf-xl 13 saverestore-support-check fail never pass test-armhf-armhf-xl-multivcpu 13 saverestore-support-check fail never pass test-armhf-armhf-xl-multivcpu 12 migrate-support-check fail never pass test-amd64-amd64-libvirt-qcow2 11 migrate-support-check fail never pass test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check fail never pass test-amd64-amd64-libvirt-raw 11 migrate-support-check fail never pass test-amd64-i386-libvirt-vhd 11 migrate-support-check fail never pass test-amd64-amd64-libvirt-vhd 11 migrate-support-check fail never pass test-amd64-i386-libvirt-raw 11 migrate-support-check fail never pass test-armhf-armhf-xl-credit2 13 saverestore-support-check fail never pass test-armhf-armhf-xl-credit2 12 migrate-support-check fail never pass test-armhf-armhf-xl-cubietruck 12 migrate-support-check fail never pass test-armhf-armhf-xl-cubietruck 13 saverestore-support-check fail never pass test-amd64-i386-libvirt-qcow2 11 migrate-support-check fail never pass test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check fail never pass test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail never pass test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail never pass baseline version: flight 62085 jobs: build-amd64-xsm pass build-armhf-xsm pass build-i386-xsm pass build-amd64 pass build-armhf pass build-i386 pass build-amd64-libvirt pass build-armhf-libvirt pass build-i386-libvirt pass build-amd64-pvops pass build-armhf-pvops pass build-i386-pvops pass build-amd64-rumpuserxen pass build-i386-rumpuserxen pass test-amd64-amd64-xl pass test-armhf-armhf-xl pass test-amd64-i386-xl pass test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm pass test-amd64-i386-xl-qemut-debianhvm-amd64-xsm pass test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm pass test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm pass test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm pass test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm pass test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm fail test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm fail test-amd64-amd64-libvirt-xsm pass test-armhf-armhf-libvirt-xsm fail test-amd64-i386-libvirt-xsm pass test-amd64-amd64-xl-xsm pass test-armhf-armhf-xl-xsm pass test-amd64-i386-xl-xsm pass test-amd64-amd64-xl-pvh-amd fail test-amd64-i386-qemut-rhel6hvm-amd pass test-amd64-i386-qemuu-rhel6hvm-amd pass test-amd64-amd64-xl-qemut-debianhvm-amd64 pass test-amd64-i386-xl-qemut-debianhvm-amd64 pass test-amd64-amd64-xl-qemuu-debianhvm-amd64 pass test-amd64-i386-xl-qemuu-debianhvm-amd64 pass test-amd64-i386-freebsd10-amd64 pass test-amd64-amd64-xl-qemuu-ovmf-amd64 fail test-amd64-i386-xl-qemuu-ovmf-amd64 pass test-amd64-amd64-rumpuserxen-amd64 pass test-amd64-amd64-xl-qemut-win7-amd64 fail test-amd64-i386-xl-qemut-win7-amd64 fail test-amd64-amd64-xl-qemuu-win7-amd64 fail test-amd64-i386-xl-qemuu-win7-amd64 fail test-armhf-armhf-xl-arndale pass test-amd64-amd64-xl-credit2 pass test-armhf-armhf-xl-credit2 pass test-armhf-armhf-xl-cubietruck pass test-amd64-i386-freebsd10-i386 pass test-amd64-i386-rumpuserxen-i386 pass test-amd64-amd64-xl-pvh-intel fail test-amd64-i386-qemut-rhel6hvm-intel pass test-amd64-i386-qemuu-rhel6hvm-intel pass test-amd64-amd64-libvirt pass test-armhf-armhf-libvirt fail test-amd64-i386-libvirt pass test-amd64-amd64-xl-multivcpu pass test-armhf-armhf-xl-multivcpu pass test-amd64-amd64-pair pass test-amd64-i386-pair pass test-amd64-amd64-libvirt-pair fail test-amd64-i386-libvirt-pair fail test-amd64-amd64-amd64-pvgrub pass test-amd64-amd64-i386-pvgrub pass test-amd64-amd64-pygrub pass test-amd64-amd64-libvirt-qcow2 pass test-armhf-armhf-libvirt-qcow2 fail test-amd64-i386-libvirt-qcow2 pass test-amd64-amd64-xl-qcow2 pass test-armhf-armhf-xl-qcow2 fail test-amd64-i386-xl-qcow2 pass test-amd64-amd64-libvirt-raw pass test-armhf-armhf-libvirt-raw fail test-amd64-i386-libvirt-raw pass test-amd64-amd64-xl-raw pass test-armhf-armhf-xl-raw fail test-amd64-i386-xl-raw pass test-amd64-amd64-xl-rtds pass test-armhf-armhf-xl-rtds fail test-amd64-i386-xl-qemut-winxpsp3-vcpus1 pass test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 pass test-amd64-amd64-libvirt-vhd pass test-armhf-armhf-libvirt-vhd fail test-amd64-i386-libvirt-vhd pass test-amd64-amd64-xl-vhd pass test-armhf-armhf-xl-vhd fail test-amd64-i386-xl-vhd pass test-amd64-amd64-xl-qemut-winxpsp3 pass test-amd64-i386-xl-qemut-winxpsp3 pass test-amd64-amd64-xl-qemuu-winxpsp3 pass test-amd64-i386-xl-qemuu-winxpsp3 pass ------------------------------------------------------------ sg-report-flight on osstest.test-lab.xenproject.org logs: /home/logs/logs images: /home/logs/images Logs, config files, etc. are available at http://logs.test-lab.xenproject.org/osstest/logs Explanation of these reports, and of osstest in general, is at http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master Test harness code can be found at http://xenbits.xen.org/gitweb?p=osstest.git;a=summary
_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
