Testes as-is (to confirm we hit the bug)
1.0.0 (12:53:43): MIGRATE: in-release migrations
1.1.0 (12:53:43): Clean testbeds
1.1.1 (12:53:43): stop containers
1.1.2 (12:53:43): orig: restore containers from snapshot: xenial
1.1.3 (12:53:43): Restore testkvm-xenial-from
1.1.4 (12:53:44): Restore testkvm-xenial-to
1.1.5 (12:53:45): Restore testkvm-xenial-tononshared
1.1.6 (12:53:45): wait until containers are fully started
1.2.0 (12:53:56): unshare non shared container
1.2.1 (12:54:00): Version info after restore
1.2.2 (12:54:00): Version at testkvm-xenial-from: - qemu:
1:2.5+dfsg-5ubuntu10.31 libvirt: 1.3.1-1ubuntu10.24
1.2.3 (12:54:00): Bios versions at testkvm-xenial-from: - ipxe:
1.0.0+git-20150424.a25a16d-1ubuntu1.2 slof: 20151103+dfsg-1ubuntu1.1 efi:
not-installed
1.2.4 (12:54:01): Version at testkvm-xenial-to: - qemu:
1:2.5+dfsg-5ubuntu10.31 libvirt: 1.3.1-1ubuntu10.24
1.2.5 (12:54:01): Bios versions at testkvm-xenial-to: - ipxe:
1.0.0+git-20150424.a25a16d-1ubuntu1.2 slof: 20151103+dfsg-1ubuntu1.1 efi:
not-installed
1.2.6 (12:54:01): Version at testkvm-xenial-tononshared: - qemu:
1:2.5+dfsg-5ubuntu10.31 libvirt: 1.3.1-1ubuntu10.24
1.2.7 (12:54:01): Bios versions at testkvm-xenial-tononshared: - ipxe:
1.0.0+git-20150424.a25a16d-1ubuntu1.2 slof: 20151103+dfsg-1ubuntu1.1 efi:
not-installed
1.2.8 (12:54:12): Ensure old migration guests of any release are removed
1.2.9 (12:54:12): Remove all test guests of release trusty
1.2.10 (12:54:26): Remove all test guests of release xenial
1.2.11 (12:54:40): Remove all test guests of release bionic
1.2.12 (12:54:54): Remove all test guests of release cosmic
1.2.13 (12:55:07): Prep xenial guest creation on testkvm-xenial-from
1.2.14 (12:55:07): spawn migration guests
1.2.15 (13:00:59): Test machine type uniqueness within xenial => Pass
1.2.16 (13:00:59): Check for expected machine type to be set => Pass
2.0.0 (13:01:00): Test migrations within xenial - round 1/5
2.1.0 (13:01:00): Test live migration (extra option '') of a xenial guest
testkvm-xenial-from/testkvm-xenial-to
2.1.1 (13:01:00): live migration (extra option '') testkvm-xenial-from ->
testkvm-xenial-to => Failed detail=live migration failed
---
Then running the same upgrading to proposed (actually all of proposed,
so I hope nothing else in there breaks us now - as we have tested that
in advance and it was good).
1.0.0 (13:37:26): MIGRATE: in-release migrations
1.1.0 (13:37:26): Clean testbeds
1.1.1 (13:37:26): stop containers
1.1.2 (13:37:26): orig: restore containers from snapshot: xenial
1.1.3 (13:37:26): Restore testkvm-xenial-from
1.1.4 (13:37:27): Restore testkvm-xenial-to
1.1.5 (13:37:27): Restore testkvm-xenial-noupd
1.1.6 (13:37:28): Restore testkvm-xenial-tononshared
1.1.7 (13:37:28): wait until containers are fully started
1.2.0 (13:37:54): unshare non shared container
1.2.1 (13:37:58): Version info after restore
1.2.2 (13:37:58): Version at testkvm-xenial-from: - qemu:
1:2.5+dfsg-5ubuntu10.32 libvirt: 1.3.1-1ubuntu10.24
1.2.3 (13:37:58): Bios versions at testkvm-xenial-from: - ipxe:
1.0.0+git-20150424.a25a16d-1ubuntu1.2 slof: 20151103+dfsg-1ubuntu1.1 efi:
not-installed
1.2.4 (13:37:59): Version at testkvm-xenial-to: - qemu:
1:2.5+dfsg-5ubuntu10.32 libvirt: 1.3.1-1ubuntu10.24
1.2.5 (13:37:59): Bios versions at testkvm-xenial-to: - ipxe:
1.0.0+git-20150424.a25a16d-1ubuntu1.2 slof: 20151103+dfsg-1ubuntu1.1 efi:
not-installed
1.2.6 (13:37:59): Version at testkvm-xenial-noupd: - qemu:
1:2.5+dfsg-5ubuntu10.31 libvirt: 1.3.1-1ubuntu10.24
1.2.7 (13:37:59): Bios versions at testkvm-xenial-noupd: - ipxe:
1.0.0+git-20150424.a25a16d-1ubuntu1.2 slof: 20151103+dfsg-1ubuntu1.1 efi:
not-installed
1.2.8 (13:38:00): Version at testkvm-xenial-tononshared: - qemu:
1:2.5+dfsg-5ubuntu10.32 libvirt: 1.3.1-1ubuntu10.24
1.2.9 (13:38:00): Bios versions at testkvm-xenial-tononshared: - ipxe:
1.0.0+git-20150424.a25a16d-1ubuntu1.2 slof: 20151103+dfsg-1ubuntu1.1 efi:
not-installed
1.2.10 (13:38:10): Ensure old migration guests of any release are removed
1.2.11 (13:38:10): Remove all test guests of release trusty
1.2.12 (13:38:27): Remove all test guests of release xenial
1.2.13 (13:38:45): Remove all test guests of release bionic
1.2.14 (13:39:02): Remove all test guests of release cosmic
1.2.15 (13:39:19): Prep xenial guest creation on testkvm-xenial-from
1.2.16 (13:39:19): spawn migration guests
1.2.17 (13:45:13): Test machine type uniqueness within xenial => Pass
1.2.18 (13:45:13): Check for expected machine type to be set => Pass
2.0.0 (13:45:14): Test migrations within xenial - round 1/5
2.1.0 (13:45:14): Test live migration (extra option '') of a xenial guest
testkvm-xenial-from/testkvm-xenial-to
2.1.1 (13:45:14): live migration (extra option '') testkvm-xenial-from ->
testkvm-xenial-to => Pass
2.1.2 (13:45:22): Check if guest kvmguest-xenial-normal on
testkvm-xenial-to is alive => Pass
2.1.3 (13:45:36): live migration back (extra option '') testkvm-xenial-to
-> testkvm-xenial-from => Pass
2.1.4 (13:45:44): Check if guest kvmguest-xenial-normal on
testkvm-xenial-from is alive => Pass
2.2.0 (13:45:45): Test saverestore migration of a xenial guest
testkvm-xenial-from/testkvm-xenial-to
2.2.1 (13:45:45): saverestore migration testkvm-xenial-from ->
testkvm-xenial-to => Pass
2.2.2 (13:45:57): Check if guest kvmguest-xenial-saverestore on
testkvm-xenial-to is alive => Pass
2.2.3 (13:46:11): saverestore migration back testkvm-xenial-to ->
testkvm-xenial-from => Pass
2.2.4 (13:46:23): Check if guest kvmguest-xenial-saverestore on
testkvm-xenial-from is alive => Pass
This goes on I'm already at test #172, but I'll spare you the flooding
of the comment box here.
Verified from proposed
** Tags removed: verification-needed verification-needed-xenial
** Tags added: verification-done verification-done-xenial
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1783140
Title:
KVM live migration fails
Status in The Ubuntu-power-systems project:
In Progress
Status in linux package in Ubuntu:
Triaged
Status in qemu package in Ubuntu:
Fix Released
Status in qemu source package in Xenial:
Fix Committed
Bug description:
[Impact]
* Backport fix from the 2.6.2 stable branch to the qemu 2.5 in Xenial
* Newer guests might use virtio attributes that are clobbered on
migration with the old qemu code.
[Test Case]
* Setup two Xenial hosts on ppc64el
* Create a guest that has a rather new kernel (>=4.14) I'd recommend
Bionic
* Migrate that guest from Host1 to Host2
[Regression Potential]
* The modification could affect virtio handling in other cases in a non
expected way, but mostly related to migrations. So the expected
regression would be issues to migrate properly.
I verified plenty of migrations in regression testing and we had
this very code in the Yakkety release as we picked 2.6.1 stable release
back then. Due to that it is actually pretty well tested and should not
really regress anything out in the wild.
[Other Info]
* So far this only triggers on the confused endian marshalling on
ppc64el, but in theory a different case could trigger it on x86 just as
much.
---
Environment:
2 POWER8 with Ubuntu 16.04.4 LTS as KVM hypervisor.
1 KVM guest with Ubuntu 18.04 LTS. Virtual disk for the guest is a qcow2 file
on an NFS share, accessible from both hypervisors, so live migration is
possible and works for all other guests (SLES, RHEL, Ubunutu 16.04),
Live migratino of Ubuntu 18.04 guest fails on ppc, while the same test on an
x86_64 environment suceeds.
root@pkvm2:~# virsh migrate --persistent --live p8lnxtst4
qemu+ssh://pkvm1/system
error: internal error: early end of file from monitor, possible problem:
2018-07-23T11:12:25.586385Z qemu-system-ppc64: VQ 0 size 0x100 Guest index
0x38aa inconsistent with Host index 0xa980: delta 0x8f2a
2018-07-23T11:12:25.586434Z qemu-system-ppc64: error while loading state for
instance 0x0 of device 'pci@800000020000000:01.0/virtio-net'
2018-07-23T11:12:25.587246Z qemu-system-ppc64: load of migration failed:
Operation not permitted
root@pkvm2:~# uname -a
Linux pkvm2 4.4.0-130-generic #156-Ubuntu SMP Thu Jun 14 08:51:21 UTC 2018
ppc64le ppc64le ppc64le GNU/Linux
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-power-systems/+bug/1783140/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
