Re: TUN_F_UFO change breaks live migration
On Tue, 2014-11-11 at 17:57 +0200, Michael S. Tsirkin wrote:
> On Tue, Nov 11, 2014 at 12:17:26PM +, Ben Hutchings wrote:
> > On Tue, 2014-11-11 at 10:58 +, Stefan Hajnoczi wrote:
> > > Commit 3d0ad09412ffe00c9afa201d01effdb6023d09b4 ("drivers/net: Disable
> > > UFO through virtio") breaks live migration of KVM guests from older to
> > > newer host kernels:
> > >
> > > https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=3d0ad09412ffe00c9afa201d01effdb6023d09b4
> > >
> > > The problem occurs when a guest running on a host kernel without commit
> > > 3d0ad0941 in tun.ko attempts to live migration to a host with commit
> > > 3d0ad0941.
> > >
> > > Live migration fails in QEMU with the following error message:
> > >
> > > virtio-net: saved image requires TUN_F_UFO support
> > >
> > > The old host tun.ko advertised support for TUN_F_UFO. The new host
> > > tun.ko does not and that's why QEMU aborts live migration. QEMU cannot
> > > change the features of a running virtio-net device.
> >
> > Yes, this is known and was mentioned in the DSA.
> >
> > > tuxcrafter provided logs from two Debian hosts migrating from
> > > 3.2.60-1+deb7u3 to 3.2.63-2+deb7u1:
> > >
> > > http://paste.debian.net/131264/
> > >
> > > I haven't investigated enough to suggest a fix, just wanted to bring it
> > > to your attention. Soon a lot of people will be hitting this problem as
> > > they upgrade their infrastructure and migrate guests - seems like a
> > > critical issue.
> >
> > You can work around this by making macvtap and tun still claim to
> > support UFO.
>
> If this is what we want userspace to do, let's just put the
> feature flag back?
Let's not *just* put the feature flag back. I accept this is probably
needed as a workaround, but UFO/IPv6 still won't work correctly over
virtio.
> Basically userspace assumed that features will only
> ever be added, never removed, so this change is
> breaking it.
OK.
> > They continue to support it even if it's not advertised
> > because the tap features don't reliably get propagated to virtio
> > devices.
> >
> > Ben.
>
> Hmm I don't understand this last sentence.
> features are actually reliably propagated to virtio devices.
They might be when using current QEMU and libvirt on the host. They
weren't when I tested on Debian stable. The warnings about 'using
disabled UFO feature' are reliably triggered if the host's tap driver is
patched and the guest's virtio_net driver is not.
Ben.
--
Ben Hutchings
Experience is directly proportional to the value of equipment destroyed.
- Carolyn Scheppner
signature.asc
Description: This is a digitally signed message part
Re: TUN_F_UFO change breaks live migration
From: "Michael S. Tsirkin" Date: Tue, 11 Nov 2014 17:57:50 +0200 > Basically userspace assumed that features will only > ever be added, never removed, so this change is > breaking it. I essentially agree. We can't just toss feature bits like this which have been present for so long. "There is no way I can figure out how to easily make it work" is not an excuse for breaking existing userspace like this. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: TUN_F_UFO change breaks live migration
On Tue, Nov 11, 2014 at 12:17:26PM +, Ben Hutchings wrote:
> On Tue, 2014-11-11 at 10:58 +, Stefan Hajnoczi wrote:
> > Commit 3d0ad09412ffe00c9afa201d01effdb6023d09b4 ("drivers/net: Disable
> > UFO through virtio") breaks live migration of KVM guests from older to
> > newer host kernels:
> >
> > https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=3d0ad09412ffe00c9afa201d01effdb6023d09b4
> >
> > The problem occurs when a guest running on a host kernel without commit
> > 3d0ad0941 in tun.ko attempts to live migration to a host with commit
> > 3d0ad0941.
> >
> > Live migration fails in QEMU with the following error message:
> >
> > virtio-net: saved image requires TUN_F_UFO support
> >
> > The old host tun.ko advertised support for TUN_F_UFO. The new host
> > tun.ko does not and that's why QEMU aborts live migration. QEMU cannot
> > change the features of a running virtio-net device.
>
> Yes, this is known and was mentioned in the DSA.
>
> > tuxcrafter provided logs from two Debian hosts migrating from
> > 3.2.60-1+deb7u3 to 3.2.63-2+deb7u1:
> >
> > http://paste.debian.net/131264/
> >
> > I haven't investigated enough to suggest a fix, just wanted to bring it
> > to your attention. Soon a lot of people will be hitting this problem as
> > they upgrade their infrastructure and migrate guests - seems like a
> > critical issue.
>
> You can work around this by making macvtap and tun still claim to
> support UFO.
If this is what we want userspace to do, let's just put the
feature flag back?
Basically userspace assumed that features will only
ever be added, never removed, so this change is
breaking it.
> They continue to support it even if it's not advertised
> because the tap features don't reliably get propagated to virtio
> devices.
>
> Ben.
Hmm I don't understand this last sentence.
features are actually reliably propagated to virtio devices.
> --
> Ben Hutchings
> Experience is directly proportional to the value of equipment destroyed.
> - Carolyn Scheppner
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: TUN_F_UFO change breaks live migration
On Tue, 2014-11-11 at 10:58 +, Stefan Hajnoczi wrote:
> Commit 3d0ad09412ffe00c9afa201d01effdb6023d09b4 ("drivers/net: Disable
> UFO through virtio") breaks live migration of KVM guests from older to
> newer host kernels:
>
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=3d0ad09412ffe00c9afa201d01effdb6023d09b4
>
> The problem occurs when a guest running on a host kernel without commit
> 3d0ad0941 in tun.ko attempts to live migration to a host with commit
> 3d0ad0941.
>
> Live migration fails in QEMU with the following error message:
>
> virtio-net: saved image requires TUN_F_UFO support
>
> The old host tun.ko advertised support for TUN_F_UFO. The new host
> tun.ko does not and that's why QEMU aborts live migration. QEMU cannot
> change the features of a running virtio-net device.
Yes, this is known and was mentioned in the DSA.
> tuxcrafter provided logs from two Debian hosts migrating from
> 3.2.60-1+deb7u3 to 3.2.63-2+deb7u1:
>
> http://paste.debian.net/131264/
>
> I haven't investigated enough to suggest a fix, just wanted to bring it
> to your attention. Soon a lot of people will be hitting this problem as
> they upgrade their infrastructure and migrate guests - seems like a
> critical issue.
You can work around this by making macvtap and tun still claim to
support UFO. They continue to support it even if it's not advertised
because the tap features don't reliably get propagated to virtio
devices.
Ben.
--
Ben Hutchings
Experience is directly proportional to the value of equipment destroyed.
- Carolyn Scheppner
signature.asc
Description: This is a digitally signed message part
