On Thu, Oct 10, 2019 at 04:58:09PM +0100, Dr. David Alan Gilbert wrote: > * Michael S. Tsirkin (m...@redhat.com) wrote: > > On Thu, Oct 10, 2019 at 02:11:15PM +0200, Mikhail Sennikovsky wrote: > > > Currently offloads disabled by guest via the > > > VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET > > > command are not preserved on VM migration. > > > Instead all offloads reported by guest features (via > > > VIRTIO_PCI_GUEST_FEATURES) > > > get enabled. > > > What happens is: first the VirtIONet::curr_guest_offloads gets restored > > > and offloads are getting set correctly: > > > > > > #0 qemu_set_offload (nc=0x555556a11400, csum=1, tso4=0, tso6=0, ecn=0, > > > ufo=0) at net/net.c:474 > > > #1 virtio_net_apply_guest_offloads (n=0x555557701ca0) at > > > hw/net/virtio-net.c:720 > > > #2 virtio_net_post_load_device (opaque=0x555557701ca0, version_id=11) > > > at hw/net/virtio-net.c:2334 > > > #3 vmstate_load_state (f=0x5555569dc010, vmsd=0x555556577c80 > > > <vmstate_virtio_net_device>, opaque=0x555557701ca0, version_id=11) > > > at migration/vmstate.c:168 > > > #4 virtio_load (vdev=0x555557701ca0, f=0x5555569dc010, version_id=11) > > > at hw/virtio/virtio.c:2197 > > > #5 virtio_device_get (f=0x5555569dc010, opaque=0x555557701ca0, size=0, > > > field=0x55555668cd00 <__compound_literal.5>) at hw/virtio/virtio.c:2036 > > > #6 vmstate_load_state (f=0x5555569dc010, vmsd=0x555556577ce0 > > > <vmstate_virtio_net>, opaque=0x555557701ca0, version_id=11) at > > > migration/vmstate.c:143 > > > #7 vmstate_load (f=0x5555569dc010, se=0x5555578189e0) at > > > migration/savevm.c:829 > > > #8 qemu_loadvm_section_start_full (f=0x5555569dc010, > > > mis=0x5555569eee20) at migration/savevm.c:2211 > > > #9 qemu_loadvm_state_main (f=0x5555569dc010, mis=0x5555569eee20) at > > > migration/savevm.c:2395 > > > #10 qemu_loadvm_state (f=0x5555569dc010) at migration/savevm.c:2467 > > > #11 process_incoming_migration_co (opaque=0x0) at > > > migration/migration.c:449 > > > > > > However later on the features are getting restored, and offloads get > > > reset to > > > everything supported by features: > > > > > > #0 qemu_set_offload (nc=0x555556a11400, csum=1, tso4=1, tso6=1, ecn=0, > > > ufo=0) at net/net.c:474 > > > #1 virtio_net_apply_guest_offloads (n=0x555557701ca0) at > > > hw/net/virtio-net.c:720 > > > #2 virtio_net_set_features (vdev=0x555557701ca0, features=5104441767) > > > at hw/net/virtio-net.c:773 > > > #3 virtio_set_features_nocheck (vdev=0x555557701ca0, val=5104441767) at > > > hw/virtio/virtio.c:2052 > > > #4 virtio_load (vdev=0x555557701ca0, f=0x5555569dc010, version_id=11) > > > at hw/virtio/virtio.c:2220 > > > #5 virtio_device_get (f=0x5555569dc010, opaque=0x555557701ca0, size=0, > > > field=0x55555668cd00 <__compound_literal.5>) at hw/virtio/virtio.c:2036 > > > #6 vmstate_load_state (f=0x5555569dc010, vmsd=0x555556577ce0 > > > <vmstate_virtio_net>, opaque=0x555557701ca0, version_id=11) at > > > migration/vmstate.c:143 > > > #7 vmstate_load (f=0x5555569dc010, se=0x5555578189e0) at > > > migration/savevm.c:829 > > > #8 qemu_loadvm_section_start_full (f=0x5555569dc010, > > > mis=0x5555569eee20) at migration/savevm.c:2211 > > > #9 qemu_loadvm_state_main (f=0x5555569dc010, mis=0x5555569eee20) at > > > migration/savevm.c:2395 > > > #10 qemu_loadvm_state (f=0x5555569dc010) at migration/savevm.c:2467 > > > #11 process_incoming_migration_co (opaque=0x0) at > > > migration/migration.c:449 > > > > > > Fix this by making the virtio_net_set_features not reset > > > the curr_guest_offloads in case the VM runstate is RUN_STATE_INMIGRATE > > > > > > Signed-off-by: Mikhail Sennikovsky <mikhail.sennikovs...@cloud.ionos.com> > > > > I thought more about it and I really think the problem here is > > different: we really should first set features and then apply offloads. > > > > It is simply dumb that we are calling device specific > > code before initializing generic virtio. > > > > The logical order of initialization should be: > > > > init base class > > init derived class > > > > > > instead we have it all spread out :( > > Yeh; we've hit problems there before. > > > > > What to do about it? > > Could we maybe teach vmstate to record post load hooks > > it needs to run *somewhere*? > > Then we could load device specific state and record post load > > hooks, then load generic state, run generic post load, > > then run device specific post load hooks. > > > > Dave, what do you think? > > I worry about changing all existing virtio devices; who knows > what existing ordering constraints those devices might have on their > post_load's - for example one field in the vmstate might rely on > another post_load having already run and set a flag; that's not > that unusual. > > I can imagine having a Notifier* chain; you initialise it at the start > of virtio_load, provide a virtio_load_post_notifier_add(...) and any > post_load that wants delaying can add itself and then you call it > at the right point in virtio_load; but I'd be too scared to change > the ordering of all post_loads of all devices. You might be > OK to do that for the top level post_load of a virtio device, > because there are only a few to check. > > Dave
Okay so let's just add a new hook for that. I'll send a patch that does this shortly. Mikhail, I'll send a couple of untested patches, please try them - either ack or rework in any way you like.
