Re: [Qemu-devel] [PATCH 1/1] RFC: net/slirp: link with libslirp
* Daniel P. Berrangé (berra...@redhat.com) wrote: > On Mon, Feb 11, 2019 at 11:22:27PM +0100, Samuel Thibault wrote: > > Marc-André Lureau, le lun. 11 févr. 2019 12:34:47 +0100, a ecrit: > > > On Mon, Feb 11, 2019 at 12:09 PM Daniel P. Berrangé > > > wrote: > > > > > > > > On Fri, Feb 08, 2019 at 07:12:26PM +0100, Marc-André Lureau wrote: > > > > > Once libslirp has received its first release, we can link with the > > > > > external libslirp library. > > > > > > > > > > The migration data should be compatible with current and older qemu > > > > > versions (same compatibility as today). See "slirp: add state > > > > > saving/loading" patch. However, the content should be treated as a > > > > > blob, as the format may change eventually in the future. > > > > > > > > How are we going to manage live migration compat if libslirp changes > > > > the blob content ? > > > > > > > > Bear in mind that we need to support all existing QEMU releases live > > > > migrating to effectively all future QEMU releases, with all future > > > > libslirp releases, in *both* directions. ie arbitrarily newer > > > > libslirp needs to be able to emit a blob format that can be read > > > > by arbitrarily older slirp inside QEMU. > > > > > > Right, this is all supported currently with the proposed patch set, > > > since it is effectively the same code. > > > > > > So register_savevm_live() get passed slirp_state_version() (currently == > > > 4) > > > > > > & slirp_state_load() get the version_id from QEMU. > > > > Mmm, but do we guarantee that the current version of slirp will be able > > to read blobs produced by future versions of slirp? > > > > Future extensions of the format would have to be so that the current > > version could discard their content without issues. > > > > Perhaps qemu should actually explicitly pass 4 to > > register_savevm_live(), for that function to only record that format > > (and thus get compatibility of course) and only bump to greater values > > when qemu is modified to make use of a functionality which requires > > extending the blob format, which then makes it unreadable by older qemu > > releases, but that is fine for qemu. > > If QEMU wants to make use of functionality that requires the new blob > format, it would have to restrict that based on machine type. That way > it can ensure full compatibility with old QEMU. Library compatibility for migration blobs is messy; we've had the same problem with usbredir. It's very difficult to keep track of; for example lets say that libslirp gets used by 3 or 4 different projects, and the package maintainer for libslirp in your favorite distro updates it - it's not necessarily obvious to them that doing so could break qemu migration. The ideal way is to work in terms of features and not versions; so that a machine type can enable libslirp-feature-foo in the new machine type and only then will the migration contents change. Dave > Regards, > Daniel > -- > |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o-https://fstop138.berrange.com :| > |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :| -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK
Re: [Qemu-devel] [PATCH 1/1] RFC: net/slirp: link with libslirp
On Mon, Feb 11, 2019 at 11:22:27PM +0100, Samuel Thibault wrote: > Marc-André Lureau, le lun. 11 févr. 2019 12:34:47 +0100, a ecrit: > > On Mon, Feb 11, 2019 at 12:09 PM Daniel P. Berrangé > > wrote: > > > > > > On Fri, Feb 08, 2019 at 07:12:26PM +0100, Marc-André Lureau wrote: > > > > Once libslirp has received its first release, we can link with the > > > > external libslirp library. > > > > > > > > The migration data should be compatible with current and older qemu > > > > versions (same compatibility as today). See "slirp: add state > > > > saving/loading" patch. However, the content should be treated as a > > > > blob, as the format may change eventually in the future. > > > > > > How are we going to manage live migration compat if libslirp changes > > > the blob content ? > > > > > > Bear in mind that we need to support all existing QEMU releases live > > > migrating to effectively all future QEMU releases, with all future > > > libslirp releases, in *both* directions. ie arbitrarily newer > > > libslirp needs to be able to emit a blob format that can be read > > > by arbitrarily older slirp inside QEMU. > > > > Right, this is all supported currently with the proposed patch set, > > since it is effectively the same code. > > > > So register_savevm_live() get passed slirp_state_version() (currently == 4) > > > > & slirp_state_load() get the version_id from QEMU. > > Mmm, but do we guarantee that the current version of slirp will be able > to read blobs produced by future versions of slirp? > > Future extensions of the format would have to be so that the current > version could discard their content without issues. > > Perhaps qemu should actually explicitly pass 4 to > register_savevm_live(), for that function to only record that format > (and thus get compatibility of course) and only bump to greater values > when qemu is modified to make use of a functionality which requires > extending the blob format, which then makes it unreadable by older qemu > releases, but that is fine for qemu. If QEMU wants to make use of functionality that requires the new blob format, it would have to restrict that based on machine type. That way it can ensure full compatibility with old QEMU. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
Re: [Qemu-devel] [PATCH 1/1] RFC: net/slirp: link with libslirp
Marc-André Lureau, le lun. 11 févr. 2019 12:34:47 +0100, a ecrit: > On Mon, Feb 11, 2019 at 12:09 PM Daniel P. Berrangé > wrote: > > > > On Fri, Feb 08, 2019 at 07:12:26PM +0100, Marc-André Lureau wrote: > > > Once libslirp has received its first release, we can link with the > > > external libslirp library. > > > > > > The migration data should be compatible with current and older qemu > > > versions (same compatibility as today). See "slirp: add state > > > saving/loading" patch. However, the content should be treated as a > > > blob, as the format may change eventually in the future. > > > > How are we going to manage live migration compat if libslirp changes > > the blob content ? > > > > Bear in mind that we need to support all existing QEMU releases live > > migrating to effectively all future QEMU releases, with all future > > libslirp releases, in *both* directions. ie arbitrarily newer > > libslirp needs to be able to emit a blob format that can be read > > by arbitrarily older slirp inside QEMU. > > Right, this is all supported currently with the proposed patch set, > since it is effectively the same code. > > So register_savevm_live() get passed slirp_state_version() (currently == 4) > > & slirp_state_load() get the version_id from QEMU. Mmm, but do we guarantee that the current version of slirp will be able to read blobs produced by future versions of slirp? Future extensions of the format would have to be so that the current version could discard their content without issues. Perhaps qemu should actually explicitly pass 4 to register_savevm_live(), for that function to only record that format (and thus get compatibility of course) and only bump to greater values when qemu is modified to make use of a functionality which requires extending the blob format, which then makes it unreadable by older qemu releases, but that is fine for qemu. Samuel
Re: [Qemu-devel] [PATCH 1/1] RFC: net/slirp: link with libslirp
On Mon, Feb 11, 2019 at 12:34:47PM +0100, Marc-André Lureau wrote: > Hi > > On Mon, Feb 11, 2019 at 12:09 PM Daniel P. Berrangé > wrote: > > > > On Fri, Feb 08, 2019 at 07:12:26PM +0100, Marc-André Lureau wrote: > > > Once libslirp has received its first release, we can link with the > > > external libslirp library. > > > > > > The migration data should be compatible with current and older qemu > > > versions (same compatibility as today). See "slirp: add state > > > saving/loading" patch. However, the content should be treated as a > > > blob, as the format may change eventually in the future. > > > > How are we going to manage live migration compat if libslirp changes > > the blob content ? > > > > Bear in mind that we need to support all existing QEMU releases live > > migrating to effectively all future QEMU releases, with all future > > libslirp releases, in *both* directions. ie arbitrarily newer > > libslirp needs to be able to emit a blob format that can be read > > by arbitrarily older slirp inside QEMU. > > > Right, this is all supported currently with the proposed patch set, > since it is effectively the same code. > > So register_savevm_live() get passed slirp_state_version() (currently == 4) > > & slirp_state_load() get the version_id from QEMU. > > > > > Normally we tie data format changes to the machine type. How are > > we going to achieve such machine type associations with an external > > libslirp ? > > Is this relevant for slirp? I don't see any version tied to machine > type. Am I missing something? Old to new would work provided new slirp can read & understand all previous slirp state versions. It is entirely possible no one ever tested new -> old live migration with slirp. I doubt we would have tested it in RHEL, since we don't really care very much about slirp in context of migration. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
Re: [Qemu-devel] [PATCH 1/1] RFC: net/slirp: link with libslirp
Hi On Mon, Feb 11, 2019 at 12:09 PM Daniel P. Berrangé wrote: > > On Fri, Feb 08, 2019 at 07:12:26PM +0100, Marc-André Lureau wrote: > > Once libslirp has received its first release, we can link with the > > external libslirp library. > > > > The migration data should be compatible with current and older qemu > > versions (same compatibility as today). See "slirp: add state > > saving/loading" patch. However, the content should be treated as a > > blob, as the format may change eventually in the future. > > How are we going to manage live migration compat if libslirp changes > the blob content ? > > Bear in mind that we need to support all existing QEMU releases live > migrating to effectively all future QEMU releases, with all future > libslirp releases, in *both* directions. ie arbitrarily newer > libslirp needs to be able to emit a blob format that can be read > by arbitrarily older slirp inside QEMU. Right, this is all supported currently with the proposed patch set, since it is effectively the same code. So register_savevm_live() get passed slirp_state_version() (currently == 4) & slirp_state_load() get the version_id from QEMU. > > Normally we tie data format changes to the machine type. How are > we going to achieve such machine type associations with an external > libslirp ? Is this relevant for slirp? I don't see any version tied to machine type. Am I missing something? thanks > > It seems to me that if a libslirp wants to change live migration > format, it will need to support both the old and new formats > indefinitely, and the new format must be an opt-in. QEMU can > then tell libslirp which migraiton format based on the machine > type. > > ok > > Regards, > Daniel > -- > |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o-https://fstop138.berrange.com :| > |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
Re: [Qemu-devel] [PATCH 1/1] RFC: net/slirp: link with libslirp
On Fri, Feb 08, 2019 at 07:12:26PM +0100, Marc-André Lureau wrote: > Once libslirp has received its first release, we can link with the > external libslirp library. > > The migration data should be compatible with current and older qemu > versions (same compatibility as today). See "slirp: add state > saving/loading" patch. However, the content should be treated as a > blob, as the format may change eventually in the future. How are we going to manage live migration compat if libslirp changes the blob content ? Bear in mind that we need to support all existing QEMU releases live migrating to effectively all future QEMU releases, with all future libslirp releases, in *both* directions. ie arbitrarily newer libslirp needs to be able to emit a blob format that can be read by arbitrarily older slirp inside QEMU. Normally we tie data format changes to the machine type. How are we going to achieve such machine type associations with an external libslirp ? It seems to me that if a libslirp wants to change live migration format, it will need to support both the old and new formats indefinitely, and the new format must be an opt-in. QEMU can then tell libslirp which migraiton format based on the machine type. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
Re: [Qemu-devel] [PATCH 1/1] RFC: net/slirp: link with libslirp
Hi On Sun, Feb 10, 2019 at 10:37 PM Peter Maydell wrote: > > On Fri, 8 Feb 2019 at 20:44, Marc-André Lureau > wrote: > > > > Once libslirp has received its first release, we can link with the > > external libslirp library. > > > > The migration data should be compatible with current and older qemu > > versions (same compatibility as today). See "slirp: add state > > saving/loading" patch. However, the content should be treated as a > > blob, as the format may change eventually in the future. > > Could you elaborate a bit on what you mean by "the format > might change" ? It's not clear to me what libslirp's API/ABI > guarantees are here to ensure that QEMU can retain cross-QEMU-version > migration compatibility and that QEMU version X with libslirp > version Y can migrate both to and from QEMU version X with > libslirp version Z. So far, the migration code should be 1:1 with existing qemu code. It's tricky to tell the compatibility on existing code, but the saved bitstream version_id is 4, and it can load some older versions (I am not sure all the way down to original migration code). QEMU bitstream version shouldn't be relevant here, as qemu will get and provide a binary blob. libslirp should aim at keeping compatibility with its own older bitstreams. A newer version will fail to be read with a library that doesn't support it. Does that sound correct? thanks
Re: [Qemu-devel] [PATCH 1/1] RFC: net/slirp: link with libslirp
On Fri, 8 Feb 2019 at 20:44, Marc-André Lureau wrote: > > Once libslirp has received its first release, we can link with the > external libslirp library. > > The migration data should be compatible with current and older qemu > versions (same compatibility as today). See "slirp: add state > saving/loading" patch. However, the content should be treated as a > blob, as the format may change eventually in the future. Could you elaborate a bit on what you mean by "the format might change" ? It's not clear to me what libslirp's API/ABI guarantees are here to ensure that QEMU can retain cross-QEMU-version migration compatibility and that QEMU version X with libslirp version Y can migrate both to and from QEMU version X with libslirp version Z. thanks -- PMM
Re: [Qemu-devel] [PATCH 1/1] RFC: net/slirp: link with libslirp
Marc-André Lureau, le ven. 08 févr. 2019 19:12:26 +0100, a ecrit: > Once libslirp has received its first release, we can link with the > external libslirp library. > > The migration data should be compatible with current and older qemu > versions (same compatibility as today). See "slirp: add state > saving/loading" patch. However, the content should be treated as a > blob, as the format may change eventually in the future. > > Signed-off-by: Marc-André Lureau Reviewed-by: Samuel Thibault Thanks so much for all this work :)
Re: [Qemu-devel] [PATCH 1/1] RFC: net/slirp: link with libslirp
On 2/8/19 7:12 PM, Marc-André Lureau wrote: > Once libslirp has received its first release, we can link with the > external libslirp library. Is there a scheduled date? > The migration data should be compatible with current and older qemu > versions (same compatibility as today). See "slirp: add state > saving/loading" patch. However, the content should be treated as a > blob, as the format may change eventually in the future. > > Signed-off-by: Marc-André Lureau > --- > include/migration/qemu-file-types.h |2 + > migration/qemu-file.h |1 - > slirp/bootp.h | 128 --- > slirp/debug.h | 48 - > slirp/dhcpv6.h | 25 - > slirp/if.h | 23 - > slirp/ip.h | 241 - > slirp/ip6.h | 159 --- > slirp/ip6_icmp.h| 231 > slirp/ip_icmp.h | 165 --- > slirp/libslirp.h| 107 -- > slirp/main.h| 18 - > slirp/mbuf.h| 126 --- > slirp/misc.h| 68 -- > slirp/ncsi-pkt.h| 419 > slirp/qtailq.h | 193 > slirp/sbuf.h| 29 - > slirp/slirp.h | 274 - > slirp/socket.h | 162 --- > slirp/state.h |9 - > slirp/tcp.h | 180 > slirp/tcp_timer.h | 127 --- > slirp/tcp_var.h | 161 --- > slirp/tcpip.h | 101 -- > slirp/tftp.h| 51 - > slirp/udp.h | 91 -- > slirp/util.h| 127 --- > net/slirp.c | 48 +- > slirp/arp_table.c | 91 -- > slirp/bootp.c | 370 --- > slirp/cksum.c | 160 --- > slirp/dhcpv6.c | 206 > slirp/dnssearch.c | 310 -- > slirp/if.c | 220 > slirp/ip6_icmp.c| 437 > slirp/ip6_input.c | 77 -- > slirp/ip6_output.c | 38 - > slirp/ip_icmp.c | 469 > slirp/ip_input.c| 471 > slirp/ip_output.c | 172 --- > slirp/mbuf.c| 237 > slirp/misc.c| 320 -- > slirp/ncsi.c| 166 --- > slirp/ndp_table.c | 86 -- > slirp/sbuf.c| 188 > slirp/slirp.c | 1121 --- > slirp/socket.c | 935 > slirp/state.c | 394 --- > slirp/tcp_input.c | 1554 --- > slirp/tcp_output.c | 524 - > slirp/tcp_subr.c| 985 - > slirp/tcp_timer.c | 293 - > slirp/tftp.c| 462 > slirp/udp.c | 361 --- > slirp/udp6.c| 172 --- > slirp/util.c| 207 > util/main-loop.c|2 - > vl.c|2 - > MAINTAINERS |1 - > Makefile.objs |1 - > configure | 25 + > net/Makefile.objs |2 + > slirp/COPYRIGHT | 61 -- > slirp/Makefile.objs | 34 - > util/Makefile.objs |1 + > 65 files changed, 76 insertions(+), 14393 deletions(-) Nice...
