From: "Dr. David Alan Gilbert" <dgilb...@redhat.com> This is the 2nd version of my work-in-progress cut of my visitor+BER format migration world; there's lots to do but it's starting to get there. I'd appreciate if anyone who has more experience with either the visitor code, or ASN.1/BER in general has a look to see if I'm doing anything particularly odd.
Since v1: * Rework of RAM ASN1 structure/encoding - Much more compact representation of all-0 pages - Killed off the 'Flags' field so that the ASN is much more natural and less tied to the old format - Support for XBZRLE (xbzrle blob is just wrapped and sent) - Support for a 'hook' used by RDMA * Added QEMUFilePart as a shim to avoid reading whole blobs into memory * End markers on sections to catch run away 0 data * Killed off debug visitor (was a pain to maintain for now while it's rather fluid) --- I've cherry picked bits of Michael Roth's visitor set from 2011 and Stefan Berger's BER code from this time last year, but not necessarily used things in the same way. The good: It can perform a migration using a BER format file that's described and verifiable against a schema. It can also perform a migration using the old binary format. A lot of the nasty detail of the old format is hidden in the compatibility visitor (e.g. the subsection end detection). The ugly: 1) There are a few shims that get passed visitors but need files or the other way that should be removable. The shim to get from a qemu file back to a visitor is patch 3 and marked as 'tmp' and the main uses are in piix4, pci.c, spapr_vscsi.c. 2) There are places where the visitor interface is too tied to the old file format, but most of those can be removed with effort. 3) I've made extensions to the Visitor type, many of these are very migration specific; and use things like get_next_type and the list visitors in quite a different way from the current uses. Should I really have a separate type? (see patch 1) 4) At the moment you select BER output format by setting an environment variable ( export QEMUMIGFORMAT=BER ) , I need to put more thought in to the right way to do this, there are some harder questions like what happens to devices that are still using pre-vmstate encodings (that are currently sent as blobs) when they eventually convert over and thus how to keep compatibility with earlier BER output versions where they were blobs. 5) I need to sort out my Error** handling (probably in a lot of places) 6) The visitors should be able to share much more code. The not-yet-done: a) block-migration b) SPAPR (it's an iterative migration like RAM and block) c) RDMA (I'm guessing I've broken it, but haven't tried) d) Floats are currently sent as blobs. e) What's the right way to deal with VMstate '_TEST' entries to make the format we produce more robust than the old one and still always have a valid ASN.1 schema for it. What I've done: There are separate output visitors (binary compatible, debug and BER) and they're selected by setting QEMUMIGFORMAT to 'BER' or just leaving it unset. The Input visitor is selected automatically from the first 4 bytes of the file. In general most types are BER sequences of indefinite length, with some types that I've allocated an Application specific type tag to. There is a hook to give any VMState it's own type tag (which I've shown using one example for cpu-common). Integers and strings are standard 'universal' types. Objects with .get/.put methods or register_savevm are saved as an 'octet string'. There are a few places where a device registered with .get/.put calls back into vmstate_save_state for part of their state (typically for devices built on PCI) and when they do that, even when in BER mode, those components get stored inside the octet string in the old format. I've used the 'asn1c' tool to validate the schema (which is in docs/specs/migration.schema) and also to verify streams that I've produced match the schema. I've tested it with virt-test (that now supports different source/dest qemu's) and tried bin-bin, ber-ber, and pre-visitor-qemu -> this and this -> pre-visitor-qemu just on the standard migration.default.tcp migration test, but I've not tried a large combination of configurations. My fix for qemu_peek_buffer that's on the list is needed to stabilise the binary format input visitor. I'll keep chipping away at that list, and would expect to pop another version out in a month or so. Dave Dr. David Alan Gilbert (16): Visitor: Add methods for migration format use QEMUSizedBuffer/QEMUFile QEMUFilePart: A shim to read part of a file qemu-file: Add set/get tmp_visitor Header/constant/types fixes for visitors Visitor: Binary compatible output visitor Visitor: Binary compatible input visitor Visitor: Output path Visitor: Load path Visitor: Common types to use visitors BER Visitor: Create output visitor BER Visitor: Create input visitor Start some BER format docs ASN.1 schema for new migration format Wire in BER visitors Add vmstate_cpu_common BER type code and wire it in. arch_init.c | 267 ++-- block-migration.c | 13 +- docs/migration.txt | 34 +- docs/specs/migration.schema | 139 +++ exec.c | 2 + hw/acpi/piix4.c | 5 +- hw/pci/pci.c | 5 +- hw/ppc/spapr.c | 9 +- hw/scsi/spapr_vscsi.c | 6 +- include/migration/migration.h | 19 + include/migration/qemu-file.h | 33 + include/migration/vmstate.h | 28 +- include/qapi/ber.h | 118 ++ include/qapi/qemu-file-ber-input-visitor.h | 26 + include/qapi/qemu-file-ber-output-visitor.h | 26 + include/qapi/qemu-file-binary-input-visitor.h | 27 + include/qapi/qemu-file-binary-output-visitor.h | 26 + include/qapi/visitor-impl.h | 23 + include/qapi/visitor.h | 51 + include/qemu/typedefs.h | 4 +- qapi/Makefile.objs | 3 +- qapi/qapi-visit-core.c | 80 ++ qapi/qemu-file-ber-input-visitor.c | 1575 ++++++++++++++++++++++++ qapi/qemu-file-ber-output-visitor.c | 1067 ++++++++++++++++ qapi/qemu-file-binary-input-visitor.c | 708 +++++++++++ qapi/qemu-file-binary-output-visitor.c | 576 +++++++++ qemu-file.c | 496 ++++++++ savevm.c | 408 ++++-- vmstate.c | 619 ++++------ 29 files changed, 5808 insertions(+), 585 deletions(-) create mode 100644 docs/specs/migration.schema create mode 100644 include/qapi/ber.h create mode 100644 include/qapi/qemu-file-ber-input-visitor.h create mode 100644 include/qapi/qemu-file-ber-output-visitor.h create mode 100644 include/qapi/qemu-file-binary-input-visitor.h create mode 100644 include/qapi/qemu-file-binary-output-visitor.h create mode 100644 qapi/qemu-file-ber-input-visitor.c create mode 100644 qapi/qemu-file-ber-output-visitor.c create mode 100644 qapi/qemu-file-binary-input-visitor.c create mode 100644 qapi/qemu-file-binary-output-visitor.c -- 1.9.0