From: "Dr. David Alan Gilbert" <dgilb...@redhat.com> This is a work in progress cut of my visitor+BER format migration world; there's lots to do but it's starting to get there. I'd appreciate if anyone who has more experience with either the visitor code, or ASN.1/BER in general has a look to see if I'm doing anything particularly odd.
I've cherry picked bits of Michael Roth's visitor set from 2011 and Stefan Berger's BER code from this time last year, but not necessarily used things in the same way. The good: It can perform a migration using a BER format file that's described and verifiable against a schema. It can also perform a migration using the old binary format. A lot of the nasty detail of the old format is hidden in the compatibility visitor (e.g. the subsection end detection). The ugly: 1) The encoding for zero pages is bloated, it can be simplified a lot (see previous discussion with Michael from a week or two back that I've not yet implemented) 2) There are a few shims that get passed visitors but need files or the other way that should be removable. The shim to get from a qemu file back to a visitor is patch 3 and marked as 'tmp' and the main uses are in piix4, pci.c, spapr_vscsi.c. 3) There are places where the visitor interface is too tied to the old file format, but most of those can be removed with effort. 4) I've made extensions to the Visitor type, many of these are very migration specific; and use things like get_next_type and the list visitors in quite a different way from the current uses. Should I really have a separate type? (see patch 1) 5) At the moment you select BER output format by setting an environment variable ( export QEMUMIGFORMAT=BER ) , I need to put more thought in to the right way to do this, there are some harder questions like what happens to devices that are still using pre-vmstate encodings (that are currently sent as blobs) when they eventually convert over and thus how to keep compatibility with earlier BER output versions where they were blobs. 6) At the moment for blobs (i.e. pre-vmstate) the whole blob gets loaded into memory and then processed; that's not necessary and can be done with a QEMUFile shim. 7) I need to sort out my Error** handling (probably in a lot of places) 8) The visitors should be able to share much more code. The not-yet-done: a) XBZRLE b) block-migration c) SPAPR (it's an iterative migration like RAM and block) d) RDMA (I'm guessing I've broken it, but haven't tried) e) Floats are currently sent as blobs. What I've done: There are separate output visitors (binary compatible, debug and BER) and they're selected by setting QEMUMIGFORMAT to one of 'debug', 'BER' or just leaving it unset. The Input visitor is selected automatically from the first 4 bytes of the file. In general most types are BER sequences of indefinite length, with some types that I've allocated an Application specific type tag to. There is a hook to give any VMState it's own type tag (which I've shown using one example for cpu-common). Integers and strings are standard 'universal' types. Objects with .get/.put methods or register_savevm are saved as an 'octet string'. There are a few places where a device registered with .get/.put calls back into vmstate_save_state for part of their state (typically for devices built on PCI) and when they do that, even when in BER mode, those components get stored inside the octet string in the old format. I've used the 'asn1c' tool to validate the schema (which is in docs/specs/migration.schema) and also to verify streams that I've produced match the schema. I've tested it with virt-test (hacked to have different source/dest qemu's) and tried bin-bin, ber-ber, and pre-visitor-qemu -> this and this -> pre-visitor-qemu just on the standard migration.default.tcp migration test, but I've not tried a large combination of configurations. My fix for qemu_peek_buffer that's on the list is needed to stabilise the binary format input visitor. I'll keep chipping away at that list, and would expect to pop another version out in a month or so. Dave Dr. David Alan Gilbert (16): Visitor: Add methods for migration format use QEMUSizedBuffer/QEMUFile qemu-file: Add set/get tmp_visitor Header/constant/types fixes for visitors Visitor: Binary compatible output visitor Visitor: Debug output visitor Visitor: Binary compatible input visitor Visitor: Output path Visitor: Load path Visitor: Common types to use visitors Choose output visitor based on env variable BER Visitor: Create output visitor BER Visitor: Create input visitor Start some BER format docs ASN.1 schema for new migration format Wire in BER visitors arch_init.c | 231 +++-- block-migration.c | 13 +- docs/migration.txt | 34 +- docs/specs/migration.schema | 113 +++ exec.c | 2 + hw/acpi/piix4.c | 5 +- hw/pci/pci.c | 5 +- hw/ppc/spapr.c | 9 +- hw/scsi/spapr_vscsi.c | 6 +- include/migration/migration.h | 17 + include/migration/qemu-file.h | 31 + include/migration/vmstate.h | 28 +- include/qapi/ber.h | 108 +++ include/qapi/qemu-file-ber-input-visitor.h | 26 + include/qapi/qemu-file-ber-output-visitor.h | 26 + include/qapi/qemu-file-binary-input-visitor.h | 27 + include/qapi/qemu-file-binary-output-visitor.h | 26 + include/qapi/qemu-file-debug-output-visitor.h | 26 + include/qapi/visitor-impl.h | 23 + include/qapi/visitor.h | 51 ++ include/qemu/typedefs.h | 4 +- qapi/Makefile.objs | 4 +- qapi/qapi-visit-core.c | 80 ++ qapi/qemu-file-ber-input-visitor.c | 1163 ++++++++++++++++++++++++ qapi/qemu-file-ber-output-visitor.c | 916 +++++++++++++++++++ qapi/qemu-file-binary-input-visitor.c | 688 ++++++++++++++ qapi/qemu-file-binary-output-visitor.c | 564 ++++++++++++ qapi/qemu-file-debug-output-visitor.c | 471 ++++++++++ qemu-file.c | 423 +++++++++ savevm.c | 412 +++++++-- vmstate.c | 619 ++++++------- 31 files changed, 5593 insertions(+), 558 deletions(-) create mode 100644 docs/specs/migration.schema create mode 100644 include/qapi/ber.h create mode 100644 include/qapi/qemu-file-ber-input-visitor.h create mode 100644 include/qapi/qemu-file-ber-output-visitor.h create mode 100644 include/qapi/qemu-file-binary-input-visitor.h create mode 100644 include/qapi/qemu-file-binary-output-visitor.h create mode 100644 include/qapi/qemu-file-debug-output-visitor.h create mode 100644 qapi/qemu-file-ber-input-visitor.c create mode 100644 qapi/qemu-file-ber-output-visitor.c create mode 100644 qapi/qemu-file-binary-input-visitor.c create mode 100644 qapi/qemu-file-binary-output-visitor.c create mode 100644 qapi/qemu-file-debug-output-visitor.c -- 1.8.5.3