On Mon, Aug 08, 2022 at 11:41:21AM +0530, Het Gala wrote: > > On 02/08/22 1:23 pm, Markus Armbruster wrote: > > Het Gala<het.g...@nutanix.com> writes: > > > > > On 26/07/22 4:43 pm, Daniel P. Berrangé wrote: > > > > On Thu, Jul 21, 2022 at 07:56:15PM +0000, Het Gala wrote: > > > > > i) Modified the format of the qemu monitor command : 'migrate' by > > > > > adding a list, > > > > > each element in the list consisting of multifd connection > > > > > parameters: source > > > > > uri, destination uri and of the number of multifd channels > > > > > between each pair. > > > > > > > > > > ii) Information of all multifd connection parameters' list and length > > > > > of the > > > > > list is stored in 'OutgoingMigrateParams' struct. > > > > > > > > > > Suggested-by: Manish Mishra<manish.mis...@nutanix.com> > > > > > Signed-off-by: Het Gala<het.g...@nutanix.com> > > > > > --- > > > > > migration/migration.c | 52 +++++++++++++++++++++++++++++-------- > > > > > migration/socket.c | 60 > > > > > ++++++++++++++++++++++++++++++++++++++++--- > > > > > migration/socket.h | 19 +++++++++++++- > > > > > monitor/hmp-cmds.c | 1 + > > > > > qapi/migration.json | 47 +++++++++++++++++++++++++++++---- > > > > > 5 files changed, 160 insertions(+), 19 deletions(-) > > > > > > > > > > diff --git a/qapi/migration.json b/qapi/migration.json > > > > > index 81185d4311..456247af8f 100644 > > > > > --- a/qapi/migration.json > > > > > +++ b/qapi/migration.json > > > > > @@ -1449,12 +1449,37 @@ > > > > > ## > > > > > { 'command': 'migrate-continue', 'data': {'state': > > > > > 'MigrationStatus'} } > > > > > +## > > > > > +# @MigrateUriParameter: > > > > > +# > > > > > +# Information regarding which source interface is connected to which > > > > > +# destination interface and number of multifd channels over each > > > > > interface. > > > > > +# > > > > > +# @source-uri: uri of the source VM. Default port number is 0. > > > > > +# > > > > > +# @destination-uri: uri of the destination VM > > > > > +# > > > > > +# @multifd-channels: number of parallel multifd channels used to > > > > > migrate data > > > > > +# for specific source-uri and destination-uri. > > > > > Default value > > > > > +# in this case is 2 (Since 7.1) > > > > > +# > > > > > +## > > > > > +{ 'struct' : 'MigrateUriParameter', > > > > > + 'data' : { 'source-uri' : 'str', > > > > > + 'destination-uri' : 'str', > > > > > + '*multifd-channels' : 'uint8'} } > > > > > + > > > > > ## > > > > > # @migrate: > > > > > # > > > > > # Migrates the current running guest to another Virtual Machine. > > > > > # > > > > > # @uri: the Uniform Resource Identifier of the destination VM > > > > > +# for migration thread > > > > > +# > > > > > +# @multi-fd-uri-list: list of pair of source and destination VM > > > > > Uniform > > > > > +# Resource Identifiers with number of > > > > > multifd-channels > > > > > +# for each pair > > > > > # > > > > > # @blk: do block migration (full disk copy) > > > > > # > > > > > @@ -1474,20 +1499,32 @@ > > > > > # 1. The 'query-migrate' command should be used to check > > > > > migration's progress > > > > > # and final result (this information is provided by the > > > > > 'status' member) > > > > > # > > > > > -# 2. All boolean arguments default to false > > > > > +# 2. The uri argument should have the Uniform Resource Identifier of > > > > > default > > > > > +# destination VM. This connection will be bound to default network > > > > > # > > > > > -# 3. The user Monitor's "detach" argument is invalid in QMP and > > > > > should not > > > > > +# 3. All boolean arguments default to false > > > > > +# > > > > > +# 4. The user Monitor's "detach" argument is invalid in QMP and > > > > > should not > > > > > # be used > > > > > # > > > > > # Example: > > > > > # > > > > > -# -> { "execute": "migrate", "arguments": { "uri": "tcp:0:4446" } } > > > > > +# -> { "execute": "migrate", > > > > > +# "arguments": { > > > > > +# "uri": "tcp:0:4446", > > > > > +# "multi-fd-uri-list": [ { "source-uri": "tcp::6900", > > > > > +# "destination-uri": "tcp:0:4480", > > > > > +# "multifd-channels": 4}, > > > > > +# { "source-uri": "tcp:10.0.0.0: ", > > > > > +# "destination-uri": > > > > > "tcp:11.0.0.0:7789", > > > > > +# "multifd-channels": 5} ] } } > > > > > # <- { "return": {} } > > > > > # > > > > > ## > > > > > { 'command': 'migrate', > > > > > - 'data': {'uri': 'str', '*blk': 'bool', '*inc': 'bool', > > > > > - '*detach': 'bool', '*resume': 'bool' } } > > > > > + 'data': {'uri': 'str', '*multi-fd-uri-list': > > > > > ['MigrateUriParameter'], > > > > > + '*blk': 'bool', '*inc': 'bool', '*detach': 'bool', > > > > > + '*resume': 'bool' } } > > > > Considering the existing migrate API from a QAPI design POV, I > > > > think there are several significant flaws with it > > > > > > > > The use of URIs is the big red flag. It is basically a data encoding > > > > scheme within a data encoding scheme. QEMU code should be able to > > > > directly work with the results from QAPI, without having todo a > > > > second level of parsing. > > Concur. > > > > > > URIs made sense in the context of HMP or the QemuOpts CLI, but do not > > > > make sense in QMP. We made a mistake in this respect when we first > > > > introduced QMP and implemented 'migrate'. > > > > > > > > If we going to extend the migrate API I think we should stop using URIs > > > > for the new fields, and instead define a QAPI discriminated union for > > > > the different data transport backends we offer. > > > > > > > > { 'enum': 'MigrateTransport', > > > > 'data': ['socket', 'exec'] } > > > > > > > > { 'union': 'MigrateAddress', > > > > 'base': { 'transport': 'MigrateTransport'}, > > > > 'discriminator': 'transport', > > > > 'data': { > > > > 'socket': 'SocketAddress', > > > > 'exec': ['str'], > > > > } > > > > > > > > NB, 'socket' should be able to cover all of 'tcp', 'unix', 'vsock' > > > > and 'fd' already. I'm fuzzy on best way to represent RDMA. > > > > > > > > > > > > IIUC, the desire of migration maintainers is that we can ultimately > > > > have multifd as the preferred, or even only, mechanism. Aside from > > > > the main outbound migration control channel, and the multifd > > > > data channels, IIUC we have a potential desire to have more channels > > > > for post-copy async requests. > > > > > > > > This all suggests to me a more general representation along the > > > > lines of: > > > > > > > > { 'enum': 'MigrateChannelType', > > > > 'data': ['control', 'data', 'async'] } > > > > > > > > { 'struct': 'MigrateChannel', > > > > 'data': { > > > > 'type': 'MigrateChannelType', > > > > 'src-addr': 'MigrateAddress', > > > > 'dst-addr': 'MigrateAddress', > > > > 'count': 'int', > > > > } } > > > > > > > > { 'comand': 'migrate', > > > > 'data': { > > > > '*uri': 'str' > > > > '*channels': ['MigrateChannel'] > > > > } > > > > } > > > > > > > > With 'uri' and 'channels' being mutually exclusive here. > > > > > > > > This whole scheme brings in redundancy wrt to the > > > > 'migrate-set-parameters' > > > > API wrt multifd - essentally the same data is now being set in two > > > > different places. IMHO, we should declare the 'multifd' capability > > > > and the 'multifd-chanels' parameter deprecated, since the information > > > > they provide is totally redundant, if you're giving an explicit list > > > > of channels to 'migrate'. > > > Hi Daniel. Initially while brainstorming this idea for the first time, we > > > also came up with the same thought of depricating the migrate > > > API. But how will we achieve this now and how is it going to work. Is it > > > like we will be making migate V2 APIs initially, integrate it and then > > > depricate the old one? would be happy to get some pointers from your end. > > Migration maintainers, please advise. > > Hi Daniel and other migraton maintainers: Dr. David and Juan, what > is your opinion on this. And how can we go forward implementing > this. Some pointers and ideas from your end would be helpful too.
Note, i'm not a migration maintainer, just offering advice from the sidelines :-) With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|