Hi Eric Thanks for replying the mail, replied in lines. BR, Eli(Li Yong)Qiao
-----Original Message----- From: Eric Blake [mailto:ebl...@redhat.com] Sent: Tuesday, June 02, 2015 8:21 PM To: Feng, Shaohe; libvir-list@redhat.com Cc: Qiao, Liyong; Bhandaru, Malini K; Ding, Jian-feng; Chylinski, Arek; Koniszewski, Pawel; Li, Liang Z; berra...@redhat.com; veill...@redhat.com Subject: Re: [RFC] libvirt support multi-thread compression for live migration [correcting list address: libvirt-list, not libver-list] On 06/02/2015 05:58 AM, Feng, Shaohe wrote: > Hi folks: > I'd like to request some comments on enabling multi-thread compression in > libvirt. > > Currently, qemu upstream has supported multi-thread compression for live > migration. > $ git log 263170e~1..362ba4e -oneline > This can preserve bandwidth and speed up the live migration process, > so this is an import feature we need to enable so for other high level such > as openstack will be benefit. > > We plan to add feature of multi-thread compression and actually the patch is > working in process. > > Now some things need for comments. > > 1. Expose new set/get multi-thread compression parameters API for live > migration. > Now qemu only supports zlib compression. Maybe LZ4 and gzip will be > supported later. > but there are 3 parameters needed by qemu currently. > "compress-level": compression level, default is 1. For zlib compression, it > is from 0-9, 0 means use default level, 9 means high compressed. > "compress-threads": compression thread count for multi-thread migration, > default is 8 in qemu. > "decompress-threads": decompression thread count for multi-thread > migration, default is 2 in qemu > > So in libvirt we will expose the public symbols as follow, we only 3 > parameters, missing compression method(zlib, LZ4 and gzip) parameters, for > qemu do not support it at present. > index d851225..103b3b9 100644 > --- a/include/libvirt/libvirt-domain.h > +++ b/include/libvirt/libvirt-domain.h > @@ -798,6 +798,17 @@ int virDomainMigrateGetMaxSpeed(virDomainPtr domain, > unsigned long *bandwidth, > unsigned int flags); > > +int virdomainMigrateSetParameters(virDomainPtr domain, > + int level, > + int threads, > + int dthreads, > + int flags) int > +virdomainMigrateGetParameters(virDomainPtr domain, > + int *level, > + int *threads, > + int *dthreads, > + int flags) > + I'd rather we used virTypedParameters, to make it easier to use the same API to pass ALL future possible tuning parameters, rather than just hard-coding to only the parameters of this one feature. Okay ,that sound good, but if virTypedParameters can be passed to qemu_driver such as qemu_monitor_json.c qemu_monitor_text.c ? > > For virdomainMigrateSetParameters, if specifying a negative value to > level, threads, and dthreads parameters, such as -1, means do not set the > parameters. > The underlying code will not pass the corresponding parameters to qemu > monitor. > Such as threads and dthreads is -1, then pass the following json streaming to > qemu. > { "execute": "migrate-set-parameters" , "arguments": { > "compress-level": 1 } } > > The virsh will expose the two commands: > # virsh migrate-setparameters <domain> [--level level] [--threads > threads] [--dthreads dthread] # virsh migrate-getparameters <domain> > > 2. How to enable multi-thread compression in application interface? > There is not a special interface for setting migration capabilities. > But we can set the capabilities when execute an virsh command as > following: > $ migrate --live] [--offline] [--p2p] [--direct] [--tunnelled] > [--persistent] [--undefinesource] [--suspend] [--copy-storage-all] > [--copy-storage-inc] [--change-protection] [--unsafe] [--verbose] > [--compressed] [--abort-on-error] <domain> <desturi> [<migrateuri>] > [<graphicsuri>] [<listen-address>] [<dname>] [--timeout <number>] > [--xml <string>] > > There is already a "compressed" option, here "compressed" means "xbzrle" not > "multi- compressed". > can I rename the "compressed" to "xbzrle"? If we think that it is worth always turning on both compression styles simultaneously, then reuse the bit. Otherwise, we need a different bit, and users can choose which (or both) of the two compression styles as desired. +1 for reuse compressed, and we can set compress-level, compress-threads, compress-dthreads by virdomainMigrateSetParameters(maybe some new virsh command--- migrate-setparameter) But how can we passing these parameter when we using `virsh migrate `, is there any parameter we can use in 'virsh migrate' command ? Can you point me out ? > so we add another option for multi-thread compression, which is better option > name? "-multi- compressed" ? > Any way this is confuse to user. We need to > distinguish<http://dict.youdao.com/w/distinguish/> these two. > > > So I wonder should we expose new API to enable the migrate multi-thread > compress. > +int virdomainMigrateEnableMultiThreadCompress (virDomainPtr domain, > +int flags) > It will pass the following command to qemu monitor. > { "execute": "migrate-set-capabilities" , "arguments": { > "capabilities": [ { "capability": " compress", "state": true } ] } } > > NOTE: Now, multiple thread compression can co-work with xbzrle. When xbzrle > is on, multiple thread compression will only work at the first round of RAM > data sync. > Qemu, $ git show 98f1138902195bd9ab8a753d0ee2cf2a0a88b6e8 > if compress and xbzrle are both on, compress only takes effect in the ram > bulk stage, after that, it will be disabled and only xbzrle takes effect, > this can help to minimize migration traffic. > > 3. Migration between different livirt/qemu version > We can support to expose 2 new API to set/get migrate capabilities. > + virdomainMigrateSetCapabilities > + virdomainMigrateGetCapabilities > And we can suggest the user application to probe the capabilities before > execute live migration in our doc. > This need discussion is it necessary support these two in libvirt? > > Without the above GetCapabilities API, user application can probe > multi-thread compress capabilities by virdomainMigrateGetParameters. > > Or > return error directly when user application execute live migration with > multi-thread flag specifying, but do not any Capabilities probe. > > > INFO: qmeu command > {"execute": "query-migrate-capabilities"} > {"return": [{"state": false, "capability": "xbzrle"}, {"state": > false, "capability": "rdma-pin-all"}, {"state": false, "capability": > "auto-converge"}, {"state": false, "capability": "zero-blocks"}, > {"state": false, "capability": "compress"}]} > > > -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
