[correcting list address: libvirt-list, not libver-list] On 06/02/2015 05:58 AM, Feng, Shaohe wrote: > Hi folks: > I'd like to request some comments on enabling multi-thread compression in > libvirt. > > Currently, qemu upstream has supported multi-thread compression for live > migration. > $ git log 263170e~1..362ba4e -oneline > This can preserve bandwidth and speed up the live migration process, so this > is an import > feature we need to enable so for other high level such as openstack will be > benefit. > > We plan to add feature of multi-thread compression and actually the patch is > working in process. > > Now some things need for comments. > > 1. Expose new set/get multi-thread compression parameters API for live > migration. > Now qemu only supports zlib compression. Maybe LZ4 and gzip will be > supported later. > but there are 3 parameters needed by qemu currently. > "compress-level": compression level, default is 1. For zlib compression, it > is from 0-9, 0 means use default level, 9 means high compressed. > "compress-threads": compression thread count for multi-thread migration, > default is 8 in qemu. > "decompress-threads": decompression thread count for multi-thread migration, > default is 2 in qemu > > So in libvirt we will expose the public symbols as follow, we only 3 > parameters, > missing compression method(zlib, LZ4 and gzip) parameters, for qemu do not > support it at present. > index d851225..103b3b9 100644 > --- a/include/libvirt/libvirt-domain.h > +++ b/include/libvirt/libvirt-domain.h > @@ -798,6 +798,17 @@ int virDomainMigrateGetMaxSpeed(virDomainPtr domain, > unsigned long *bandwidth, > unsigned int flags); > > +int virdomainMigrateSetParameters(virDomainPtr domain, > + int level, > + int threads, > + int dthreads, > + int flags) > +int virdomainMigrateGetParameters(virDomainPtr domain, > + int *level, > + int *threads, > + int *dthreads, > + int flags) > +
I'd rather we used virTypedParameters, to make it easier to use the same
API to pass ALL future possible tuning parameters, rather than just
hard-coding to only the parameters of this one feature.
>
> For virdomainMigrateSetParameters, if specifying a negative value to level,
> threads, and dthreads parameters, such as -1,
> means do not set the parameters.
> The underlying code will not pass the corresponding parameters to qemu
> monitor.
> Such as threads and dthreads is -1, then pass the following json streaming to
> qemu.
> { "execute": "migrate-set-parameters" , "arguments": { "compress-level": 1 }
> }
>
> The virsh will expose the two commands:
> # virsh migrate-setparameters <domain> [--level level] [--threads threads]
> [--dthreads dthread]
> # virsh migrate-getparameters <domain>
>
> 2. How to enable multi-thread compression in application interface?
> There is not a special interface for setting migration capabilities.
> But we can set the capabilities when execute an virsh command as
> following:
> $ migrate --live] [--offline] [--p2p] [--direct] [--tunnelled]
> [--persistent] [--undefinesource] [--suspend] [--copy-storage-all]
> [--copy-storage-inc] [--change-protection] [--unsafe] [--verbose]
> [--compressed] [--abort-on-error] <domain> <desturi> [<migrateuri>]
> [<graphicsuri>] [<listen-address>] [<dname>] [--timeout <number>] [--xml
> <string>]
>
> There is already a "compressed" option, here "compressed" means "xbzrle" not
> "multi- compressed".
> can I rename the "compressed" to "xbzrle"?
If we think that it is worth always turning on both compression styles
simultaneously, then reuse the bit. Otherwise, we need a different bit,
and users can choose which (or both) of the two compression styles as
desired.
> so we add another option for multi-thread compression, which is better option
> name? "-multi- compressed" ?
> Any way this is confuse to user. We need to
> distinguish<http://dict.youdao.com/w/distinguish/> these two.
>
>
> So I wonder should we expose new API to enable the migrate multi-thread
> compress.
> +int virdomainMigrateEnableMultiThreadCompress (virDomainPtr domain, int
> flags)
> It will pass the following command to qemu monitor.
> { "execute": "migrate-set-capabilities" , "arguments": { "capabilities": [
> { "capability": " compress", "state": true } ] } }
>
> NOTE: Now, multiple thread compression can co-work with xbzrle. When xbzrle
> is on, multiple thread compression will only work at the first round of RAM
> data sync.
> Qemu, $ git show 98f1138902195bd9ab8a753d0ee2cf2a0a88b6e8
> if compress and xbzrle are both on, compress only takes effect in the ram
> bulk stage, after that, it will be disabled and only xbzrle takes effect,
> this can help to minimize migration traffic.
>
> 3. Migration between different livirt/qemu version
> We can support to expose 2 new API to set/get migrate capabilities.
> + virdomainMigrateSetCapabilities
> + virdomainMigrateGetCapabilities
> And we can suggest the user application to probe the capabilities before
> execute live migration in our doc.
> This need discussion is it necessary support these two in libvirt?
>
> Without the above GetCapabilities API, user application can probe
> multi-thread compress capabilities by virdomainMigrateGetParameters.
>
> Or
> return error directly when user application execute live migration with
> multi-thread flag specifying, but do not any Capabilities probe.
>
>
> INFO: qmeu command
> {"execute": "query-migrate-capabilities"}
> {"return": [{"state": false, "capability": "xbzrle"}, {"state": false,
> "capability": "rdma-pin-all"}, {"state": false, "capability":
> "auto-converge"}, {"state": false, "capability": "zero-blocks"}, {"state":
> false, "capability": "compress"}]}
>
>
>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
signature.asc
Description: OpenPGP digital signature
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
