Re: [PATCH v5 4/8] multifd: Add multifd-zlib-level parameter
Daniel P. Berrangé wrote:
> On Thu, Jan 30, 2020 at 09:03:00AM +0100, Markus Armbruster wrote:
>> Juan Quintela writes:
>>
>> > It will indicate which level use for compression.
>> >
>> > Signed-off-by: Juan Quintela
>>
>> This is slightly confusing (there is no zlib compression), unless you
>> peek at the next patch (which adds zlib compression).
>>
>> Three ways to make it less confusing:
>>
>> * Squash the two commits
>>
>> * Swap them: first add zlib compression with level hardcoded to 1, then
>> make the level configurable.
>>
>> * Have the first commit explain itself better. Something like
>>
>> multifd: Add multifd-zlib-level parameter
>>
>> This parameter specifies zlib compression level. The next patch
>> will put it to use.
>
> Wouldn't the "normal" best practice for QAPI design be to use a
> enum and discriminated union. eg
>
> { 'enum': 'MigrationCompression',
> 'data': ['none', 'zlib'] }
>
> { 'struct': 'MigrationCompressionParamsZLib',
> 'data': { 'compression-level' } }
>
> { 'union': 'MigrationCompressionParams',
> 'base': { 'mode': 'MigrationCompression' },
> 'discriminator': 'mode',
> 'data': {
> 'zlib': 'MigrationCompressionParamsZLib',
> }
How is this translate into HMP?
Markus says to start over, so lets see the dependencies:
Announce: Allawys there
announce-initial
announce-max
announce-rounds
announce-step
Osd compression (deprecated)
compress-level
compress-threads
compress-wait-thread
decompress-threads
cpu-throttles-initial
cpu-throottle-incroment
max-cpu-throotle
tls-creds
tls-hostname
tls-auth
Real params
max-bandwidth
downtime-limit
colo
x-checkpoint-delay
block-incremental
multifd-channels
xbzrle-cache-size
max-postcopy-bandwidth
New things:
- multifd method
- multifd-zlib-level
- multifd-zstd-level
What is a good way to define them?
Why do I ask, because the current method is as bad as it can be.
To add a new parameter:
- for qapi, add it in three places (as Markus said)
- go to hmp-cmds.c and do things by hand
- qemu_migrate_set_parameters
- migrate_params_check
- migrate_params_apply
(last three functions are almost identical in structure, not in
content).
So, if you can give me something that is _easier_ of maintaining, I am
all ears.
Later, Juan.
>
> Of course this is quite different from how migration parameters are
> done today. Maybe it makes sense to stick with the flat list of
> migration parameters for consistency & ignore normal QAPI design
> practice ?
>
>
> Regards,
> Daniel
Re: [PATCH v5 4/8] multifd: Add multifd-zlib-level parameter
Daniel P. Berrangé writes:
> On Thu, Jan 30, 2020 at 09:03:00AM +0100, Markus Armbruster wrote:
>> Juan Quintela writes:
>>
>> > It will indicate which level use for compression.
>> >
>> > Signed-off-by: Juan Quintela
>>
>> This is slightly confusing (there is no zlib compression), unless you
>> peek at the next patch (which adds zlib compression).
>>
>> Three ways to make it less confusing:
>>
>> * Squash the two commits
>>
>> * Swap them: first add zlib compression with level hardcoded to 1, then
>> make the level configurable.
>>
>> * Have the first commit explain itself better. Something like
>>
>> multifd: Add multifd-zlib-level parameter
>>
>> This parameter specifies zlib compression level. The next patch
>> will put it to use.
>
> Wouldn't the "normal" best practice for QAPI design be to use a
> enum and discriminated union. eg
>
> { 'enum': 'MigrationCompression',
> 'data': ['none', 'zlib'] }
>
> { 'struct': 'MigrationCompressionParamsZLib',
> 'data': { 'compression-level' } }
>
> { 'union': 'MigrationCompressionParams',
> 'base': { 'mode': 'MigrationCompression' },
> 'discriminator': 'mode',
> 'data': {
> 'zlib': 'MigrationCompressionParamsZLib',
> }
This expresses the connection between compression mode and level. In
general, we prefer that to a bunch of optional members where the
comments say things like "member A can be present only when member B has
value V", or worse "member A is silently ignored unless member B has
value V". However:
> Of course this is quite different from how migration parameters are
> done today. Maybe it makes sense to stick with the flat list of
> migration parameters for consistency & ignore normal QAPI design
> practice ?
Unsure. It's certainly ugly now. Each parameter is defined in three
places: enum MigrationParameter (for HMP), struct MigrateSetParameters
(for QMP migrate-set-parameters), and struct MigrationParameters (for
QMP query-migrate-parameters).
I don't know how to make this better other than by starting over.
I don't know whether starting over would result in enough of an
improvement to make it worthwhile.
Re: [PATCH v5 4/8] multifd: Add multifd-zlib-level parameter
On Thu, Jan 30, 2020 at 09:03:00AM +0100, Markus Armbruster wrote:
> Juan Quintela writes:
>
> > It will indicate which level use for compression.
> >
> > Signed-off-by: Juan Quintela
>
> This is slightly confusing (there is no zlib compression), unless you
> peek at the next patch (which adds zlib compression).
>
> Three ways to make it less confusing:
>
> * Squash the two commits
>
> * Swap them: first add zlib compression with level hardcoded to 1, then
> make the level configurable.
>
> * Have the first commit explain itself better. Something like
>
> multifd: Add multifd-zlib-level parameter
>
> This parameter specifies zlib compression level. The next patch
> will put it to use.
Wouldn't the "normal" best practice for QAPI design be to use a
enum and discriminated union. eg
{ 'enum': 'MigrationCompression',
'data': ['none', 'zlib'] }
{ 'struct': 'MigrationCompressionParamsZLib',
'data': { 'compression-level' } }
{ 'union': 'MigrationCompressionParams',
'base': { 'mode': 'MigrationCompression' },
'discriminator': 'mode',
'data': {
'zlib': 'MigrationCompressionParamsZLib',
}
Of course this is quite different from how migration parameters are
done today. Maybe it makes sense to stick with the flat list of
migration parameters for consistency & ignore normal QAPI design
practice ?
Regards,
Daniel
--
|: https://berrange.com -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
Re: [PATCH v5 4/8] multifd: Add multifd-zlib-level parameter
Markus Armbruster wrote: > Juan Quintela writes: > >> It will indicate which level use for compression. >> >> Signed-off-by: Juan Quintela > > This is slightly confusing (there is no zlib compression), unless you > peek at the next patch (which adds zlib compression). > > Three ways to make it less confusing: > > * Squash the two commits As a QAPI begginer, I feel it easier to put it in a different patch. It makes it also easier to add other parameters, just copy whatewer is here. > * Swap them: first add zlib compression with level hardcoded to 1, then > make the level configurable. That could work. > * Have the first commit explain itself better. Something like > > multifd: Add multifd-zlib-level parameter > > This parameter specifies zlib compression level. The next patch > will put it to use. Will take this approach. The reason that I put the qapi bits first is that I *know* how to do them. Once that I got approval for how to add one parameter, I add the rest exactly the same. For the rest of the patch, it needs to be developed, and normally needs more iterations. > > For QAPI: > Acked-by: Markus Armbruster Thanks very much. Later, Juan.
Re: [PATCH v5 4/8] multifd: Add multifd-zlib-level parameter
Juan Quintela writes: > It will indicate which level use for compression. > > Signed-off-by: Juan Quintela This is slightly confusing (there is no zlib compression), unless you peek at the next patch (which adds zlib compression). Three ways to make it less confusing: * Squash the two commits * Swap them: first add zlib compression with level hardcoded to 1, then make the level configurable. * Have the first commit explain itself better. Something like multifd: Add multifd-zlib-level parameter This parameter specifies zlib compression level. The next patch will put it to use. For QAPI: Acked-by: Markus Armbruster
[PATCH v5 4/8] multifd: Add multifd-zlib-level parameter
It will indicate which level use for compression.
Signed-off-by: Juan Quintela
---
migration/migration.c | 15 +++
monitor/hmp-cmds.c| 4
qapi/migration.json | 30 +++---
3 files changed, 46 insertions(+), 3 deletions(-)
diff --git a/migration/migration.c b/migration/migration.c
index 06f6c2d529..4f88f8e958 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -89,6 +89,8 @@
#define DEFAULT_MIGRATE_X_CHECKPOINT_DELAY (200 * 100)
#define DEFAULT_MIGRATE_MULTIFD_CHANNELS 2
#define DEFAULT_MIGRATE_MULTIFD_METHOD MULTIFD_METHOD_NONE
+/*0: means nocompress, 1: best speed, ... 9: best compress ratio */
+#define DEFAULT_MIGRATE_MULTIFD_ZLIB_LEVEL 1
/* Background transfer rate for postcopy, 0 means unlimited, note
* that page requests can still exceed this limit.
@@ -801,6 +803,8 @@ MigrationParameters *qmp_query_migrate_parameters(Error
**errp)
params->multifd_channels = s->parameters.multifd_channels;
params->has_multifd_method = true;
params->multifd_method = s->parameters.multifd_method;
+params->has_multifd_zlib_level = true;
+params->multifd_zlib_level = s->parameters.multifd_zlib_level;
params->has_xbzrle_cache_size = true;
params->xbzrle_cache_size = s->parameters.xbzrle_cache_size;
params->has_max_postcopy_bandwidth = true;
@@ -1208,6 +1212,13 @@ static bool migrate_params_check(MigrationParameters
*params, Error **errp)
return false;
}
+if (params->has_multifd_zlib_level &&
+(params->multifd_zlib_level > 9)) {
+error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "multifd_zlib_level",
+ "is invalid, it should be in the range of 0 to 9");
+return false;
+}
+
if (params->has_xbzrle_cache_size &&
(params->xbzrle_cache_size < qemu_target_page_size() ||
!is_power_of_2(params->xbzrle_cache_size))) {
@@ -3536,6 +3547,9 @@ static Property migration_properties[] = {
DEFINE_PROP_MULTIFD_METHOD("multifd-method", MigrationState,
parameters.multifd_method,
DEFAULT_MIGRATE_MULTIFD_METHOD),
+DEFINE_PROP_UINT8("multifd-zlib-level", MigrationState,
+ parameters.multifd_zlib_level,
+ DEFAULT_MIGRATE_MULTIFD_ZLIB_LEVEL),
DEFINE_PROP_SIZE("xbzrle-cache-size", MigrationState,
parameters.xbzrle_cache_size,
DEFAULT_MIGRATE_XBZRLE_CACHE_SIZE),
@@ -3627,6 +3641,7 @@ static void migration_instance_init(Object *obj)
params->has_block_incremental = true;
params->has_multifd_channels = true;
params->has_multifd_method = true;
+params->has_multifd_zlib_level = true;
params->has_xbzrle_cache_size = true;
params->has_max_postcopy_bandwidth = true;
params->has_max_cpu_throttle = true;
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
index 16f01d4244..7f11866446 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
@@ -1836,6 +1836,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict
*qdict)
}
p->multifd_method = compress_type;
break;
+case MIGRATION_PARAMETER_MULTIFD_ZLIB_LEVEL:
+p->has_multifd_zlib_level = true;
+visit_type_int(v, param, >multifd_zlib_level, );
+break;
case MIGRATION_PARAMETER_XBZRLE_CACHE_SIZE:
p->has_xbzrle_cache_size = true;
visit_type_size(v, param, _size, );
diff --git a/qapi/migration.json b/qapi/migration.json
index 96a126751c..289dce0da7 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -602,6 +602,13 @@
# @multifd-method: Which compression method to use.
# Defaults to none. (Since 5.0)
#
+# @multifd-zlib-level: Set the compression level to be used in live
+# migration, the compression level is an integer between 0
+# and 9, where 0 means no compression, 1 means the best
+# compression speed, and 9 means best compression ratio which
+# will consume more CPU.
+# Defaults to 1. (Since 5.0)
+#
# Since: 2.4
##
{ 'enum': 'MigrationParameter',
@@ -614,7 +621,8 @@
'downtime-limit', 'x-checkpoint-delay', 'block-incremental',
'multifd-channels',
'xbzrle-cache-size', 'max-postcopy-bandwidth',
- 'max-cpu-throttle', 'multifd-method' ] }
+ 'max-cpu-throttle', 'multifd-method',
+ 'multifd-zlib-level' ] }
##
# @MigrateSetParameters:
@@ -707,6 +715,13 @@
# @multifd-method: Which compression method to use.
# Defaults to none. (Since 5.0)
#
+# @multifd-zlib-level: Set the compression level to be used in live
+# migration, the compression level is an integer between 0
+# and 9, where 0 means no compression, 1 means the best
+# compression speed, and 9 means best compression ratio which
+# will consume more CPU.
+# Defaults to 1. (Since 5.0)
+#
# Since:
