Re: [PATCH for-8.2 3/4] migration/qapi: Replace @MigrateSetParameters with @MigrationParameters
On Mon, Aug 14, 2023 at 06:19:46PM -0400, Peter Xu wrote: > Here to deduplicate the two objects, logically it'll be safe only if we use > "StrOrNull" to replace "str" type, not vice versa. However we may face > difficulty using StrOrNull as part of MigrationState.parameters [1] when > replacing existing @MigrationParameters to use StrOrNull. With the fact > that nobody seems to be using "null" for tls-* fields (see the long > standing qemu crash bug on tls-authz when "null" was passed in), let's use > "str" to represent both objects. > > This greatly deduplicates the code not only in qapi/migration.json, but > also in the generic migration code on handling transitions between > StrOrNull <-> str types. > > [1] https://lore.kernel.org/all/ZNKfoqM0V6pcvrz%2F@x1n/ Markus helped me to work out this problem. I'll send a new version soon to switch to StrOrNull for all tls* fields. Thanks, -- Peter Xu
Re: [PATCH for-8.2 3/4] migration/qapi: Replace @MigrateSetParameters with @MigrationParameters
On Wed, Aug 23, 2023 at 03:26:25PM +0200, Markus Armbruster wrote:
> Peter Xu writes:
>
> > These two structs are mostly identical besides some fields (quote from
> > Daniel P. Berrangé in his reply):
> >
> > 1c1
> > < { 'struct': 'MigrationParameters',
> > ---
> > > { 'struct': 'MigrateSetParameters',
> > 14,16c14,16
> > < '*tls-creds': 'str',
> > < '*tls-hostname': 'str',
> > < '*tls-authz': 'str',
> > ---
> > > '*tls-creds': 'StrOrNull',
> > > '*tls-hostname': 'StrOrNull',
> > > '*tls-authz': 'StrOrNull',
> >
> > Here the difference is @MigrateSetParameters object would allow 'null'
> > values for any tls-* fields passed in.
> >
> > Markus used to describe why it happened to be StrOrNull, and also his
> > concern on having a pure "str" type to be problematic as the reset
> > indicator in the commit 01fa559826 ("migration: Use JSON null instead of ""
> > to reset parameter to default"). There, "null" is introduced for the tls
> > fields even though being treated as "" (empty string) internally to match
> > the code.
>
> Suggest
>
> migration/qapi: Replace @MigrateSetParameters with @MigrationParameters
>
> migrate-set-parameters sets migration parameters, and
> query-migrate-parameters gets them. Unsurprisingly, the former's
> argument type MigrateSetParameters is quite close to the latter's
> return type MigrationParameters. The differences are subtle:
>
> 1. Since migrate-set-parameters supports setting selected parameters,
> its arguments must all be optional (so you can omit the ones you
> don't want to change). query-migrate-parameters results are also
> all optional, but almost all of them are in fact always present.
>
> 2. For parameters @tls_creds, @tls_hostname, @tls_authz,
> migrate-set-parameters interprets special value "" as "reset to
> default". Works, because "" is semantically invalid. Not a
> general solution, because a semantically invalid value need not
> exist. Markus added a general solution in commit 01fa559826
> ("migration: Use JSON null instead of "" to reset parameter to
> default"). This involved changing the type from 'str' to
> 'StrOrNull'.
>
> 3. When parameter @block-bitmap-mapping has not been set,
> query-migrate-parameters does not return it (absent optional
> member). Clean (but undocumented). When parameters @tls_creds,
> @tls_hostname, @tls_authz have not been set, it returns the
> semantically invalid value "". Not so clean (and just as
> undocumented).
>
> Items 2. and 3. need fact-checking.
For 2: "reset to default" is correct, assuming the default is not enabling
TLS. Or perhaps we can also say "TLS disabled" to be clear?
Currently in the code we rely on the string non-null (of tls-creds) to
detect whether tls is enabled in general for migration:
bool migrate_tls(void)
{
MigrationState *s = migrate_get_current();
return s->parameters.tls_creds && *s->parameters.tls_creds;
}
Definitely not as clean (which I agree..), but we probably need to keep it
working anyway.
>
> > Here to deduplicate the two objects, logically it'll be safe only if we use
> > "StrOrNull" to replace "str" type, not vice versa. However we may face
> > difficulty using StrOrNull as part of MigrationState.parameters [1] when
> > replacing existing @MigrationParameters to use StrOrNull. With the fact
> > that nobody seems to be using "null" for tls-* fields (see the long
> > standing qemu crash bug on tls-authz when "null" was passed in), let's use
> > "str" to represent both objects.
>
> "May face difficulty" is insufficiently strong to justify such
> incompatible change.
>
> I'll have a look at the difficulties you mentioned in [1]. If we can
> overcome them with reasonable effort, we can and should avoid the
> compatibility break. If we can't, we add proper rationale here.
Thanks, Markus. Just in case of anything helpful, I've pushed the other
version here:
https://github.com/xzpeter/qemu/tree/mig-param-dedup-strornull
--
Peter Xu
Re: [PATCH for-8.2 3/4] migration/qapi: Replace @MigrateSetParameters with @MigrationParameters
Peter Xu writes:
> These two structs are mostly identical besides some fields (quote from
> Daniel P. Berrangé in his reply):
>
> 1c1
> < { 'struct': 'MigrationParameters',
> ---
> > { 'struct': 'MigrateSetParameters',
> 14,16c14,16
> < '*tls-creds': 'str',
> < '*tls-hostname': 'str',
> < '*tls-authz': 'str',
> ---
> > '*tls-creds': 'StrOrNull',
> > '*tls-hostname': 'StrOrNull',
> > '*tls-authz': 'StrOrNull',
>
> Here the difference is @MigrateSetParameters object would allow 'null'
> values for any tls-* fields passed in.
>
> Markus used to describe why it happened to be StrOrNull, and also his
> concern on having a pure "str" type to be problematic as the reset
> indicator in the commit 01fa559826 ("migration: Use JSON null instead of ""
> to reset parameter to default"). There, "null" is introduced for the tls
> fields even though being treated as "" (empty string) internally to match
> the code.
Suggest
migration/qapi: Replace @MigrateSetParameters with @MigrationParameters
migrate-set-parameters sets migration parameters, and
query-migrate-parameters gets them. Unsurprisingly, the former's
argument type MigrateSetParameters is quite close to the latter's
return type MigrationParameters. The differences are subtle:
1. Since migrate-set-parameters supports setting selected parameters,
its arguments must all be optional (so you can omit the ones you
don't want to change). query-migrate-parameters results are also
all optional, but almost all of them are in fact always present.
2. For parameters @tls_creds, @tls_hostname, @tls_authz,
migrate-set-parameters interprets special value "" as "reset to
default". Works, because "" is semantically invalid. Not a
general solution, because a semantically invalid value need not
exist. Markus added a general solution in commit 01fa559826
("migration: Use JSON null instead of "" to reset parameter to
default"). This involved changing the type from 'str' to
'StrOrNull'.
3. When parameter @block-bitmap-mapping has not been set,
query-migrate-parameters does not return it (absent optional
member). Clean (but undocumented). When parameters @tls_creds,
@tls_hostname, @tls_authz have not been set, it returns the
semantically invalid value "". Not so clean (and just as
undocumented).
Items 2. and 3. need fact-checking.
> Here to deduplicate the two objects, logically it'll be safe only if we use
> "StrOrNull" to replace "str" type, not vice versa. However we may face
> difficulty using StrOrNull as part of MigrationState.parameters [1] when
> replacing existing @MigrationParameters to use StrOrNull. With the fact
> that nobody seems to be using "null" for tls-* fields (see the long
> standing qemu crash bug on tls-authz when "null" was passed in), let's use
> "str" to represent both objects.
"May face difficulty" is insufficiently strong to justify such
incompatible change.
I'll have a look at the difficulties you mentioned in [1]. If we can
overcome them with reasonable effort, we can and should avoid the
compatibility break. If we can't, we add proper rationale here.
> This greatly deduplicates the code not only in qapi/migration.json, but
> also in the generic migration code on handling transitions between
> StrOrNull <-> str types.
>
> [1] https://lore.kernel.org/all/ZNKfoqM0V6pcvrz%2F@x1n/
>
> Signed-off-by: Peter Xu
[PATCH for-8.2 3/4] migration/qapi: Replace @MigrateSetParameters with @MigrationParameters
These two structs are mostly identical besides some fields (quote from
Daniel P. Berrangé in his reply):
1c1
< { 'struct': 'MigrationParameters',
---
> { 'struct': 'MigrateSetParameters',
14,16c14,16
< '*tls-creds': 'str',
< '*tls-hostname': 'str',
< '*tls-authz': 'str',
---
> '*tls-creds': 'StrOrNull',
> '*tls-hostname': 'StrOrNull',
> '*tls-authz': 'StrOrNull',
Here the difference is @MigrateSetParameters object would allow 'null'
values for any tls-* fields passed in.
Markus used to describe why it happened to be StrOrNull, and also his
concern on having a pure "str" type to be problematic as the reset
indicator in the commit 01fa559826 ("migration: Use JSON null instead of ""
to reset parameter to default"). There, "null" is introduced for the tls
fields even though being treated as "" (empty string) internally to match
the code.
Here to deduplicate the two objects, logically it'll be safe only if we use
"StrOrNull" to replace "str" type, not vice versa. However we may face
difficulty using StrOrNull as part of MigrationState.parameters [1] when
replacing existing @MigrationParameters to use StrOrNull. With the fact
that nobody seems to be using "null" for tls-* fields (see the long
standing qemu crash bug on tls-authz when "null" was passed in), let's use
"str" to represent both objects.
This greatly deduplicates the code not only in qapi/migration.json, but
also in the generic migration code on handling transitions between
StrOrNull <-> str types.
[1] https://lore.kernel.org/all/ZNKfoqM0V6pcvrz%2F@x1n/
Signed-off-by: Peter Xu
---
qapi/migration.json| 185 +
migration/migration-hmp-cmds.c | 16 +--
migration/options.c| 145 ++
3 files changed, 12 insertions(+), 334 deletions(-)
diff --git a/qapi/migration.json b/qapi/migration.json
index 8843e74b59..0416da65b5 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -851,189 +851,6 @@
{ 'name': 'x-vcpu-dirty-limit-period', 'features': ['unstable'] },
'vcpu-dirty-limit'] }
-##
-# @MigrateSetParameters:
-#
-# @announce-initial: Initial delay (in milliseconds) before sending
-# the first announce (Since 4.0)
-#
-# @announce-max: Maximum delay (in milliseconds) between packets in
-# the announcement (Since 4.0)
-#
-# @announce-rounds: Number of self-announce packets sent after
-# migration (Since 4.0)
-#
-# @announce-step: Increase in delay (in milliseconds) between
-# subsequent packets in the announcement (Since 4.0)
-#
-# @compress-level: compression level
-#
-# @compress-threads: compression thread count
-#
-# @compress-wait-thread: Controls behavior when all compression
-# threads are currently busy. If true (default), wait for a free
-# compression thread to become available; otherwise, send the page
-# uncompressed. (Since 3.1)
-#
-# @decompress-threads: decompression thread count
-#
-# @throttle-trigger-threshold: The ratio of bytes_dirty_period and
-# bytes_xfer_period to trigger throttling. It is expressed as
-# percentage. The default value is 50. (Since 5.0)
-#
-# @cpu-throttle-initial: Initial percentage of time guest cpus are
-# throttled when migration auto-converge is activated. The
-# default value is 20. (Since 2.7)
-#
-# @cpu-throttle-increment: throttle percentage increase each time
-# auto-converge detects that migration is not making progress.
-# The default value is 10. (Since 2.7)
-#
-# @cpu-throttle-tailslow: Make CPU throttling slower at tail stage At
-# the tail stage of throttling, the Guest is very sensitive to CPU
-# percentage while the @cpu-throttle -increment is excessive
-# usually at tail stage. If this parameter is true, we will
-# compute the ideal CPU percentage used by the Guest, which may
-# exactly make the dirty rate match the dirty rate threshold.
-# Then we will choose a smaller throttle increment between the one
-# specified by @cpu-throttle-increment and the one generated by
-# ideal CPU percentage. Therefore, it is compatible to
-# traditional throttling, meanwhile the throttle increment won't
-# be excessive at tail stage. The default value is false. (Since
-# 5.1)
-#
-# @tls-creds: ID of the 'tls-creds' object that provides credentials
-# for establishing a TLS connection over the migration data
-# channel. On the outgoing side of the migration, the credentials
-# must be for a 'client' endpoint, while for the incoming side the
-# credentials must be for a 'server' endpoint. Setting this to a
-# non-empty string enables TLS for all migrations. An empty
-# string means that QEMU will use plain text mode for migration,
-# rather than TLS (Since 2.9) Previously (since 2.7), this was
-# reported by omitting tls-creds instead.
-#
-# @tls-hostname: hostname
