[Bug 1849644] Re: QEMU VNC websocket proxy requires non-standard 'binary' subprotocol

2020-09-24 Thread Christian Ehrhardt 
** Description changed: - When running a machine using "-vnc" and the "websocket" option QEMU - seems to require the subprotocol called 'binary'. This subprotocol does - not exist in the WebSocket specification. In fact it has never existed - in the spec, in one of the very early drafts of

Re: [PATCH v5 09/20] docs/sphinx: Add new qapi-doc Sphinx extension

2020-09-24 Thread Markus Armbruster
Peter Maydell writes: > On Mon, 21 Sep 2020 at 19:06, Peter Maydell wrote: >> (I've just noticed that with Sphinx 1.6, which we still have >> to support, the file/line info isn't passed through, so you get: >> >> Warning, treated as error: >>

Re: [PATCH 00/16] hw/block/nvme: zoned namespace command set

2020-09-24 Thread Klaus Jensen
On Sep 24 19:24, Keith Busch wrote: > On Thu, Sep 24, 2020 at 10:45:00PM +0200, Klaus Jensen wrote: > > Finally, I wrote this. I am *NOT* saying that this somehow makes it > > better, but as a maintainer, is a big deal to me since both series are > > arguably a lot of code to maintain and support

Re: [PATCH] input-linux: Reset il->fd handler before closing it

2020-09-24 Thread Li Qiang
Colin Xu 于2020年9月25日周五 上午10:18写道: > > If object-del input-linux object on-the-fly, instance finalize will > close evdev fd without resetting it. However the main thread is still > trying to lock_acquire/lock_release during ppoll, which leads to a very > high CPU utilization. > > Signed-off-by:

Re: [PATCH V9 6/6] hw/mips: Add Loongson-3 machine support

2020-09-24 Thread Huacai Chen
Hi, Philippe, On Thu, Sep 24, 2020 at 11:40 PM Philippe Mathieu-Daudé wrote: > > On 9/16/20 11:49 AM, Huacai Chen wrote: > > Hi, Philippe, > > > > On Wed, Sep 16, 2020 at 3:56 PM Philippe Mathieu-Daudé > > wrote: > >> > >> Hi Huacai, > >> > >> On 9/16/20 4:12 AM, Huacai Chen wrote: > ... > >>

Re: [PATCH v2 2/6] spapr_numa: forbid asymmetrical NUMA setups

2020-09-24 Thread David Gibson
On Thu, Sep 24, 2020 at 04:50:54PM -0300, Daniel Henrique Barboza wrote: > The pSeries machine does not support asymmetrical NUMA > configurations. This doesn't make much of a different > since we're not using user input for pSeries NUMA setup, > but this will change in the next patches. > > To

Re: [PATCH v2 5/6] spapr_numa: consider user input when defining associativity

2020-09-24 Thread David Gibson
On Thu, Sep 24, 2020 at 04:50:57PM -0300, Daniel Henrique Barboza wrote: > This patch puts all the pieces together to finally allow user > input when defining the NUMA topology of the spapr guest. > > We have one more kernel restriction to handle in this patch: > the associativity array of node 0

Re: [PATCH v2 4/6] spapr_numa: change reference-points and maxdomain settings

2020-09-24 Thread David Gibson
On Thu, Sep 24, 2020 at 04:50:56PM -0300, Daniel Henrique Barboza wrote: > This is the first guest visible change introduced in > spapr_numa.c. The previous settings of both reference-points > and maxdomains were too restrictive, but enough for the > existing associativity we're setting in the

Re: [PATCH v2 2/6] spapr_numa: forbid asymmetrical NUMA setups

2020-09-24 Thread David Gibson
On Thu, Sep 24, 2020 at 04:50:54PM -0300, Daniel Henrique Barboza wrote: > The pSeries machine does not support asymmetrical NUMA > configurations. This doesn't make much of a different > since we're not using user input for pSeries NUMA setup, > but this will change in the next patches. > > To

Re: [PATCH v2 6/6] specs/ppc-spapr-numa: update with new NUMA support

2020-09-24 Thread David Gibson
On Thu, Sep 24, 2020 at 04:50:58PM -0300, Daniel Henrique Barboza wrote: > This update provides more in depth information about the > choices and drawbacks of the new NUMA support for the > spapr machine. > > Signed-off-by: Daniel Henrique Barboza > --- > docs/specs/ppc-spapr-numa.rst | 213

Re: [PATCH v2 3/6] spapr_numa: translate regular NUMA distance to PAPR distance

2020-09-24 Thread David Gibson
On Thu, Sep 24, 2020 at 04:50:55PM -0300, Daniel Henrique Barboza wrote: > QEMU allows the user to set NUMA distances in the command line. > For ACPI architectures like x86, this means that user input is > used to populate the SLIT table, and the guest perceives the > distances as the user chooses

[PATCH v5] hw/i386/pc: add max combined fw size as machine configuration option

2020-09-24 Thread Erich Mcmillan
From: Erich McMillan At HPi we have a need for increased fw size to enable testing of our custom fw. Signed-off-by: Erich McMillan Change since v4: Add explicit return to pc_machine_set_max_fw_size. Remove /* default */ from max_fw_size initialization. --- hw/i386/pc.c |

[PATCH] input-linux: Reset il->fd handler before closing it

2020-09-24 Thread Colin Xu
If object-del input-linux object on-the-fly, instance finalize will close evdev fd without resetting it. However the main thread is still trying to lock_acquire/lock_release during ppoll, which leads to a very high CPU utilization. Signed-off-by: Colin Xu --- ui/input-linux.c | 1 + 1 file

[Bug 1897194] [NEW] Test failure in test-crypto-secret.c

2020-09-24 Thread Toolybird
Public bug reported: When running qemu test suite I'm seeing a test failure: ERROR:../qemu/tests/test-crypto-secret.c:144:test_secret_keyring_good: assertion failed: (key >= 0) Host is Arch Linux running in the standard Arch build environment (essentially an nspawn container). I first noticed

Re: [PATCH 16/16] qapi/expr.py: Use an expression checker dispatch table

2020-09-24 Thread John Snow
On 9/24/20 9:18 PM, Cleber Rosa wrote: I have to say the style of this line bothers me, but it's just that, style. So, What don't you like?

Re: [PATCH 04/16] qapi/expr.py: Add assertion for union type 'check_dict'

2020-09-24 Thread John Snow
On 9/24/20 9:08 PM, Eduardo Habkost wrote: On Thu, Sep 24, 2020 at 08:47:31PM -0400, John Snow wrote: On 9/23/20 3:53 PM, Eduardo Habkost wrote: On Tue, Sep 22, 2020 at 05:13:01PM -0400, John Snow wrote: mypy isn't fond of allowing you to check for bool membership in a collection of str

Re: [PATCH 14/16] qapi/expr.py: Use tuples instead of lists for static data

2020-09-24 Thread John Snow
On 9/24/20 9:03 PM, Cleber Rosa wrote: On Tue, Sep 22, 2020 at 05:13:11PM -0400, John Snow wrote: It is -- maybe -- possibly -- three nanoseconds faster. Kevin O'Leary would clearly say here: "stop the madness!" :) Signed-off-by: John Snow Reviewed-by: Cleber Rosa I can explain!

Re: [PATCH 00/16] hw/block/nvme: zoned namespace command set

2020-09-24 Thread Keith Busch
On Thu, Sep 24, 2020 at 10:45:00PM +0200, Klaus Jensen wrote: > Finally, I wrote this. I am *NOT* saying that this somehow makes it > better, but as a maintainer, is a big deal to me since both series are > arguably a lot of code to maintain and support (both series are about > the same size). But

Re: [PATCH 04/16] qapi/expr.py: Add assertion for union type 'check_dict'

2020-09-24 Thread Eduardo Habkost
On Thu, Sep 24, 2020 at 08:47:31PM -0400, John Snow wrote: > On 9/23/20 3:53 PM, Eduardo Habkost wrote: > > On Tue, Sep 22, 2020 at 05:13:01PM -0400, John Snow wrote: > > > mypy isn't fond of allowing you to check for bool membership in a > > > collection of str elements. Guard this lookup for

Re: [PATCH 12/16] qapi/expr.py: Add docstrings

2020-09-24 Thread Cleber Rosa
On Tue, Sep 22, 2020 at 05:13:09PM -0400, John Snow wrote: > Signed-off-by: John Snow > --- > scripts/qapi/expr.py | 157 ++- > 1 file changed, 155 insertions(+), 2 deletions(-) > > diff --git a/scripts/qapi/expr.py b/scripts/qapi/expr.py > index

Re: [PATCH 16/16] qapi/expr.py: Use an expression checker dispatch table

2020-09-24 Thread Cleber Rosa
On Tue, Sep 22, 2020 at 05:13:13PM -0400, John Snow wrote: > This enforces a type signature against all of the top-level expression > check routines without necessarily needing to create some > overcomplicated class hierarchy for them. > > Signed-off-by: John Snow > --- > scripts/qapi/expr.py |

Re: [PATCH 03/16] qapi/expr.py: constrain incoming expression types

2020-09-24 Thread John Snow
On 9/23/20 3:42 PM, Eduardo Habkost wrote: On Tue, Sep 22, 2020 at 05:13:00PM -0400, John Snow wrote: mypy does not know the types of values stored in Dicts that masquerade as objects. Help the type checker out by constraining the type. Signed-off-by: John Snow --- scripts/qapi/expr.py | 25

Re: [PATCH 16/16] qapi/expr.py: Use an expression checker dispatch table

2020-09-24 Thread John Snow
On 9/23/20 4:36 PM, Eduardo Habkost wrote: I see lots of meta.value expressions below. Maybe setting meta = kind.value will make the code more readable? I can do that. I don't think this should block an obvious improvement to the code, so: Reviewed-by: Eduardo Habkost Thanks! --js

Re: [PATCH 16/37] qapi: establish mypy type-checking baseline

2020-09-24 Thread Eduardo Habkost
On Mon, Sep 21, 2020 at 10:05:24AM +0200, Markus Armbruster wrote: [...] > I sense a certain wobbliness in mypy. Perhaps to be expected from a > tool with major version zero. There's a risk that developers' local > "make check" and our gating CI differ too much. We'll see. This possibility

Re: [PATCH v2 04/38] qapi: Prefer explicit relative imports

2020-09-24 Thread Beraldo Leal
On Thu, Sep 24, 2020 at 03:25:50PM -0400, Cleber Rosa wrote: > On Wed, Sep 23, 2020 at 01:12:09PM -0400, John Snow wrote: > > On 9/23/20 9:18 AM, Cleber Rosa wrote: > > > Relative imports are a source of heated debates, but when properly > > > used in a self contained module like here, they are

Re: [PATCH 15/16] qapi/expr.py: move related checks inside check_xxx functions

2020-09-24 Thread Cleber Rosa
On Tue, Sep 22, 2020 at 05:13:12PM -0400, John Snow wrote: > There's not a big obvious difference between the types of checks that > happen in the main function versus the kind that happen in the > functions. Now they're in one place for each of the main types. > > As part of the move, spell out

Re: tools/virtiofs: Multi threading seems to hurt performance

2020-09-24 Thread Venegas Munoz, Jose Carlos
Hi Folks, Sorry for the delay about how to reproduce `fio` data. I have some code to automate testing for multiple kata configs and collect info like: - Kata-env, kata configuration.toml, qemu command, virtiofsd command. See: https://github.com/jcvenegas/mrunner/ Last time we agreed to

Re: [PATCH 11/16] qapi/expr.py: enable pylint checks

2020-09-24 Thread Cleber Rosa
On Tue, Sep 22, 2020 at 05:13:08PM -0400, John Snow wrote: > Signed-off-by: John Snow Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa signature.asc Description: PGP signature

Re: [PATCH 12/16] qapi/expr.py: Add docstrings

2020-09-24 Thread John Snow
On 9/23/20 4:16 PM, Eduardo Habkost wrote: On Tue, Sep 22, 2020 at 05:13:09PM -0400, John Snow wrote: Signed-off-by: John Snow --- [...] @@ -160,6 +244,18 @@ def check_type(value: Optional[object], source: str, allow_array: bool = False,

Re: [PATCH 12/16] qapi/expr.py: Add docstrings

2020-09-24 Thread John Snow
On 9/24/20 8:59 PM, Cleber Rosa wrote: On Tue, Sep 22, 2020 at 05:13:09PM -0400, John Snow wrote: Signed-off-by: John Snow --- scripts/qapi/expr.py | 157 ++- 1 file changed, 155 insertions(+), 2 deletions(-) diff --git a/scripts/qapi/expr.py

Re: [PATCH 07/16] qapi/expr.py: Add casts in a few select cases

2020-09-24 Thread Cleber Rosa
On Tue, Sep 22, 2020 at 05:13:04PM -0400, John Snow wrote: > Casts are instructions to the type checker only, they aren't "safe" and > should probably be avoided in general. In this case, when we perform > type checking on a nested structure, the type of each field does not > "stick". > > We

Re: [PATCH 13/16] qapi/expr.py: Modify check_keys to accept any Iterable

2020-09-24 Thread Cleber Rosa
On Tue, Sep 22, 2020 at 05:13:10PM -0400, John Snow wrote: > This is a very minor adjustment. > > a + b is list-specific behavior, but we can accept a wider variety of > types in a more pythonic fashion if we avoid that behavior. > > Typing it this way allows callers to use things like

Re: [PATCH 06/16] qapi/expr.py: Check type of 'data' member

2020-09-24 Thread John Snow
On 9/24/20 8:31 PM, Cleber Rosa wrote: On Tue, Sep 22, 2020 at 05:13:03PM -0400, John Snow wrote: Iterating over the members of data isn't going to work if it's not some form of dict anyway, but for type safety, formalize it. Signed-off-by: John Snow --- scripts/qapi/expr.py | 7 +++ 1

[PATCH v3 42/47] qapi/introspect.py: create a typed 'Node' data structure

2020-09-24 Thread John Snow
This replaces _make_tree with Node.__init__(), effectively. By creating it as a generic container, we can more accurately describe the exact nature of this particular Node. Signed-off-by: John Snow --- scripts/qapi/introspect.py | 77 +++--- 1 file changed, 38

Re: [PATCH 15/16] qapi/expr.py: move related checks inside check_xxx functions

2020-09-24 Thread John Snow
On 9/23/20 4:25 PM, Eduardo Habkost wrote: On Tue, Sep 22, 2020 at 05:13:12PM -0400, John Snow wrote: There's not a big obvious difference between the types of checks that happen in the main function versus the kind that happen in the functions. Now they're in one place for each of the main

[PATCH v3 45/47] qapi/visit.py: assert tag_member contains a QAPISchemaEnumType

2020-09-24 Thread John Snow
This is true by design, but not presently able to be expressed in the type system. An assertion helps mypy understand our constraints. Signed-off-by: John Snow Reviewed-by: Eduardo Habkost Reviewed-by: Cleber Rosa --- scripts/qapi/visit.py | 12 +++- 1 file changed, 7 insertions(+), 5

Re: [PATCH 14/16] qapi/expr.py: Use tuples instead of lists for static data

2020-09-24 Thread Cleber Rosa
On Tue, Sep 22, 2020 at 05:13:11PM -0400, John Snow wrote: > It is -- maybe -- possibly -- three nanoseconds faster. > Kevin O'Leary would clearly say here: "stop the madness!" :) > Signed-off-by: John Snow Reviewed-by: Cleber Rosa signature.asc Description: PGP signature

Re: [PATCH 04/16] qapi/expr.py: Add assertion for union type 'check_dict'

2020-09-24 Thread John Snow
On 9/23/20 3:53 PM, Eduardo Habkost wrote: On Tue, Sep 22, 2020 at 05:13:01PM -0400, John Snow wrote: mypy isn't fond of allowing you to check for bool membership in a collection of str elements. Guard this lookup for precisely when we were given a name. Signed-off-by: John Snow ---

Re: [PATCH 10/16] qapi/expr.py: Remove single-letter variable

2020-09-24 Thread Cleber Rosa
On Tue, Sep 22, 2020 at 05:13:07PM -0400, John Snow wrote: > Signed-off-by: John Snow Reviewed-by: Cleber Rosa signature.asc Description: PGP signature

Re: [PATCH 09/16] qapi/expr.py: rewrite check_if

2020-09-24 Thread Cleber Rosa
On Tue, Sep 22, 2020 at 05:13:06PM -0400, John Snow wrote: > This is a only minor rewrite to address some minor style nits. Don't > compare against the empty list to check for the empty condition, and > move the normalization forward to unify the check on the now-normalized > structure. > >

[PATCH v3 43/47] qapi/types.py: add type hint annotations

2020-09-24 Thread John Snow
Annotations do not change runtime behavior. This commit *only* adds annotations. Signed-off-by: John Snow Reviewed-by: Eduardo Habkost Reviewed-by: Cleber Rosa --- scripts/qapi/mypy.ini | 5 --- scripts/qapi/types.py | 86 --- 2 files changed, 64

Re: [PATCH 08/16] qapi/expr.py: add type hint annotations

2020-09-24 Thread Cleber Rosa
On Tue, Sep 22, 2020 at 05:13:05PM -0400, John Snow wrote: > Annotations do not change runtime behavior. > This commit *only* adds annotations. > > Signed-off-by: John Snow Reviewed-by: Cleber Rosa signature.asc Description: PGP signature

Re: [PATCH 03/16] qapi/expr.py: constrain incoming expression types

2020-09-24 Thread John Snow
On 9/24/20 8:05 PM, Cleber Rosa wrote: On Wed, Sep 23, 2020 at 03:42:24PM -0400, Eduardo Habkost wrote: On Tue, Sep 22, 2020 at 05:13:00PM -0400, John Snow wrote: mypy does not know the types of values stored in Dicts that masquerade as objects. Help the type checker out by constraining the

[PATCH v3 44/47] qapi/types.py: remove one-letter variables

2020-09-24 Thread John Snow
"John, if pylint told you to jump off a bridge, would you?" Hey, if it looked like fun, I might. Now that this file is clean, enable pylint checks on this file. Signed-off-by: John Snow Reviewed-by: Eduardo Habkost Reviewed-by: Cleber Rosa --- scripts/qapi/pylintrc | 1 -

[PATCH v3 37/47] qapi/introspect.py: assert obj is a dict when features are given

2020-09-24 Thread John Snow
This is necessary to keep mypy passing in the next patch when we add preliminary type hints. It will be removed shortly. Signed-off-by: John Snow --- scripts/qapi/introspect.py | 1 + 1 file changed, 1 insertion(+) diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py index

[PATCH v3 40/47] qapi/introspect.py: Unify return type of _make_tree()

2020-09-24 Thread John Snow
Returning a *something* or a Tuple of *something* is hard to accurately type. Let's just always return a tuple for structural consistency. Instances of the 'TreeNode' type can be replaced with the slightly more specific 'AnnotatedNode' type. Signed-off-by: John Snow ---

[PATCH v3 29/47] qapi/source.py: add type hint annotations

2020-09-24 Thread John Snow
Annotations do not change runtime behavior. This commit *only* adds annotations. Signed-off-by: John Snow Reviewed-by: Eduardo Habkost --- scripts/qapi/mypy.ini | 5 - scripts/qapi/source.py | 31 ++- 2 files changed, 18 insertions(+), 18 deletions(-) diff

[PATCH v3 47/47] qapi/visit.py: add type hint annotations

2020-09-24 Thread John Snow
Annotations do not change runtime behavior. This commit *only* adds annotations. Signed-off-by: John Snow Reviewed-by: Eduardo Habkost Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa --- scripts/qapi/mypy.ini | 5 --- scripts/qapi/visit.py | 73 +-- 2

Re: [PATCH 06/16] qapi/expr.py: Check type of 'data' member

2020-09-24 Thread Cleber Rosa
On Tue, Sep 22, 2020 at 05:13:03PM -0400, John Snow wrote: > Iterating over the members of data isn't going to work if it's not some > form of dict anyway, but for type safety, formalize it. > > Signed-off-by: John Snow > --- > scripts/qapi/expr.py | 7 +++ > 1 file changed, 7 insertions(+)

[PATCH v3 38/47] qapi/instrospect.py: add preliminary type hint annotations

2020-09-24 Thread John Snow
From: Eduardo Habkost The typing of _make_tree and friends is a bit involved, but it can be done with some stubbed out types and a bit of elbow grease. The forthcoming patches attempt to make some simplifications, but having the type hints in advance can aid in review. Signed-off-by: Eduardo

[PATCH v3 26/47] qapi/commands.py: Don't re-bind to variable of different type

2020-09-24 Thread John Snow
Mypy isn't a fan of rebinding a variable with a new data type. It's easy enough to avoid. Signed-off-by: John Snow Reviewed-by: Eduardo Habkost Reviewed-by: Cleber Rosa --- scripts/qapi/commands.py | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/scripts/qapi/commands.py

[PATCH v3 31/47] qapi/gen.py: Fix edge-case of _is_user_module

2020-09-24 Thread John Snow
The edge case is that if the name is '', this expression returns a string instead of a bool, which violates our declared type. Signed-off-by: John Snow --- scripts/qapi/gen.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/qapi/gen.py b/scripts/qapi/gen.py index

[PATCH v3 41/47] qapi/introspect.py: replace 'extra' dict with 'comment' argument

2020-09-24 Thread John Snow
This is only used to pass in a dictionary with a comment already set, so skip the runaround and just accept the comment. Signed-off-by: John Snow --- scripts/qapi/introspect.py | 17 - 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/scripts/qapi/introspect.py

[PATCH v3 36/47] qapi/gen.py: delint with pylint

2020-09-24 Thread John Snow
'fp' and 'fd' are self-evident in context, add them to the list of OK names. _top and _bottom also need to stay standard methods because some users override the method and need to use `self`. Tell pylint to shush. Signed-off-by: John Snow Reviewed-by: Eduardo Habkost Reviewed-by: Cleber Rosa

[PATCH v3 46/47] qapi/visit.py: remove unused parameters from gen_visit_object

2020-09-24 Thread John Snow
And this fixes the pylint report for this file, so make sure we check this in the future, too. Signed-off-by: John Snow Reviewed-by: Eduardo Habkost Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa --- scripts/qapi/pylintrc | 1 - scripts/qapi/visit.py | 4 ++-- 2 files changed, 2

[PATCH v3 30/47] qapi/source.py: delint with pylint

2020-09-24 Thread John Snow
Shush an error and leave a hint for future cleanups when we're allowed to use Python 3.7+. Signed-off-by: John Snow Reviewed-by: Eduardo Habkost Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa --- scripts/qapi/pylintrc | 1 - scripts/qapi/source.py | 3 +++ 2 files changed, 3 insertions(+),

[PATCH v3 25/47] qapi/events.py: Move comments into docstrings

2020-09-24 Thread John Snow
Signed-off-by: John Snow Reviewed-by: Eduardo Habkost Reviewed-by: Cleber Rosa --- scripts/qapi/events.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/scripts/qapi/events.py b/scripts/qapi/events.py index 57e0939e96..9f868589bb 100644 --- a/scripts/qapi/events.py +++

[PATCH v3 35/47] qapi/gen.py: update write() to be more idiomatic

2020-09-24 Thread John Snow
Make the file handling here just a tiny bit more idiomatic. (I realize this is heavily subjective.) Use exist_ok=True for os.makedirs and remove the exception, use fdopen() to wrap the file descriptor in a File-like object, and use a context manager for managing the file pointer. Signed-off-by:

[PATCH v3 39/47] qapi/introspect.py: add _gen_features helper

2020-09-24 Thread John Snow
_make_tree might receive a dict or some other type. Adding features information should arguably be performed by the caller at such a time when we know the type of the object and don't have to re-interrogate it. Signed-off-by: John Snow --- scripts/qapi/introspect.py | 19 --- 1

[PATCH v3 22/47] qapi/common.py: move build_params into gen.py

2020-09-24 Thread John Snow
Including it in common.py creates a circular import dependency; schema relies on common, but common.build_params requires a type annotation from schema. To type this properly, it needs to be moved outside the cycle. Signed-off-by: John Snow Reviewed-by: Eduardo Habkost Reviewed-by: Cleber Rosa

[PATCH v3 32/47] qapi/gen.py: add type hint annotations

2020-09-24 Thread John Snow
Annotations do not change runtime behavior. This commit *only* adds annotations. Signed-off-by: John Snow Reviewed-by: Eduardo Habkost Reviewed-by: Cleber Rosa --- scripts/qapi/gen.py | 107 1 file changed, 58 insertions(+), 49 deletions(-) diff

[PATCH v3 34/47] qapi/gen.py: Remove unused parameter

2020-09-24 Thread John Snow
module_basename doesn't use the 'what' argument, so remove it. Signed-off-by: John Snow Reviewed-by: Eduardo Habkost Reviewed-by: Cleber Rosa --- scripts/qapi/gen.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/qapi/gen.py b/scripts/qapi/gen.py index

[PATCH v3 23/47] qapi: establish mypy type-checking baseline

2020-09-24 Thread John Snow
Fix two very minor issues, and then establish a mypy type-checking baseline. Like pylint, this should be run from the folder above: > mypy --config-file=qapi/mypy.ini qapi/ This is designed and tested for mypy 0.770 or greater. Signed-off-by: John Snow Tested-by: Eduardo Habkost ---

[PATCH v3 14/47] qapi: add pylintrc

2020-09-24 Thread John Snow
Using `pylint --generate-rcfile > pylintrc`, generate a skeleton pylintrc file. Sections that are not presently relevant (by the end of this series) are removed leaving just the empty section as a search engine / documentation hint to future authors. I am targeting pylint 2.6.0. In the future

[PATCH v3 21/47] qapi/common.py: Convert comments into docstrings, and elaborate

2020-09-24 Thread John Snow
As docstrings, they'll show up in documentation and IDE help. The docstring style being targeted is the Sphinx documentation style. Sphinx uses an extension of ReST with "domains". We use the (implicit) Python domain, which supports a number of custom "info fields". Those info fields are

[PATCH v3 20/47] qapi/common.py: add type hint annotations

2020-09-24 Thread John Snow
Annotations do not change runtime behavior. This commit *only* adds annotations. Signed-off-by: John Snow Reviewed-by: Eduardo Habkost Reviewed-by: Cleber Rosa --- scripts/qapi/common.py | 27 --- 1 file changed, 16 insertions(+), 11 deletions(-) diff --git

[PATCH v3 17/47] qapi/common.py: delint with pylint

2020-09-24 Thread John Snow
At this point, that just means using a consistent strategy for constant names. constants get UPPER_CASE and names not used externally get a leading underscore. As a preference, while renaming constants to be UPPERCASE, move them to the head of the file. Generally, it's nice to be able to audit

[PATCH v3 27/47] qapi/commands.py: add type hint annotations

2020-09-24 Thread John Snow
Annotations do not change runtime behavior. This commit *only* adds annotations. Signed-off-by: John Snow Reviewed-by: Eduardo Habkost Reviewed-by: Cleber Rosa --- scripts/qapi/commands.py | 71 ++-- 1 file changed, 54 insertions(+), 17 deletions(-) diff

[PATCH v3 24/47] qapi/events.py: add type hint annotations

2020-09-24 Thread John Snow
Annotations do not change runtime behavior. This commit *only* adds annotations. Signed-off-by: John Snow Reviewed-by: Eduardo Habkost Reviewed-by: Cleber Rosa --- scripts/qapi/events.py | 46 -- scripts/qapi/mypy.ini | 5 - 2 files changed, 35

[PATCH v3 10/47] qapi: Prefer explicit relative imports

2020-09-24 Thread John Snow
All of the QAPI include statements are changed to be package-aware, as explicit relative imports. A quirk of Python packages is that the name of the package exists only *outside* of the package. This means that to a module inside of the qapi folder, there is inherently no such thing as the "qapi"

[PATCH v3 19/47] qapi/common.py: check with pylint

2020-09-24 Thread John Snow
Remove qapi/common.py from the pylintrc ignore list. Signed-off-by: John Snow Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa Tested-by: Eduardo Habkost --- scripts/qapi/pylintrc | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/scripts/qapi/pylintrc

[PATCH v3 11/47] qapi: Remove wildcard includes

2020-09-24 Thread John Snow
Wildcard includes become hard to manage when refactoring and dealing with circular dependencies with strictly typed mypy. flake8 also flags each one as a warning, as it is not smart enough to know which names exist in the imported file. Remove them and include things explicitly by name instead.

[PATCH v3 33/47] qapi/gen.py: Enable checking with mypy

2020-09-24 Thread John Snow
Signed-off-by: John Snow Reviewed-by: Eduardo Habkost Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa --- scripts/qapi/mypy.ini | 5 - 1 file changed, 5 deletions(-) diff --git a/scripts/qapi/mypy.ini b/scripts/qapi/mypy.ini index 43c8bd1973..dbfeda748c 100644 --- a/scripts/qapi/mypy.ini

[PATCH v3 12/47] qapi: enforce import order/styling with isort

2020-09-24 Thread John Snow
While we're mucking around with imports, we might as well formalize the style we use. Let's use isort to do it for us. force_sort_within_sections: Intermingles "from x" and "import x" style statements, such that sorting is always performed strictly on the module name itself. force_grid_wrap=4:

[PATCH v3 18/47] qapi/common.py: Replace one-letter 'c' variable

2020-09-24 Thread John Snow
Signed-off-by: John Snow Reviewed-by: Cleber Rosa Reviewed-by: Eduardo Habkost --- scripts/qapi/common.py | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py index c096df0d2d..9e50070f62 100644 --- a/scripts/qapi/common.py

[PATCH v3 09/47] [DO-NOT-MERGE] docs: add scripts/qapi/main to python manual

2020-09-24 Thread John Snow
Signed-off-by: John Snow --- docs/devel/python/qapi.main.rst | 7 +++ docs/devel/python/qapi.rst | 1 + 2 files changed, 8 insertions(+) create mode 100644 docs/devel/python/qapi.main.rst diff --git a/docs/devel/python/qapi.main.rst b/docs/devel/python/qapi.main.rst new file mode

[PATCH v3 28/47] qapi/commands.py: enable checking with mypy

2020-09-24 Thread John Snow
Signed-off-by: John Snow Reviewed-by: Eduardo Habkost Reviewed-by: Cleber Rosa --- scripts/qapi/mypy.ini | 5 - 1 file changed, 5 deletions(-) diff --git a/scripts/qapi/mypy.ini b/scripts/qapi/mypy.ini index b668776d94..9da1dccef4 100644 --- a/scripts/qapi/mypy.ini +++

[PATCH v3 06/47] [DO-NOT-MERGE] docs: enable sphinx-autodoc for scripts/qapi

2020-09-24 Thread John Snow
Signed-off-by: John Snow --- docs/conf.py | 6 +- docs/devel/index.rst | 1 + docs/devel/python/index.rst | 7 +++ docs/devel/python/qapi.commands.rst | 7 +++ docs/devel/python/qapi.common.rst | 7 +++

[PATCH v3 05/47] qapi/doc.py: Change code templates from function to string

2020-09-24 Thread John Snow
For whatever reason, when these are stored as functions instead of strings, it confuses sphinx-autodoc into believing them to be docstrings, and it chokes on the syntax. Keeping them as dumb strings instead avoids the problem. Signed-off-by: John Snow --- scripts/qapi/doc.py | 40

[PATCH v3 16/47] qapi/common.py: Add indent manager

2020-09-24 Thread John Snow
Code style tools really dislike the use of global keywords, because it generally involves re-binding the name at runtime which can have strange effects depending on when and how that global name is referenced in other modules. Make a little indent level manager instead. Signed-off-by: John Snow

[PATCH v3 01/47] [DO-NOT-MERGE] docs: replace single backtick (`) with double-backtick (``)

2020-09-24 Thread John Snow
The single backtick in ReST is the "default role". Currently, Sphinx's default role is called "content". Sphinx suggests you can use the "Any" role instead to turn any single-backtick enclosed item into a cross-reference. Before we do that, though, we'll need to turn all existing usages of the

[PATCH v3 00/47] qapi: static typing conversion, pt1

2020-09-24 Thread John Snow
Hi, this series adds static type hints to the QAPI module. This is part one! Part 1: https://gitlab.com/jsnow/qemu/-/tree/python-qapi-cleanup-pt1 Everything: https://gitlab.com/jsnow/qemu/-/tree/python-qapi-cleanup-pt6 - Requires Python 3.6+ - Requires mypy 0.770 or newer (for type analysis

[PATCH v3 08/47] qapi: move generator entrypoint into module

2020-09-24 Thread John Snow
As part of delinting and adding type hints to the QAPI generator, it's helpful for the entrypoint to be part of the package, only leaving a very tiny entrypoint shim outside of the module. Signed-off-by: John Snow Reviewed-by: Eduardo Habkost Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa

[PATCH v3 15/47] qapi/common.py: Remove python compatibility workaround

2020-09-24 Thread John Snow
Signed-off-by: John Snow Reviewed-by: Eduardo Habkost Reviewed-by: Cleber Rosa --- scripts/qapi/common.py | 5 + 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py index ba35abea47..cee63eb95c 100644 --- a/scripts/qapi/common.py +++

[PATCH v3 04/47] qapi: modify docstrings to be sphinx-compatible

2020-09-24 Thread John Snow
I did not say "sphinx beautiful", just "sphinx compatible". They will not throw errors when parsed and interpreted as ReST. Signed-off-by: John Snow --- scripts/qapi/doc.py| 10 +- scripts/qapi/gen.py| 6 -- scripts/qapi/parser.py | 9 + 3 files changed, 14

[PATCH v3 07/47] qapi-gen: Separate arg-parsing from generation

2020-09-24 Thread John Snow
This is a minor re-work of the entrypoint script. It isolates a generate() method from the actual command-line mechanism. The regex match error checking was modified slightly to check that we actually got a regex match. Signed-off-by: John Snow Reviewed-by: Eduardo Habkost ---

[PATCH v3 13/47] qapi: delint using flake8

2020-09-24 Thread John Snow
Petty style guide fixes and line length enforcement. Not a big win, not a big loss, but flake8 passes 100% on the qapi module, which gives us an easy baseline to enforce hereafter. A note on the flake8 exception: flake8 will warn on *any* bare except, but pylint's is context-aware and will

[PATCH v3 02/47] [DO-NOT-MERGE] docs: repair broken references

2020-09-24 Thread John Snow
Signed-off-by: John Snow --- docs/devel/multi-thread-tcg.rst | 2 +- docs/devel/testing.rst | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/devel/multi-thread-tcg.rst b/docs/devel/multi-thread-tcg.rst index 21483870db..92a9eba13c 100644 ---

[PATCH v3 03/47] [DO-NOT-MERGE] docs/sphinx: change default role to "any"

2020-09-24 Thread John Snow
Signed-off-by: John Snow --- docs/conf.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/conf.py b/docs/conf.py index 0dbd90dc11..a68f616d5a 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -81,6 +81,9 @@ # The master toctree document. master_doc = 'index' +# Interpret `this` to

Re: [PATCH 05/16] qapi/expr.py: move string check upwards in check_type

2020-09-24 Thread Cleber Rosa
On Tue, Sep 22, 2020 at 05:13:02PM -0400, John Snow wrote: > It's a simple case, shimmy the early return upwards. > > Signed-off-by: John Snow I'm only acking this because of the very graphical "shimmy dance" movie that played in my mind. Reviewed-by: Cleber Rosa signature.asc Description:

Re: [PATCH 04/16] qapi/expr.py: Add assertion for union type 'check_dict'

2020-09-24 Thread Cleber Rosa
On Tue, Sep 22, 2020 at 05:13:01PM -0400, John Snow wrote: > mypy isn't fond of allowing you to check for bool membership in a > collection of str elements. Guard this lookup for precisely when we were > given a name. > > Signed-off-by: John Snow Reviewed-by: Cleber Rosa signature.asc

Re: [PATCH 03/16] qapi/expr.py: constrain incoming expression types

2020-09-24 Thread Cleber Rosa
On Tue, Sep 22, 2020 at 05:13:00PM -0400, John Snow wrote: > mypy does not know the types of values stored in Dicts that masquerade > as objects. Help the type checker out by constraining the type. > > Signed-off-by: John Snow Reviewed-by: Cleber Rosa signature.asc Description: PGP signature

Re: [PATCH 03/16] qapi/expr.py: constrain incoming expression types

2020-09-24 Thread Cleber Rosa
On Wed, Sep 23, 2020 at 03:42:24PM -0400, Eduardo Habkost wrote: > On Tue, Sep 22, 2020 at 05:13:00PM -0400, John Snow wrote: > > mypy does not know the types of values stored in Dicts that masquerade > > as objects. Help the type checker out by constraining the type. > > > > Signed-off-by: John

Re: [PATCH v2 36/38] qapi/visit.py: assert tag_member contains a QAPISchemaEnumType

2020-09-24 Thread Cleber Rosa
On Thu, Sep 24, 2020 at 03:36:23PM -0400, John Snow wrote: > On 9/24/20 3:10 PM, Cleber Rosa wrote: > > On Tue, Sep 22, 2020 at 05:00:59PM -0400, John Snow wrote: > > > Signed-off-by: John Snow > > > --- > > > scripts/qapi/visit.py | 15 ++- > > > 1 file changed, 10 insertions(+),

Re: [PATCH 02/16] qapi/expr.py: Check for dict instead of OrderedDict

2020-09-24 Thread Cleber Rosa
On Tue, Sep 22, 2020 at 05:12:59PM -0400, John Snow wrote: > OrderedDict is a subtype of dict, so we can check for a more general form. > > Signed-off-by: John Snow Reviewed-by: Cleber Rosa signature.asc Description: PGP signature

Re: [PATCH 01/16] qapi/expr.py: Remove 'info' argument from nested check_if_str

2020-09-24 Thread Cleber Rosa
On Tue, Sep 22, 2020 at 05:12:58PM -0400, John Snow wrote: > The function can just use the argument from the scope above. Otherwise, > we get shadowed argument errors because the parameter name clashes with > the name of a variable already in-scope. > > Signed-off-by: John Snow Reviewed-by:

Re: [PATCH v26 03/17] vfio: Add save and load functions for VFIO PCI devices

2020-09-24 Thread Alex Williamson
On Wed, 23 Sep 2020 04:54:05 +0530 Kirti Wankhede wrote: > These functions save and restore PCI device specific data - config > space of PCI device. > Used VMStateDescription to save and restore interrupt state. > > Signed-off-by: Kirti Wankhede > Reviewed-by: Neo Jia > --- > hw/vfio/pci.c

Re: [PATCH 00/16] hw/block/nvme: zoned namespace command set

2020-09-24 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20200924204516.1881843-1-...@irrelevant.dk/ Hi, This series seems to have some coding style problems. See output below for more information: Type: series Message-id: 20200924204516.1881843-1-...@irrelevant.dk Subject: [PATCH 00/16] hw/block/nvme: zoned

Re: [PATCH 0/3] Hyper-V Dynamic Memory Protocol driver (hv-balloon)

2020-09-24 Thread Maciej S. Szmigiero
On 23.09.2020 14:48, David Hildenbrand wrote: (...) >> >> I know that this a source of significant hot removal slowdown, especially >> when a "ripple effect" happens on removal: >> 1) There are 3 extra DIMMs plugged into the guest: A, B, C. >>A and B are nearly empty, but C is nearly full. >>

Re: [PATCH v2 04/38] qapi: Prefer explicit relative imports

2020-09-24 Thread Cleber Rosa
On Thu, Sep 24, 2020 at 07:17:47PM -0300, Beraldo Leal wrote: > On Thu, Sep 24, 2020 at 03:25:50PM -0400, Cleber Rosa wrote: > > On Wed, Sep 23, 2020 at 01:12:09PM -0400, John Snow wrote: > > > On 9/23/20 9:18 AM, Cleber Rosa wrote: > > > > Relative imports are a source of heated debates, but when

  1   2   3   4   5   6   >