[PATCH 3/4] python: disable too-many-positional-arguments warning
Newest versions of pylint complain about specifically positional arguments in addition to too many in general. We already disable the general case, so silence this new warning too. Signed-off-by: John Snow --- python/setup.cfg| 1 + tests/qemu-iotests/pylintrc | 1 + 2 files changed, 2 insertions(+) diff --git a/python/setup.cfg b/python/setup.cfg index 3b4e2cc5501..cf5af7e6641 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -142,6 +142,7 @@ ignore_missing_imports = True disable=consider-using-f-string, consider-using-with, too-many-arguments, +too-many-positional-arguments, too-many-function-args, # mypy handles this with less false positives. too-many-instance-attributes, no-member, # mypy also handles this better. diff --git a/tests/qemu-iotests/pylintrc b/tests/qemu-iotests/pylintrc index 05b75ee59bf..c5f4833e458 100644 --- a/tests/qemu-iotests/pylintrc +++ b/tests/qemu-iotests/pylintrc @@ -13,6 +13,7 @@ disable=invalid-name, no-else-return, too-few-public-methods, too-many-arguments, +too-many-positional-arguments, too-many-branches, too-many-lines, too-many-locals, -- 2.47.0
[PATCH 4/4] python: silence pylint raising-non-exception error
As of (at least) pylint 3.3.1, this code trips pylint up into believing we are raising something other than an Exception. We are not: the first two values may indeed be "None", but the last and final value must by definition be a SystemExit exception. Signed-off-by: John Snow --- python/scripts/mkvenv.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/python/scripts/mkvenv.py b/python/scripts/mkvenv.py index f2526af0a04..8ac5b0b2a05 100644 --- a/python/scripts/mkvenv.py +++ b/python/scripts/mkvenv.py @@ -379,6 +379,9 @@ def make_venv( # pylint: disable=too-many-arguments try: builder.create(str(env_dir)) except SystemExit as exc: +# pylint 3.3 bug: +# pylint: disable=raising-non-exception, raise-missing-from + # Some versions of the venv module raise SystemExit; *nasty*! # We want the exception that prompted it. It might be a subprocess # error that has output we *really* want to see. -- 2.47.0
[PATCH 1/4] iotests: reflow ReproducibleTestRunner arguments
Trivial reflow to let the type names breathe. (I need to add a longer type name.) Signed-off-by: John Snow --- tests/qemu-iotests/iotests.py | 11 +++ 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index ea48af4a7b6..673bbcd3563 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -1614,10 +1614,13 @@ def write(self, arg=None): self.stream.write(arg) class ReproducibleTestRunner(unittest.TextTestRunner): -def __init__(self, stream: Optional[TextIO] = None, - resultclass: Type[unittest.TestResult] = - ReproducibleTestResult, - **kwargs: Any) -> None: +def __init__( +self, +stream: Optional[TextIO] = None, +resultclass: Type[unittest.TestResult] = +ReproducibleTestResult, +**kwargs: Any +) -> None: rstream = ReproducibleStreamWrapper(stream or sys.stdout) super().__init__(stream=rstream, # type: ignore descriptions=True, -- 2.47.0
[PATCH 2/4] iotests: correct resultclass type in ReproducibleTestRunner
I have a vague memory that I suggested this base class to Vladimir and said "Maybe someday it will break, and I'll just fix it then." Guess that's today. Fixes various mypy errors in the "make check-tox" python test for at least Python3.8; seemingly requires a fairly modern mypy and/or Python base version to trigger. Signed-off-by: John Snow --- tests/qemu-iotests/iotests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 673bbcd3563..19817c73530 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -1617,7 +1617,7 @@ class ReproducibleTestRunner(unittest.TextTestRunner): def __init__( self, stream: Optional[TextIO] = None, -resultclass: Type[unittest.TestResult] = +resultclass: Type[unittest.TextTestResult] = ReproducibleTestResult, **kwargs: Any ) -> None: -- 2.47.0
[PATCH 0/4] python: update linting for new mypy/pylint releases
Various python tests in the "check-python-tox" test case on GitLab have begun failing due to newer package versions. This patch set corrects those issues and also improves the reliability of local developer tests which may be using these tooling versions outside of GitLab pinned version tests. There are remaining issues with the "check-dev" test I have yet to rectify, but appear unrelated to linter versions specifically and will be handled separately. As a result of this patch, the optionally-run and may-fail "check-python-tox" test case on GitLab will become green again, and local invocations of "make check-tox" in the python subdirectory will also pass again. "check-python-minreqs" on GitLab and "make check-minreqs" in the local developer environment were/are unaffected. local iotest invocations for test case #297 ought to now begin passing on developer workstations with bleeding-edge python packages. John Snow (4): iotests: reflow ReproducibleTestRunner arguments iotests: correct resultclass type in ReproducibleTestRunner python: disable too-many-positional-arguments warning python: silence pylint raising-non-exception error python/scripts/mkvenv.py | 3 +++ python/setup.cfg | 1 + tests/qemu-iotests/iotests.py | 11 +++ tests/qemu-iotests/pylintrc | 1 + 4 files changed, 12 insertions(+), 4 deletions(-) -- 2.47.0
Re: qemu-iotests test 297 tries to run python linters on editor backup files
On Tue, Oct 8, 2024, 1:03 PM Peter Maydell wrote: > On Tue, 8 Oct 2024 at 17:50, John Snow wrote: > > > > > > > > On Tue, Oct 8, 2024, 12:31 PM Peter Maydell > wrote: > >> > >> I made some changes to a block backend so I wanted to run the iotests. > >> I ran into an unrelated failure of iotest 297. The bulk of this > >> seems to be because the iotest tries to run on all files > >> in qemu-iotests, even on editor backups like in this case "040~" > >> (which is an old editor backup of 040). But there are also > >> some warnings about files that do exist in the tree and which > >> I haven't modified: > >> > >> +tests/migrate-bitmaps-test:63:4: R0201: Method could be a function > >> (no-self-use) > >> +tests/mirror-change-copy-mode:109:4: R0201: Method could be a > >> function (no-self-use) > >> +fat16.py:239:4: R0201: Method could be a function (no-self-use) > >> > >> Q1: can we make this test not run the linters on editor backup files? > > > > > > Shouldn't be a problem. AFAIK we decide what to lint based on looking > for the shebang in the file and exclude a known-bad list, but we can > exclude the emacs confetti files too. > > > > I'll fix it. > > Thanks! > > >> Q2: why do I see the errors above but they aren't in the reference file > >> output? > > > > > > You mean, why are there errors in files you haven't modified? > > Yes, and/or "why isn't the test forcing a pylint version?" > and/or "why is the test run by default if it depends on > the pylint version?" :-) > "because it's always been like that and I've had difficulties fixing it", mostly :) qemu configure venv was the first step to finally fixing it, but I've run into some troubles since but have been too busy with Sphinx junk. I know it's bonkers, sorry! > > Very likely: pylint version differences. I've been meaning to remove > iotest 297 for a long time, but when you run it directly through iotests > (i.e. not through the python directory tests, the ones that run on gitlab > ci), the linter versions are not controlled for. It's a remaining ugly spot > of the python consistency work. (sparing you the details on why but it's a > known thing I need to fix.) > > > > In this case I bet "check-python-tox" (an optionally run, may-fail job) > is also failing on gitlab and I didn't notice yet. > > I kicked off a job by hand, and yes, it fails, but apparently > not for the same set of errors as the above... > > https://gitlab.com/qemu-project/qemu/-/jobs/8009902380 > > > for now (assuming my guesses above are right): I'll fix 297 to tolerate > the newest versions. As soon as I'm done my sphinx work, I'll pivot back to > finally adding a "check python" subtest to "make check" that *does* control > linter versions, and delete iotest 297. > > > > Just in case my guesses are wrong, can you please go to your QEMU build > directory (post-configure) and type: > > > > > source pyvenv/bin/activate.[whatever shell suffix you use] > > > pylint --version > > > deactivate > > > > and let me know what version of pylint you have in the qemu build > environment? > > (pyvenv) e104462:jammy:x86-tgts$ pylint --version > pylint 2.12.2 > astroid 2.9.3 > Python 3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0] > > (This is an Ubuntu 22.04 system.) > > thanks > -- PMM > Great, thanks. will work on this today and get everything green again. >
Re: qemu-iotests test 297 tries to run python linters on editor backup files
On Tue, Oct 8, 2024, 12:31 PM Peter Maydell wrote: > I made some changes to a block backend so I wanted to run the iotests. > I ran into an unrelated failure of iotest 297. The bulk of this > seems to be because the iotest tries to run on all files > in qemu-iotests, even on editor backups like in this case "040~" > (which is an old editor backup of 040). But there are also > some warnings about files that do exist in the tree and which > I haven't modified: > > +tests/migrate-bitmaps-test:63:4: R0201: Method could be a function > (no-self-use) > +tests/mirror-change-copy-mode:109:4: R0201: Method could be a > function (no-self-use) > +fat16.py:239:4: R0201: Method could be a function (no-self-use) > > Q1: can we make this test not run the linters on editor backup files? > Shouldn't be a problem. AFAIK we decide what to lint based on looking for the shebang in the file and exclude a known-bad list, but we can exclude the emacs confetti files too. I'll fix it. (Guess I haven't been editing the tests for a while... and nobody else has mentioned it. oops.) Q2: why do I see the errors above but they aren't in the reference file > output? > You mean, why are there errors in files you haven't modified? Very likely: pylint version differences. I've been meaning to remove iotest 297 for a long time, but when you run it directly through iotests (i.e. not through the python directory tests, the ones that run on gitlab ci), the linter versions are not controlled for. It's a remaining ugly spot of the python consistency work. (sparing you the details on why but it's a known thing I need to fix.) In this case I bet "check-python-tox" (an optionally run, may-fail job) is also failing on gitlab and I didn't notice yet. for now (assuming my guesses above are right): I'll fix 297 to tolerate the newest versions. As soon as I'm done my sphinx work, I'll pivot back to finally adding a "check python" subtest to "make check" that *does* control linter versions, and delete iotest 297. Just in case my guesses are wrong, can you please go to your QEMU build directory (post-configure) and type: > source pyvenv/bin/activate.[whatever shell suffix you use] > pylint --version > deactivate and let me know what version of pylint you have in the qemu build environment? > > e104462:jammy:qemu-iotests$ ./check 297 > QEMU -- > > "/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/x86-tgts/qemu-system-x86_64" > -nodefaults -display none -accel qtest > QEMU_IMG -- > "/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/x86-tgts/qemu-img" > QEMU_IO -- > "/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/x86-tgts/qemu-io" > --cache writeback --aio threads -f raw > QEMU_NBD -- > "/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/x86-tgts/qemu-nbd" > IMGFMT-- raw > IMGPROTO -- file > PLATFORM -- Linux/x86_64 e104462 5.15.0-89-generic > TEST_DIR -- > > /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/x86-tgts/tests/qemu-iotests/scratch > SOCK_DIR -- /tmp/qemu-iotests-0c1ft_vw > GDB_OPTIONS -- > VALGRIND_QEMU -- > PRINT_QEMU_OUTPUT -- > > 297 fail [17:26:10] [17:26:23] 13.1soutput > mismatch (see > /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/x86-tgts/tests/qemu-iotests/scratch/raw-file-297/297.out.bad) > [case not run] 'mypy' not found > > --- /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/tests/qemu-iotests/297.out > +++ > /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/x86-tgts/tests/qemu-iotests/scratch/raw-file-297/297.out.bad > @@ -1,2 +1,74 @@ > === pylint === > +* Module migrate-bitmaps-test > +tests/migrate-bitmaps-test:63:4: R0201: Method could be a function > (no-self-use) > +* Module mirror-change-copy-mode > +tests/mirror-change-copy-mode:109:4: R0201: Method could be a > function (no-self-use) > +* Module fat16 > +fat16.py:239:4: R0201: Method could be a function (no-self-use) > +* Module 040~ > +040~:50:0: C0301: Line too long (85/79) (line-too-long) > +040~:64:0: C0301: Line too long (86/79) (line-too-long) > +040~:91:0: C0301: Line too long (138/79) (line-too-long) > [PMM: deleted a lot more warnings about this editor backup file] > === mypy === > Some cases not run in: > /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/tests/qemu-iotests/297 > Failures: 297 > Failed 1 of 1 iotests > > thanks > -- PMM > Thanks for the report. --js >
Re: [PATCH v2 6/9] qapi: convert "Example" sections without titles
On Wed, Jul 17, 2024, 3:44 AM Markus Armbruster wrote:
> John Snow writes:
>
> > Use the no-option form of ".. qmp-example::" to convert any Examples
> > that do not have any form of caption or explanation whatsoever. Note
> > that in a few cases, example sections are split into two or more
> > separate example blocks. This is only done stylistically to create a
> > delineation between two or more logically independent examples.
> >
> > See commit-3: "docs/qapidoc: create qmp-example directive", for a
> > detailed explanation of this custom directive syntax.
> >
> > See commit+3: "qapi: remove "Example" doc section" for a detailed
> > explanation of why.
> >
> > Note: an empty "TODO" line was added to announce-self to keep the
> > example from floating up into the body; this will be addressed more
> > rigorously in the new qapidoc generator.
> >
> > Signed-off-by: John Snow
>
> [...]
>
> > diff --git a/qapi/run-state.json b/qapi/run-state.json
> > index 4d40c88876c..ab7116680b3 100644
> > --- a/qapi/run-state.json
> > +++ b/qapi/run-state.json
>
> [...]
>
> > @@ -469,7 +469,7 @@
> > #
> > # Since: 9.1
> > #
> > -# Example:
> > +# ..qmp-example::
>
> Lacks a space, output is messed up. Can fix in my tree.
>
Whoops. Wish rST was a bit pickier sometimes...
> > #
> > # <- { "event": "GUEST_PVSHUTDOWN",
> > # "timestamp": { "seconds": 1648245259, "microseconds": 893771
> } }
>
> [...]
>
> R-by stands.
>
>
Re: [PATCH v2 0/9] qapi: convert example sections to qmp-example rST directives
On Wed, Jul 17, 2024, 6:47 AM Markus Armbruster wrote: > John Snow writes: > > > This patchset focuses on converting example sections to rST directives > > using a new `.. qmp-example::` directive. > > Queued, thanks! > Yay! >
[PATCH v2 7/9] qapi: convert "Example" sections with titles
When an Example section has a brief explanation, convert it to a
qmp-example:: section using the :title: option.
Rule of thumb: If the title can fit on a single line and requires no rST
markup, it's a good candidate for using the :title: option of
qmp-example.
In this patch, trailing punctuation is removed from the title section
for consistent headline aesthetics. In just one case, specifics of the
example are removed to make the title read better.
See commit-4: "docs/qapidoc: create qmp-example directive", for a
detailed explanation of this custom directive syntax.
See commit+2: "qapi: remove "Example" doc section" for a detailed
explanation of why.
Signed-off-by: John Snow
Reviewed-by: Markus Armbruster
---
qapi/block-core.json | 24
qapi/block.json | 13 ++---
qapi/migration.json | 25 ++---
qapi/qom.json| 8
qapi/ui.json | 11 ++-
qapi/virtio.json | 19 ++-
6 files changed, 52 insertions(+), 48 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 84a020aa9e3..f400b334c84 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -5881,9 +5881,8 @@
#
# Since: 2.7
#
-# Examples:
-#
-# 1. Add a new node to a quorum
+# .. qmp-example::
+#:title: Add a new node to a quorum
#
# -> { "execute": "blockdev-add",
# "arguments": {
@@ -5897,7 +5896,8 @@
# "node": "new_node" } }
# <- { "return": {} }
#
-# 2. Delete a quorum's node
+# .. qmp-example::
+#:title: Delete a quorum's node
#
# -> { "execute": "x-blockdev-change",
# "arguments": { "parent": "disk1",
@@ -5933,16 +5933,16 @@
#
# Since: 2.12
#
-# Examples:
-#
-# 1. Move a node into an IOThread
+# .. qmp-example::
+#:title: Move a node into an IOThread
#
# -> { "execute": "x-blockdev-set-iothread",
# "arguments": { "node-name": "disk1",
# "iothread": "iothread0" } }
# <- { "return": {} }
#
-# 2. Move a node into the main loop
+# .. qmp-example::
+#:title: Move a node into the main loop
#
# -> { "execute": "x-blockdev-set-iothread",
# "arguments": { "node-name": "disk1",
@@ -6018,16 +6018,16 @@
#
# Since: 2.0
#
-# Examples:
-#
-# 1. Read operation
+# .. qmp-example::
+#:title: Read operation
#
# <- { "event": "QUORUM_REPORT_BAD",
# "data": { "node-name": "node0", "sector-num": 345435,
"sectors-count": 5,
#"type": "read" },
# "timestamp": { "seconds": 1344522075, "microseconds": 745528 } }
#
-# 2. Flush operation
+# .. qmp-example::
+#:title: Flush operation
#
# <- { "event": "QUORUM_REPORT_BAD",
# "data": { "node-name": "node0", "sector-num": 0, "sectors-count":
2097120,
diff --git a/qapi/block.json b/qapi/block.json
index c8e52bc2d29..5ddd061e964 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -342,9 +342,8 @@
#
# Since: 2.5
#
-# Examples:
-#
-# 1. Change a removable medium
+# .. qmp-example::
+#:title: Change a removable medium
#
# -> { "execute": "blockdev-change-medium",
# "arguments": { "id": "ide0-1-0",
@@ -352,7 +351,8 @@
# "format": "raw" } }
# <- { "return": {} }
#
-# 2. Load a read-only medium into a writable drive
+# .. qmp-example::
+#:title: Load a read-only medium into a writable drive
#
# -> { "execute": "blockdev-change-medium",
# "arguments": { "id": "floppyA",
@@ -577,9 +577,8 @@
# "boundaries-write": [1000, 5000] } }
# <- { "return": {} }
#
-# Example:
-#
-# Remove all latency histograms:
+# .. qmp-example::
+#:title: Remove all latency histograms
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0" } }
diff --git a/qapi/migration.json b/qapi/migration.json
index 3c657202389..74968a14173 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -287,14 +287,14 @@
#
# Since: 0.14
#
-# Examples:
-#
-# 1. Before the first migration
+# .. qmp-example::
+#:title: Before the first migration
#
# -> { "execute": "query
[PATCH v2 8/9] qapi: convert "Example" sections with longer prose
These examples require longer explanations or have explanations that
require markup to look reasonable when rendered and so use the longer
form of the ".. qmp-example::" directive.
By using the :annotated: option, the content in the example block is
assumed *not* to be a code block literal and is instead parsed as normal
rST - with the exception that any code literal blocks after `::` will
assumed to be a QMP code literal block.
Note: There's one title-less conversion in this patch that comes along
for the ride because it's part of a larger "Examples" block that was
better to convert all at once.
See commit-5: "docs/qapidoc: create qmp-example directive", for a
detailed explanation of this custom directive syntax.
See commit+1: "qapi: remove "Example" doc section" for a detailed
explanation of why.
Signed-off-by: John Snow
Reviewed-by: Markus Armbruster
---
qapi/block.json | 30 +++---
qapi/machine.json | 30 --
qapi/migration.json | 7 +--
qapi/virtio.json| 24 ++--
4 files changed, 62 insertions(+), 29 deletions(-)
diff --git a/qapi/block.json b/qapi/block.json
index 5ddd061e964..ce9490a3678 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -545,31 +545,37 @@
#
# Since: 4.0
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# Set new histograms for all io types with intervals
-# [0, 10), [10, 50), [50, 100), [100, +inf):
+#Set new histograms for all io types with intervals
+#[0, 10), [10, 50), [50, 100), [100, +inf)::
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0",
# "boundaries": [10, 50, 100] } }
# <- { "return": {} }
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# Set new histogram only for write, other histograms will remain
-# not changed (or not created):
+#Set new histogram only for write, other histograms will remain
+#not changed (or not created)::
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0",
# "boundaries-write": [10, 50, 100] } }
# <- { "return": {} }
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# Set new histograms with the following intervals:
-# read, flush: [0, 10), [10, 50), [50, 100), [100, +inf)
-# write: [0, 1000), [1000, 5000), [5000, +inf)
+#Set new histograms with the following intervals:
+#
+#- read, flush: [0, 10), [10, 50), [50, 100), [100, +inf)
+#- write: [0, 1000), [1000, 5000), [5000, +inf)
+#
+#::
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0",
@@ -578,7 +584,9 @@
# <- { "return": {} }
#
# .. qmp-example::
-#:title: Remove all latency histograms
+#:annotated:
+#
+#Remove all latency histograms::
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0" } }
diff --git a/qapi/machine.json b/qapi/machine.json
index 83f60b319c7..0a5ffe652b7 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -1047,10 +1047,11 @@
#
# Since: 2.7
#
-# Examples:
+# .. qmp-example::
+#:annotated:
#
-# For pseries machine type started with -smp 2,cores=2,maxcpus=4
-# -cpu POWER8:
+#For pseries machine type started with
+#``-smp 2,cores=2,maxcpus=4 -cpu POWER8``::
#
# -> { "execute": "query-hotpluggable-cpus" }
# <- {"return": [
@@ -1060,7 +1061,10 @@
#"vcpus-count": 1, "qom-path": "/machine/unattached/device[0]"}
#]}
#
-# For pc machine type started with -smp 1,maxcpus=2:
+# .. qmp-example::
+#:annotated:
+#
+#For pc machine type started with ``-smp 1,maxcpus=2``::
#
# -> { "execute": "query-hotpluggable-cpus" }
# <- {"return": [
@@ -1075,8 +1079,11 @@
# }
#]}
#
-# For s390x-virtio-ccw machine type started with -smp 1,maxcpus=2
-# -cpu qemu (Since: 2.11):
+# .. qmp-example::
+#:annotated:
+#
+#For s390x-virtio-ccw machine type started with
+#``-smp 1,maxcpus=2 -cpu qemu`` (Since: 2.11)::
#
# -> { "execute": "query-hotpluggable-cpus" }
# <- {"return": [
@@ -1130,12 +1137,15 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# -> { "execute": "balloon", "arguments": { "value": 536870912 } }
-# <- { "return": {} }
+#
[PATCH v2 9/9] qapi: remove "Example" doc section
Fully eliminate the "Example" sections in QAPI doc blocks now that they
have all been converted to arbitrary rST syntax using the
".. qmp-example::" directive. Update tests to match.
Migrating to the new syntax
---
The old "Example:" or "Examples:" section syntax is now caught as an
error, but "Example::" is stil permitted as explicit rST syntax for an
un-lexed, generic preformatted text block.
('Example' is not special in this case, any sentence that ends with "::"
will start an indented code block in rST.)
Arbitrary rST for Examples is now possible, but it's strongly
recommended that documentation authors use the ".. qmp-example::"
directive for consistent visual formatting in rendered HTML docs. The
":title:" directive option may be used to add extra information into the
title bar for the example. The ":annotated:" option can be used to write
arbitrary rST instead, with nested "::" blocks applying QMP formatting
where desired.
Other choices available are ".. code-block:: QMP" which will not create
an "Example:" box, or the short-form "::" code-block syntax which will
not apply QMP highlighting when used outside of the qmp-example
directive.
Why?
This patch has several benefits:
1. Example sections can now be written more arbitrarily, mixing
explanatory paragraphs and code blocks however desired.
2. Example sections can now use fully arbitrary rST.
3. All code blocks are now lexed and validated as QMP; increasing
usability of the docs and ensuring validity of example snippets.
(To some extent - This patch only gaurantees it lexes correctly, not
that it's valid under the JSON or QMP grammars. It will catch most
small mistakes, however.)
4. Each qmp-example can be titled or annotated independently without
bypassing the QMP lexer/validator.
(i.e. code blocks are now for *code* only, so we don't have to
sacrifice exposition for having lexically valid examples.)
NOTE: As with the "Notes" conversion (d461c279737), this patch (and the
three preceding) may change the rendering order for Examples in
the current generator. The forthcoming qapidoc rewrite will fix
this by always generating documentation in source order.
Signed-off-by: John Snow
---
docs/devel/qapi-code-gen.rst| 58 -
scripts/qapi/parser.py | 10 +-
tests/qapi-schema/doc-good.json | 19 +++
tests/qapi-schema/doc-good.out | 26 ++-
tests/qapi-schema/doc-good.txt | 23 ++---
5 files changed, 98 insertions(+), 38 deletions(-)
diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index ae97b335cbf..583207a8ec2 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -899,7 +899,7 @@ Documentation markup
Documentation comments can use most rST markup. In particular,
-a ``::`` literal block can be used for examples::
+a ``::`` literal block can be used for pre-formatted text::
# ::
#
@@ -995,8 +995,8 @@ line "Features:", like this::
# @feature: Description text
A tagged section begins with a paragraph that starts with one of the
-following words: "Since:", "Example:"/"Examples:", "Returns:",
-"Errors:", "TODO:". It ends with the start of a new section.
+following words: "Since:", "Returns:", "Errors:", "TODO:". It ends with
+the start of a new section.
The second and subsequent lines of tagged sections must be indented
like this::
@@ -1020,13 +1020,53 @@ detailing a relevant error condition. For example::
A "Since: x.y.z" tagged section lists the release that introduced the
definition.
-An "Example" or "Examples" section is rendered entirely
-as literal fixed-width text. "TODO" sections are not rendered at all
-(they are for developers, not users of QMP). In other sections, the
-text is formatted, and rST markup can be used.
+"TODO" sections are not rendered (they are for developers, not users of
+QMP). In other sections, the text is formatted, and rST markup can be
+used.
+
+QMP Examples can be added by using the ``.. qmp-example::``
+directive. In its simplest form, this can be used to contain a single
+QMP code block which accepts standard JSON syntax with additional server
+directionality indicators (``->`` and ``<-``), and elisions (``...``).
+
+Optionally, a plaintext title may be provided by using the ``:title:``
+directive option. If the title is omitted, the example title will
+default to "Example:".
+
+A simple QMP example::
+
+ # .. qmp-example::
+ #:title: Using query-block
+ #
+ #-> { "execute": "query-blo
[PATCH v2 3/9] docs/qapidoc: create qmp-example directive
This is a directive that creates a syntactic sugar for creating
"Example" boxes very similar to the ones already used in the bitmaps.rst
document, please see e.g.
https://www.qemu.org/docs/master/interop/bitmaps.html#creation-block-dirty-bitmap-add
In its simplest form, when a custom title is not needed or wanted, and
the example body is *solely* a QMP example:
```
.. qmp-example::
{body}
```
is syntactic sugar for:
```
.. admonition:: Example:
.. code-block:: QMP
{body}
```
When a custom, plaintext title that describes the example is desired,
this form:
```
.. qmp-example::
:title: Defrobnification
{body}
```
Is syntactic sugar for:
```
.. admonition:: Example: Defrobnification
.. code-block:: QMP
{body}
```
Lastly, when Examples are multi-step processes that require non-QMP
exposition, have lengthy titles, or otherwise involve prose with rST
markup (lists, cross-references, etc), the most complex form:
```
.. qmp-example::
:annotated:
This example shows how to use `foo-command`::
{body}
For more information, please see `frobnozz`.
```
Is desugared to:
```
.. admonition:: Example:
This example shows how to use `foo-command`::
{body}
For more information, please see `frobnozz`.
```
Note that :annotated: and :title: options can be combined together, if
desired.
The primary benefit here being documentation source consistently using
the same directive for all forms of examples to ensure consistent visual
styling, and ensuring all relevant prose is visually grouped alongside
the code literal block.
Note that as of this commit, the code-block rST syntax "::" does not
apply QMP highlighting; you would need to use ".. code-block:: QMP". The
very next commit changes this behavior to assume all "::" code blocks
within this directive are QMP blocks.
Signed-off-by: John Snow
Acked-by: Markus Armbruster
---
docs/sphinx/qapidoc.py | 55 ++
1 file changed, 55 insertions(+)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index b3be82998a8..11defcfa3f6 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -27,6 +27,7 @@
import os
import re
import textwrap
+from typing import List
from docutils import nodes
from docutils.parsers.rst import Directive, directives
@@ -35,6 +36,7 @@
from qapi.gen import QAPISchemaVisitor
from qapi.schema import QAPISchema
+from sphinx.directives.code import CodeBlock
from sphinx.errors import ExtensionError
from sphinx.util.docutils import switch_source_input
from sphinx.util.nodes import nested_parse_with_titles
@@ -538,10 +540,63 @@ def run(self):
raise ExtensionError(str(err)) from err
+class QMPExample(CodeBlock, NestedDirective):
+"""
+Custom admonition for QMP code examples.
+
+When the :annotated: option is present, the body of this directive
+is parsed as normal rST instead. Code blocks must be explicitly
+written by the user, but this allows for intermingling explanatory
+paragraphs with arbitrary rST syntax and code blocks for more
+involved examples.
+
+When :annotated: is absent, the directive body is treated as a
+simple standalone QMP code block literal.
+"""
+
+required_argument = 0
+optional_arguments = 0
+has_content = True
+option_spec = {
+"annotated": directives.flag,
+"title": directives.unchanged,
+}
+
+def admonition_wrap(self, *content) -> List[nodes.Node]:
+title = "Example:"
+if "title" in self.options:
+title = f"{title} {self.options['title']}"
+
+admon = nodes.admonition(
+"",
+nodes.title("", title),
+*content,
+classes=["admonition", "admonition-example"],
+)
+return [admon]
+
+def run_annotated(self) -> List[nodes.Node]:
+content_node: nodes.Element = nodes.section()
+self.do_parse(self.content, content_node)
+return content_node.children
+
+def run(self) -> List[nodes.Node]:
+annotated = "annotated" in self.options
+
+if annotated:
+content_nodes = self.run_annotated()
+else:
+self.arguments = ["QMP"]
+content_nodes = super().run()
+
+return self.admonition_wrap(*content_nodes)
+
+
def setup(app):
"""Register qapi-doc directive with Sphinx"""
app.add_config_value("qapidoc_srctree", None, "env")
app.add_directive("qapi-doc", QAPIDocDirective)
+app.add_directive("qmp-example", QMPExample)
return {
"version": __version__,
--
2.45.0
[PATCH v2 1/9] [DO-NOT-MERGE]: Add some ad-hoc linting helpers.
These aren't ready for upstream inclusion, because they do not properly manage version dependencies, execution environment and so on. These are just the tools I use in my Own Special Environment :tm: for testing and debugging. They've been tested only on Fedora 38 for right now, which means: Python 3.11.4-1.fc38 pylint 2.17.4-2.fc38 mypy 1.4.0-1.fc38 isort 5.12.0-1.fc38 flake8 5.0.3-2.fc38 "Soon" :tm: I'll move the qapi generator code under the python/ directory to take advantage of the more robust linting infrastructure there, but that's going to happen after this series. Sorry about that! Signed-off-by: John Snow --- scripts/qapi-lint.sh | 61 +++ scripts/qapi/Makefile | 5 2 files changed, 66 insertions(+) create mode 100755 scripts/qapi-lint.sh create mode 100644 scripts/qapi/Makefile diff --git a/scripts/qapi-lint.sh b/scripts/qapi-lint.sh new file mode 100755 index 000..732716f2444 --- /dev/null +++ b/scripts/qapi-lint.sh @@ -0,0 +1,61 @@ +#!/usr/bin/env bash +set -e + +if [[ -f qapi/.flake8 ]]; then +echo "flake8 --config=qapi/.flake8 qapi/" +flake8 --config=qapi/.flake8 qapi/ +fi +if [[ -f qapi/pylintrc ]]; then +echo "pylint --rcfile=qapi/pylintrc qapi/" +pylint --rcfile=qapi/pylintrc qapi/ +fi +if [[ -f qapi/mypy.ini ]]; then +echo "mypy --config-file=qapi/mypy.ini qapi/" +mypy --config-file=qapi/mypy.ini qapi/ +fi + +if [[ -f qapi/.isort.cfg ]]; then +pushd qapi +echo "isort -c ." +isort -c . +popd +fi + +if [[ -f ../docs/sphinx/qapi-domain.py ]]; then +pushd ../docs/sphinx + +echo "mypy --strict qapi-domain.py" +mypy --strict qapi-domain.py +echo "flake8 qapi-domain.py --max-line-length=99" +flake8 qapi-domain.py --max-line-length=99 +echo "isort qapi-domain.py" +isort qapi-domain.py +echo "black --check qapi-domain.py" +black --check qapi-domain.py + +popd +fi + +if [[ -f ../docs/sphinx/qapidoc.py ]]; then +pushd ../docs/sphinx + +echo "pylint --rc-file ../../scripts/qapi/pylintrc qapidoc.py" +PYTHONPATH=../scripts/ pylint \ + --rc-file ../../scripts/qapi/pylintrc \ + qapidoc.py +echo "flake8 qapidoc.py --max-line-length=80" +flake8 qapidoc.py --max-line-length=80 +echo "isort qapidoc.py" +isort qapidoc.py +echo "black --line-length 80 --check qapidoc.py" +black --line-length 80 --check qapidoc.py + +popd +fi + +pushd ../build +make -j13 +make check-qapi-schema +make docs +make sphinxdocs +popd diff --git a/scripts/qapi/Makefile b/scripts/qapi/Makefile new file mode 100644 index 000..314e8a5505e --- /dev/null +++ b/scripts/qapi/Makefile @@ -0,0 +1,5 @@ +check: + isort -c . + flake8 . + cd .. && pylint --rcfile=qapi/pylintrc qapi + cd .. && mypy -p qapi --config-file=qapi/mypy.ini -- 2.45.0
[PATCH v2 6/9] qapi: convert "Example" sections without titles
Use the no-option form of ".. qmp-example::" to convert any Examples
that do not have any form of caption or explanation whatsoever. Note
that in a few cases, example sections are split into two or more
separate example blocks. This is only done stylistically to create a
delineation between two or more logically independent examples.
See commit-3: "docs/qapidoc: create qmp-example directive", for a
detailed explanation of this custom directive syntax.
See commit+3: "qapi: remove "Example" doc section" for a detailed
explanation of why.
Note: an empty "TODO" line was added to announce-self to keep the
example from floating up into the body; this will be addressed more
rigorously in the new qapidoc generator.
Signed-off-by: John Snow
---
qapi/acpi.json | 4 +--
qapi/block-core.json | 64 +---
qapi/block.json | 18 ++-
qapi/char.json | 24 +--
qapi/control.json| 8 ++---
qapi/dump.json | 8 ++---
qapi/machine-target.json | 2 +-
qapi/machine.json| 38
qapi/migration.json | 58 ++--
qapi/misc-target.json| 22 +++---
qapi/misc.json | 32 ++--
qapi/net.json| 22 --
qapi/pci.json| 2 +-
qapi/qdev.json | 10 ---
qapi/qom.json| 8 ++---
qapi/replay.json | 8 ++---
qapi/rocker.json | 8 ++---
qapi/run-state.json | 32 ++--
qapi/tpm.json| 6 ++--
qapi/trace.json | 4 +--
qapi/transaction.json| 2 +-
qapi/ui.json | 34 ++---
qapi/vfio.json | 2 +-
qapi/virtio.json | 2 +-
qapi/yank.json | 4 +--
25 files changed, 219 insertions(+), 203 deletions(-)
diff --git a/qapi/acpi.json b/qapi/acpi.json
index aa4dbe57943..045dab6228b 100644
--- a/qapi/acpi.json
+++ b/qapi/acpi.json
@@ -111,7 +111,7 @@
#
# Since: 2.1
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-acpi-ospm-status" }
# <- { "return": [ { "device": "d1", "slot": "0", "slot-type": "DIMM",
"source": 1, "status": 0},
@@ -131,7 +131,7 @@
#
# Since: 2.1
#
-# Example:
+# .. qmp-example::
#
# <- { "event": "ACPI_DEVICE_OST",
# "data": { "info": { "device": "d1", "slot": "0",
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 096bdbe0aad..84a020aa9e3 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -764,7 +764,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-block" }
# <- {
@@ -1168,7 +1168,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-blockstats" }
# <- {
@@ -1461,7 +1461,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "block_resize",
# "arguments": { "device": "scratch", "size": 1073741824 } }
@@ -1680,7 +1680,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "blockdev-snapshot-sync",
# "arguments": { "device": "ide-hd0",
@@ -1711,7 +1711,7 @@
#
# Since: 2.5
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "blockdev-add",
# "arguments": { "driver": "qcow2",
@@ -1857,7 +1857,7 @@
#
# Since: 1.3
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "block-commit",
# "arguments": { "device": "virtio0",
@@ -1895,7 +1895,7 @@
#
# Since: 1.6
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "drive-backup",
# "arguments": { "device": "drive0",
@@ -1921,7 +1921,7 @@
#
# Since: 2.3
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "blockdev-backup",
# "arguments": { "device": "src-id",
@@ -1945,7 +1945,7 @@
#
# Since: 2.0
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-named-block-nodes" }
# <- { "return": [ { "ro":false,
@@ -2126,7 +2126,7 @@
#
# Since: 1.3
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "drive-mirror",
# "arguments": { "device": "ide-hd0",
@@ -2303,7 +2303,7 @@
#
# Since: 2.4
#
-# Example:
+# .. qmp-exampl
[PATCH v2 4/9] docs/qapidoc: add QMP highlighting to annotated qmp-example blocks
For any code literal blocks inside of a qmp-example directive, apply and
enforce the QMP lexer/highlighter to those blocks.
This way, you won't need to write:
```
.. qmp-example::
:annotated:
Blah blah
.. code-block:: QMP
-> { "lorem": "ipsum" }
```
But instead, simply:
```
.. qmp-example::
:annotated:
Blah blah::
-> { "lorem": "ipsum" }
```
Once the directive block is exited, whatever the previous default
highlight language was will be restored; localizing the forced QMP
lexing to exclusively this directive.
Note, if the default language is *already* QMP, this directive will not
generate and restore redundant highlight configuration nodes. We may
well decide that the default language ought to be QMP for any QAPI
reference pages, but this way the directive behaves consistently no
matter where it is used.
Signed-off-by: John Snow
Acked-by: Markus Armbruster
---
docs/sphinx/qapidoc.py | 53 ++
1 file changed, 49 insertions(+), 4 deletions(-)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index 11defcfa3f6..738b2450fb1 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -26,6 +26,7 @@
import os
import re
+import sys
import textwrap
from typing import List
@@ -36,6 +37,7 @@
from qapi.gen import QAPISchemaVisitor
from qapi.schema import QAPISchema
+from sphinx import addnodes
from sphinx.directives.code import CodeBlock
from sphinx.errors import ExtensionError
from sphinx.util.docutils import switch_source_input
@@ -545,10 +547,10 @@ class QMPExample(CodeBlock, NestedDirective):
Custom admonition for QMP code examples.
When the :annotated: option is present, the body of this directive
-is parsed as normal rST instead. Code blocks must be explicitly
-written by the user, but this allows for intermingling explanatory
-paragraphs with arbitrary rST syntax and code blocks for more
-involved examples.
+is parsed as normal rST, but with any '::' code blocks set to use
+the QMP lexer. Code blocks must be explicitly written by the user,
+but this allows for intermingling explanatory paragraphs with
+arbitrary rST syntax and code blocks for more involved examples.
When :annotated: is absent, the directive body is treated as a
simple standalone QMP code block literal.
@@ -562,6 +564,33 @@ class QMPExample(CodeBlock, NestedDirective):
"title": directives.unchanged,
}
+def _highlightlang(self) -> addnodes.highlightlang:
+"""Return the current highlightlang setting for the document"""
+node = None
+doc = self.state.document
+
+if hasattr(doc, "findall"):
+# docutils >= 0.18.1
+for node in doc.findall(addnodes.highlightlang):
+pass
+else:
+for elem in doc.traverse():
+if isinstance(elem, addnodes.highlightlang):
+node = elem
+
+if node:
+return node
+
+# No explicit directive found, use defaults
+node = addnodes.highlightlang(
+lang=self.env.config.highlight_language,
+force=False,
+# Yes, Sphinx uses this value to effectively disable line
+# numbers and not 0 or None or -1 or something. ¯\_(ツ)_/¯
+linenothreshold=sys.maxsize,
+)
+return node
+
def admonition_wrap(self, *content) -> List[nodes.Node]:
title = "Example:"
if "title" in self.options:
@@ -576,8 +605,24 @@ def admonition_wrap(self, *content) -> List[nodes.Node]:
return [admon]
def run_annotated(self) -> List[nodes.Node]:
+lang_node = self._highlightlang()
+
content_node: nodes.Element = nodes.section()
+
+# Configure QMP highlighting for "::" blocks, if needed
+if lang_node["lang"] != "QMP":
+content_node += addnodes.highlightlang(
+lang="QMP",
+force=False, # "True" ignores lexing errors
+linenothreshold=lang_node["linenothreshold"],
+)
+
self.do_parse(self.content, content_node)
+
+# Restore prior language highlighting, if needed
+if lang_node["lang"] != "QMP":
+content_node += addnodes.highlightlang(**lang_node.attributes)
+
return content_node.children
def run(self) -> List[nodes.Node]:
--
2.45.0
[PATCH v2 2/9] docs/qapidoc: factor out do_parse()
Factor out the compatibility parser helper into a base class, so it can be shared by other directives. Signed-off-by: John Snow Reviewed-by: Markus Armbruster --- docs/sphinx/qapidoc.py | 32 +++- 1 file changed, 19 insertions(+), 13 deletions(-) diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py index 62b39833ca0..b3be82998a8 100644 --- a/docs/sphinx/qapidoc.py +++ b/docs/sphinx/qapidoc.py @@ -481,7 +481,25 @@ def visit_module(self, name): super().visit_module(name) -class QAPIDocDirective(Directive): +class NestedDirective(Directive): +def run(self): +raise NotImplementedError + +def do_parse(self, rstlist, node): +""" +Parse rST source lines and add them to the specified node + +Take the list of rST source lines rstlist, parse them as +rST, and add the resulting docutils nodes as children of node. +The nodes are parsed in a way that allows them to include +subheadings (titles) without confusing the rendering of +anything else. +""" +with switch_source_input(self.state, rstlist): +nested_parse_with_titles(self.state, rstlist, node) + + +class QAPIDocDirective(NestedDirective): """Extract documentation from the specified QAPI .json file""" required_argument = 1 @@ -519,18 +537,6 @@ def run(self): # so they are displayed nicely to the user raise ExtensionError(str(err)) from err -def do_parse(self, rstlist, node): -"""Parse rST source lines and add them to the specified node - -Take the list of rST source lines rstlist, parse them as -rST, and add the resulting docutils nodes as children of node. -The nodes are parsed in a way that allows them to include -subheadings (titles) without confusing the rendering of -anything else. -""" -with switch_source_input(self.state, rstlist): -nested_parse_with_titles(self.state, rstlist, node) - def setup(app): """Register qapi-doc directive with Sphinx""" -- 2.45.0
[PATCH v2 5/9] docs/sphinx: add CSS styling for qmp-example directive
From: Harmonie Snow
Add CSS styling for qmp-example directives to increase readability and
consistently style all example blocks.
Signed-off-by: Harmonie Snow
Signed-off-by: John Snow
Acked-by: Markus Armbruster
---
docs/sphinx-static/theme_overrides.css | 49 ++
1 file changed, 49 insertions(+)
diff --git a/docs/sphinx-static/theme_overrides.css
b/docs/sphinx-static/theme_overrides.css
index c70ef951286..965ecac54fd 100644
--- a/docs/sphinx-static/theme_overrides.css
+++ b/docs/sphinx-static/theme_overrides.css
@@ -87,6 +87,55 @@ div[class^="highlight"] pre {
padding-bottom: 1px;
}
+/* qmp-example directive styling */
+
+.rst-content .admonition-example {
+/* do not apply the standard admonition background */
+background-color: transparent;
+border: solid #ffd2ed 1px;
+}
+
+.rst-content .admonition-example > .admonition-title:before {
+content: "▷";
+}
+
+.rst-content .admonition-example > .admonition-title {
+background-color: #5980a6;
+}
+
+.rst-content .admonition-example > div[class^="highlight"] {
+/* make code boxes take up the full width of the admonition w/o margin */
+margin-left: -12px;
+margin-right: -12px;
+
+border-top: 1px solid #ffd2ed;
+border-bottom: 1px solid #ffd2ed;
+border-left: 0px;
+border-right: 0px;
+}
+
+.rst-content .admonition-example > div[class^="highlight"]:nth-child(2) {
+/* If a code box is the second element in an example admonition,
+ * it is the first child after the title. let it sit flush against
+ * the title. */
+margin-top: -12px;
+border-top: 0px;
+}
+
+.rst-content .admonition-example > div[class^="highlight"]:last-child {
+/* If a code box is the final element in an example admonition, don't
+ * render margin below it; let it sit flush with the end of the
+ * admonition box */
+margin-bottom: -12px;
+border-bottom: 0px;
+}
+
+.rst-content .admonition-example .highlight {
+background-color: #fffafd;
+}
+
+/* end qmp-example styling */
+
@media screen {
/* content column
--
2.45.0
[PATCH v2 0/9] qapi: convert example sections to qmp-example rST directives
This patchset focuses on converting example sections to rST directives using a new `.. qmp-example::` directive. V2: - Rebased on origin/master; converted one more example - Fixed (most?) minor nits from last review - Didn't address lack of newline in text mode or enhanced lexer (yet) Changes since this was split out from the prior series: - Harmonie updated the CSS for the example block section. I think it's really tidy now! Thanks Harmonie! - Dependence on SphinxDirective was removed, but it will likely re-appear in the next series anyway. - qapi-code-gen.rst was updated with a section on how to write examples. - Various minor tweaks to comments, commit messages, docs, etc. Harmonie Snow (1): docs/sphinx: add CSS styling for qmp-example directive John Snow (8): [DO-NOT-MERGE]: Add some ad-hoc linting helpers. docs/qapidoc: factor out do_parse() docs/qapidoc: create qmp-example directive docs/qapidoc: add QMP highlighting to annotated qmp-example blocks qapi: convert "Example" sections without titles qapi: convert "Example" sections with titles qapi: convert "Example" sections with longer prose qapi: remove "Example" doc section docs/devel/qapi-code-gen.rst | 58 +-- docs/sphinx-static/theme_overrides.css | 49 ++ docs/sphinx/qapidoc.py | 130 ++--- qapi/acpi.json | 4 +- qapi/block-core.json | 88 + qapi/block.json| 57 ++- qapi/char.json | 24 +++-- qapi/control.json | 8 +- qapi/dump.json | 8 +- qapi/machine-target.json | 2 +- qapi/machine.json | 68 +++-- qapi/migration.json| 90 + qapi/misc-target.json | 22 ++--- qapi/misc.json | 32 +++--- qapi/net.json | 22 +++-- qapi/pci.json | 2 +- qapi/qdev.json | 10 +- qapi/qom.json | 16 +-- qapi/replay.json | 8 +- qapi/rocker.json | 8 +- qapi/run-state.json| 32 +++--- qapi/tpm.json | 6 +- qapi/trace.json| 4 +- qapi/transaction.json | 2 +- qapi/ui.json | 45 - qapi/vfio.json | 2 +- qapi/virtio.json | 45 ++--- qapi/yank.json | 4 +- scripts/qapi-lint.sh | 61 scripts/qapi/Makefile | 5 + scripts/qapi/parser.py | 10 +- tests/qapi-schema/doc-good.json| 19 ++-- tests/qapi-schema/doc-good.out | 26 +++-- tests/qapi-schema/doc-good.txt | 23 ++--- 34 files changed, 662 insertions(+), 328 deletions(-) create mode 100755 scripts/qapi-lint.sh create mode 100644 scripts/qapi/Makefile -- 2.45.0
[PULL 5/6] Python: bump minimum sphinx version to 3.4.3
With RHEL 8 support retired (It's been two years since RHEL9 released),
our very oldest build platform version of Sphinx is now 3.4.3; and
keeping backwards compatibility for versions as old as v1.6 when using
domain extensions is a lot of work we don't need to do.
This patch is motivated by my work creating a new QAPI domain, which
unlike the dbus documentation, cannot be allowed to regress by creating
a "dummy" doc when operating under older sphinx versions. Easier is to
raise our minimum version as far as we can push it forwards, reducing my
burden in creating cross-compatibility hacks and patches.
A sampling of sphinx versions from various distributions, courtesy
https://repology.org/project/python:sphinx/versions
Alpine 3.16: v4.3.0 (QEMU support ended 2024-05-23)
Alpine 3.17: v5.3.0
Alpine 3.18: v6.1.3
Alpine 3.19: v6.2.1
Ubuntu 20.04 LTS: EOL
Ubuntu 22.04 LTS: v4.3.2
Ubuntu 22.10: EOL
Ubuntu 23.04: EOL
Ubuntu 23.10: v5.3.0
Ubuntu 24.04 LTS: v7.2.6
Debian 11: v3.4.3 (QEMU support ends 2024-07-xx)
Debian 12: v5.3.0
Fedora 38: EOL
Fedora 39: v6.2.1
Fedora 40: v7.2.6
CentOS Stream 8: v1.7.6 (QEMU support ended 2024-05-17)
CentOS Stream 9: v3.4.3
OpenSUSE Leap 15.4: EOL
OpenSUSE Leap 15.5: 2.3.1, 4.2.0 and 7.2.6
RHEL9 / CentOS Stream 9 becomes the new defining factor in staying at
Sphinx 3.4.3 due to downstream offline build requirements that force us
to use platform Sphinx instead of newer packages from PyPI.
Signed-off-by: John Snow
Acked-by: Paolo Bonzini
Acked-by: Markus Armbruster
Reviewed-by: Thomas Huth
Message-id: 20240703175235.239004-2-js...@redhat.com
Signed-off-by: John Snow
---
docs/conf.py| 7 +++
pythondeps.toml | 2 +-
2 files changed, 4 insertions(+), 5 deletions(-)
diff --git a/docs/conf.py b/docs/conf.py
index aae0304ac6e..876f6768815 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -53,10 +53,9 @@
# If your documentation needs a minimal Sphinx version, state it here.
#
-# Sphinx 1.5 and earlier can't build our docs because they are too
-# picky about the syntax of the argument to the option:: directive
-# (see Sphinx bugs #646, #3366).
-needs_sphinx = '1.6'
+# 3.4.3 is the oldest version of Sphinx that ships on a platform we
+# pledge build support for.
+needs_sphinx = '3.4.3'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
diff --git a/pythondeps.toml b/pythondeps.toml
index 6aba0c9daaa..f6e590fdd86 100644
--- a/pythondeps.toml
+++ b/pythondeps.toml
@@ -23,7 +23,7 @@ meson = { accepted = ">=1.1.0", installed = "1.2.3", canary =
"meson" }
[docs]
# Please keep the installed versions in sync with docs/requirements.txt
-sphinx = { accepted = ">=1.6", installed = "5.3.0", canary = "sphinx-build" }
+sphinx = { accepted = ">=3.4.3", installed = "5.3.0", canary = "sphinx-build" }
sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.1.1" }
[avocado]
--
2.45.0
[PULL 6/6] docs: remove Sphinx 1.x compatibility code
In general, the Use_SSI workaround is no longer needed, and neither is
the pre-1.6 logging shim for kerneldoc.
Signed-off-by: John Snow
Acked-by: Markus Armbruster
Reviewed-by: Thomas Huth
Message-id: 20240703175235.239004-3-js...@redhat.com
[rebased on top of origin/master. --js]
Signed-off-by: John Snow
---
docs/sphinx/hxtool.py| 21 -
docs/sphinx/kerneldoc.py | 38 --
docs/sphinx/kernellog.py | 28
docs/sphinx/qapidoc.py | 33 +++--
4 files changed, 19 insertions(+), 101 deletions(-)
delete mode 100644 docs/sphinx/kernellog.py
diff --git a/docs/sphinx/hxtool.py b/docs/sphinx/hxtool.py
index 3729084a36c..a84723be19e 100644
--- a/docs/sphinx/hxtool.py
+++ b/docs/sphinx/hxtool.py
@@ -24,16 +24,10 @@
from docutils.statemachine import ViewList
from docutils.parsers.rst import directives, Directive
from sphinx.errors import ExtensionError
+from sphinx.util.docutils import switch_source_input
from sphinx.util.nodes import nested_parse_with_titles
import sphinx
-# Sphinx up to 1.6 uses AutodocReporter; 1.7 and later
-# use switch_source_input. Check borrowed from kerneldoc.py.
-Use_SSI = sphinx.__version__[:3] >= '1.7'
-if Use_SSI:
-from sphinx.util.docutils import switch_source_input
-else:
-from sphinx.ext.autodoc import AutodocReporter
__version__ = '1.0'
@@ -185,16 +179,9 @@ def run(self):
# of title_styles and section_level that kerneldoc.py does,
# because nested_parse_with_titles() does that for us.
def do_parse(self, result, node):
-if Use_SSI:
-with switch_source_input(self.state, result):
-nested_parse_with_titles(self.state, result, node)
-else:
-save = self.state.memo.reporter
-self.state.memo.reporter = AutodocReporter(result,
self.state.memo.reporter)
-try:
-nested_parse_with_titles(self.state, result, node)
-finally:
-self.state.memo.reporter = save
+with switch_source_input(self.state, result):
+nested_parse_with_titles(self.state, result, node)
+
def setup(app):
""" Register hxtool-doc directive with Sphinx"""
diff --git a/docs/sphinx/kerneldoc.py b/docs/sphinx/kerneldoc.py
index 72c403a7379..3aa972f2e89 100644
--- a/docs/sphinx/kerneldoc.py
+++ b/docs/sphinx/kerneldoc.py
@@ -38,20 +38,14 @@
from docutils.statemachine import ViewList
from docutils.parsers.rst import directives, Directive
-#
-# AutodocReporter is only good up to Sphinx 1.7
-#
import sphinx
+from sphinx.util import logging
+from sphinx.util.docutils import switch_source_input
-Use_SSI = sphinx.__version__[:3] >= '1.7'
-if Use_SSI:
-from sphinx.util.docutils import switch_source_input
-else:
-from sphinx.ext.autodoc import AutodocReporter
-
-import kernellog
__version__ = '1.0'
+logger = logging.getLogger('kerneldoc')
+
class KernelDocDirective(Directive):
"""Extract kernel-doc comments from the specified file"""
@@ -111,8 +105,7 @@ def run(self):
cmd += [filename]
try:
-kernellog.verbose(env.app,
- 'calling kernel-doc \'%s\'' % (" ".join(cmd)))
+logger.verbose('calling kernel-doc \'%s\'' % (" ".join(cmd)))
p = subprocess.Popen(cmd, stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
out, err = p.communicate()
@@ -122,8 +115,10 @@ def run(self):
if p.returncode != 0:
sys.stderr.write(err)
-kernellog.warn(env.app,
- 'kernel-doc \'%s\' failed with return code %d'
% (" ".join(cmd), p.returncode))
+logger.warning(
+'kernel-doc \'%s\' failed with return code %d' %
+(" ".join(cmd), p.returncode)
+)
return [nodes.error(None, nodes.paragraph(text = "kernel-doc
missing"))]
elif env.config.kerneldoc_verbosity > 0:
sys.stderr.write(err)
@@ -149,22 +144,13 @@ def run(self):
return node.children
except Exception as e: # pylint: disable=W0703
-kernellog.warn(env.app, 'kernel-doc \'%s\' processing failed with:
%s' %
+logger.warning('kernel-doc \'%s\' processing failed with: %s' %
(" ".join(cmd), str(e)))
return [nodes.error(None, nodes.paragraph(text = "kernel-doc
missing"))]
def do_parse(self, result, node):
-if Use_SSI:
-with switch_source_input(self.state, result):
-se
[PULL 2/6] python: Do not use pylint 3.2.4 with python 3.8
There is a bug in this version, see: https://github.com/pylint-dev/pylint/issues/9751 Signed-off-by: John Snow Reviewed-by: Alex Bennée Message-id: 20240626232230.408004-3-js...@redhat.com Signed-off-by: John Snow --- python/setup.cfg | 1 + 1 file changed, 1 insertion(+) diff --git a/python/setup.cfg b/python/setup.cfg index 48668609d3e..8ebd345d7ed 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -41,6 +41,7 @@ devel = isort >= 5.1.2 mypy >= 1.4.0 pylint >= 2.17.3 +pylint != 3.2.4; python_version<"3.9" tox >= 3.18.0 urwid >= 2.1.2 urwid-readline >= 0.13 -- 2.45.0
[PULL 1/6] python: linter changes for pylint 3.x
New bleeding edge versions, new nits to iron out. This addresses the 'check-python-tox' optional GitLab test, while 'check-python-minreqs' saw no regressions, since it's frozen on an older version of pylint. Fixes: qemu/machine/machine.py:345:52: E0606: Possibly using variable 'sock' before assignment (possibly-used-before-assignment) qemu/utils/qemu_ga_client.py:168:4: R1711: Useless return at end of function or method (useless-return) Signed-off-by: John Snow Reviewed-by: Alex Bennée Message-id: 20240626232230.408004-2-js...@redhat.com Signed-off-by: John Snow --- python/qemu/machine/machine.py | 1 + python/qemu/utils/qemu_ga_client.py | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index f648f6af451..ebb58d5b68c 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -335,6 +335,7 @@ def binary(self) -> str: def _pre_launch(self) -> None: if self._qmp_set: +sock = None if self._monitor_address is None: self._sock_pair = socket.socketpair() os.set_inheritable(self._sock_pair[0].fileno(), True) diff --git a/python/qemu/utils/qemu_ga_client.py b/python/qemu/utils/qemu_ga_client.py index 9a665e6e990..cf0fcf9a8bb 100644 --- a/python/qemu/utils/qemu_ga_client.py +++ b/python/qemu/utils/qemu_ga_client.py @@ -174,7 +174,7 @@ def suspend(self, mode: str) -> None: # On error exception will raise except asyncio.TimeoutError: # On success command will timed out -return +pass def shutdown(self, mode: str = 'powerdown') -> None: if mode not in ['powerdown', 'halt', 'reboot']: -- 2.45.0
[PULL 3/6] iotests: Change imports for Python 3.13
Python 3.13 isn't out yet, but it's in beta and Fedora is ramping up to make it the default system interpreter for Fedora 41. They moved our cheese for where ContextManager lives; add a conditional to locate it while we support both pre-3.9 and 3.13+. Signed-off-by: John Snow Message-id: 20240626232230.408004-4-js...@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/testenv.py| 7 ++- tests/qemu-iotests/testrunner.py | 9 ++--- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py index 588f30a4f14..96d69e56963 100644 --- a/tests/qemu-iotests/testenv.py +++ b/tests/qemu-iotests/testenv.py @@ -25,7 +25,12 @@ import random import subprocess import glob -from typing import List, Dict, Any, Optional, ContextManager +from typing import List, Dict, Any, Optional + +if sys.version_info >= (3, 9): +from contextlib import AbstractContextManager as ContextManager +else: +from typing import ContextManager DEF_GDB_OPTIONS = 'localhost:12345' diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py index 7b322272e92..2e236c8fa39 100644 --- a/tests/qemu-iotests/testrunner.py +++ b/tests/qemu-iotests/testrunner.py @@ -27,11 +27,14 @@ import shutil import sys from multiprocessing import Pool -from typing import List, Optional, Any, Sequence, Dict, \ -ContextManager - +from typing import List, Optional, Any, Sequence, Dict from testenv import TestEnv +if sys.version_info >= (3, 9): +from contextlib import AbstractContextManager as ContextManager +else: +from typing import ContextManager + def silent_unlink(path: Path) -> None: try: -- 2.45.0
[PULL 4/6] python: enable testing for 3.13
Python 3.13 is in beta and Fedora 41 is preparing to make it the default system interpreter; enable testing for it. (In the event problems develop prior to release, it should only impact the check-python-tox job, which is not run by default and is allowed to fail.) Signed-off-by: John Snow Reviewed-by: Alex Bennée Tested-by: Alex Bennée Message-id: 20240626232230.408004-5-js...@redhat.com Signed-off-by: John Snow --- python/setup.cfg | 3 ++- tests/docker/dockerfiles/python.docker | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/python/setup.cfg b/python/setup.cfg index 8ebd345d7ed..3b4e2cc5501 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -19,6 +19,7 @@ classifiers = Programming Language :: Python :: 3.10 Programming Language :: Python :: 3.11 Programming Language :: Python :: 3.12 +Programming Language :: Python :: 3.13 Typing :: Typed [options] @@ -184,7 +185,7 @@ multi_line_output=3 # of python available on your system to run this test. [tox:tox] -envlist = py38, py39, py310, py311, py312 +envlist = py38, py39, py310, py311, py312, py313 skip_missing_interpreters = true [testenv] diff --git a/tests/docker/dockerfiles/python.docker b/tests/docker/dockerfiles/python.docker index a3c1321190c..8f0af9ef25f 100644 --- a/tests/docker/dockerfiles/python.docker +++ b/tests/docker/dockerfiles/python.docker @@ -14,6 +14,7 @@ ENV PACKAGES \ python3.10 \ python3.11 \ python3.12 \ +python3.13 \ python3.8 \ python3.9 -- 2.45.0
[PULL 0/6] Python patches
The following changes since commit 4469bee2c529832d762af4a2f89468c926f02fe4: Merge tag 'nvme-next-pull-request' of https://gitlab.com/birkelund/qemu into staging (2024-07-11 14:32:51 -0700) are available in the Git repository at: https://gitlab.com/jsnow/qemu.git tags/python-pull-request for you to fetch changes up to dd23f9ec519db9c424223cff8767715de5532718: docs: remove Sphinx 1.x compatibility code (2024-07-12 16:46:21 -0400) Python: 3.13 compat & sphinx minver bump ---- John Snow (6): python: linter changes for pylint 3.x python: Do not use pylint 3.2.4 with python 3.8 iotests: Change imports for Python 3.13 python: enable testing for 3.13 Python: bump minimum sphinx version to 3.4.3 docs: remove Sphinx 1.x compatibility code docs/conf.py | 7 ++--- docs/sphinx/hxtool.py | 21 +++--- docs/sphinx/kerneldoc.py | 38 -- docs/sphinx/kernellog.py | 28 --- docs/sphinx/qapidoc.py | 33 ++ python/qemu/machine/machine.py | 1 + python/qemu/utils/qemu_ga_client.py| 2 +- python/setup.cfg | 4 ++- pythondeps.toml| 2 +- tests/docker/dockerfiles/python.docker | 1 + tests/qemu-iotests/testenv.py | 7 - tests/qemu-iotests/testrunner.py | 9 -- 12 files changed, 41 insertions(+), 112 deletions(-) delete mode 100644 docs/sphinx/kernellog.py -- 2.45.0
Re: [PATCH 8/8] qapi: remove "Example" doc section
On Wed, Jul 10, 2024 at 1:45 AM Markus Armbruster wrote:
> John Snow writes:
>
> > On Tue, Jul 9, 2024 at 6:52 AM Markus Armbruster
> wrote:
> >
> >> John Snow writes:
> >>
> >> > Fully eliminate the "Example" sections in QAPI doc blocks now that
> they
> >> > have all been converted to arbitrary rST syntax using the
> >> > ".. qmp-example::" directive. Update tests to match.
> >> >
> >> > Migrating to the new syntax
> >> > ---
> >> >
> >> > The old "Example:" or "Examples:" section syntax is now caught as an
> >> > error, but "Example::" is stil permitted as explicit rST syntax for an
> >> > un-lexed, generic preformatted text block.
> >> >
> >> > ('Example' is not special in this case, any sentence that ends with
> "::"
> >> > will start an indented code block in rST.)
> >> >
> >> > Arbitrary rST for Examples is now possible, but it's strongly
> >> > recommended that documentation authors use the ".. qmp-example::"
> >> > directive for consistent visual formatting in rendered HTML docs. The
> >> > ":title:" directive option may be used to add extra information into
> the
> >> > title bar for the example. The ":annotated:" option can be used to
> write
> >> > arbitrary rST instead, with nested "::" blocks applying QMP formatting
> >> > where desired.
> >> >
> >> > Other choices available are ".. code-block:: QMP" which will not
> create
> >> > an "Example:" box, or the short-form "::" code-block syntax which will
> >> > not apply QMP highlighting when used outside of the qmp-example
> >> > directive.
> >> >
> >> > Why?
> >> >
> >> >
> >> > This patch has several benefits:
> >> >
> >> > 1. Example sections can now be written more arbitrarily, mixing
> >> >explanatory paragraphs and code blocks however desired.
> >> >
> >> > 2. Example sections can now use fully arbitrary rST.
> >> >
> >> > 3. All code blocks are now lexed and validated as QMP; increasing
> >> >usability of the docs and ensuring validity of example snippets.
> >> >
> >> >(To some extent - This patch only gaurantees it lexes correctly,
> not
> >> >that it's valid under the JSON or QMP grammars. It will catch most
> >> >small mistakes, however.)
> >> >
> >> > 4. Each qmp-example can be titled or annotated independently without
> >> >bypassing the QMP lexer/validator.
> >> >
> >> >(i.e. code blocks are now for *code* only, so we don't have to
> >> >sacrifice exposition for having lexically valid examples.)
> >> >
> >> > NOTE: As with the "Notes" conversion patch,
> >>
> >> Commit d461c279737 (qapi: convert "Note" sections to plain rST).
> >>
> >
> > Didn't have a stable commit ID at the time, will work it in if/when the
> > notes patches hit main.
>
> They have.
>
> >> > this patch (and those
> >> > preceding) may change the rendering order for Examples in the
> >>
> >> The three preceding ones, to be precise.
> >>
> >> > current generator. The forthcoming qapidoc rewrite will fix this
> >> > by always generating documentation in source order.
> >>
> >> Conversions from "Example" section to plain reST may change order. This
> >> patch converts a test, and the preceding three convert the real uses.
> >>
> >> Does any of the patches actually change order?
> >
> > I do not actually know ...! It has the *potential* in the same exact way
> > that the notes patch did, but I don't actually know if it *did*. My hunch
> > is "no" because there's only one intermediate section we identified with
> > the notes series, but I didn't exhaustively prove it. That's why I used
> the
> > "may" weasel wording.
>
> Alright, I checked.
>
> In documentation of command announce-self, the example moves from after
> the arguments to before. Unwanted change.
>
> I can keep it in place if I insert a TODO before the exam
Re: [PATCH 8/8] qapi: remove "Example" doc section
On Tue, Jul 9, 2024 at 6:52 AM Markus Armbruster wrote:
> John Snow writes:
>
> > Fully eliminate the "Example" sections in QAPI doc blocks now that they
> > have all been converted to arbitrary rST syntax using the
> > ".. qmp-example::" directive. Update tests to match.
> >
> > Migrating to the new syntax
> > ---
> >
> > The old "Example:" or "Examples:" section syntax is now caught as an
> > error, but "Example::" is stil permitted as explicit rST syntax for an
> > un-lexed, generic preformatted text block.
> >
> > ('Example' is not special in this case, any sentence that ends with "::"
> > will start an indented code block in rST.)
> >
> > Arbitrary rST for Examples is now possible, but it's strongly
> > recommended that documentation authors use the ".. qmp-example::"
> > directive for consistent visual formatting in rendered HTML docs. The
> > ":title:" directive option may be used to add extra information into the
> > title bar for the example. The ":annotated:" option can be used to write
> > arbitrary rST instead, with nested "::" blocks applying QMP formatting
> > where desired.
> >
> > Other choices available are ".. code-block:: QMP" which will not create
> > an "Example:" box, or the short-form "::" code-block syntax which will
> > not apply QMP highlighting when used outside of the qmp-example
> > directive.
> >
> > Why?
> >
> >
> > This patch has several benefits:
> >
> > 1. Example sections can now be written more arbitrarily, mixing
> >explanatory paragraphs and code blocks however desired.
> >
> > 2. Example sections can now use fully arbitrary rST.
> >
> > 3. All code blocks are now lexed and validated as QMP; increasing
> >usability of the docs and ensuring validity of example snippets.
> >
> >(To some extent - This patch only gaurantees it lexes correctly, not
> >that it's valid under the JSON or QMP grammars. It will catch most
> >small mistakes, however.)
> >
> > 4. Each qmp-example can be titled or annotated independently without
> >bypassing the QMP lexer/validator.
> >
> >(i.e. code blocks are now for *code* only, so we don't have to
> >sacrifice exposition for having lexically valid examples.)
> >
> > NOTE: As with the "Notes" conversion patch,
>
> Commit d461c279737 (qapi: convert "Note" sections to plain rST).
>
Didn't have a stable commit ID at the time, will work it in if/when the
notes patches hit main.
>
> > this patch (and those
> > preceding) may change the rendering order for Examples in the
>
> The three preceding ones, to be precise.
>
> > current generator. The forthcoming qapidoc rewrite will fix this
> > by always generating documentation in source order.
>
> Conversions from "Example" section to plain reST may change order. This
> patch converts a test, and the preceding three convert the real uses.
>
> Does any of the patches actually change order?
>
I do not actually know ...! It has the *potential* in the same exact way
that the notes patch did, but I don't actually know if it *did*. My hunch
is "no" because there's only one intermediate section we identified with
the notes series, but I didn't exhaustively prove it. That's why I used the
"may" weasel wording.
>
> > Signed-off-by: John Snow
> > ---
> > docs/devel/qapi-code-gen.rst| 58 -
> > scripts/qapi/parser.py | 10 +-
> > tests/qapi-schema/doc-good.json | 19 +++
> > tests/qapi-schema/doc-good.out | 26 ++-
> > tests/qapi-schema/doc-good.txt | 23 ++---
> > 5 files changed, 98 insertions(+), 38 deletions(-)
> >
> > diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
> > index ae97b335cbf..2e10a3cbd69 100644
> > --- a/docs/devel/qapi-code-gen.rst
> > +++ b/docs/devel/qapi-code-gen.rst
> > @@ -899,7 +899,7 @@ Documentation markup
> >
> >
> > Documentation comments can use most rST markup. In particular,
> > -a ``::`` literal block can be used for examples::
> > +a ``::`` literal block can be used for pre-formatted text::
> >
> > # ::
> > #
> > @@ -995,8 +995,8 @@ line "Features:", like this::
> &g
Re: [PATCH 7/8] qapi: convert "Example" sections with longer prose
On Tue, Jul 9, 2024 at 7:35 AM Markus Armbruster wrote:
> John Snow writes:
>
> > These examples require longer explanations or have explanations that
> > require markup to look reasonable when rendered and so use the longer
> > form of the ".. qmp-example::" directive.
> >
> > By using the :annotated: option, the content in the example block is
> > assumed *not* to be a code block literal and is instead parsed as normal
> > rST - with the exception that any code literal blocks after `::` will
> > assumed to be a QMP code literal block.
> >
> > Note: There's one title-less conversion in this patch that comes along
> > for the ride because it's part of a larger "Examples" block that was
> > better to convert all at once.
> >
> > See commit-5: "docs/qapidoc: create qmp-example directive", for a
> > detailed explanation of this custom directive syntax.
> >
> > See commit+1: "qapi: remove "Example" doc section" for a detailed
> > explanation of why.
> >
> > Signed-off-by: John Snow
> > ---
> > qapi/block.json | 26 --
> > qapi/machine.json | 30 --
> > qapi/migration.json | 7 +--
> > qapi/virtio.json| 24 ++--
> > 4 files changed, 59 insertions(+), 28 deletions(-)
> >
> > diff --git a/qapi/block.json b/qapi/block.json
> > index 5ddd061e964..d95e9fd8140 100644
> > --- a/qapi/block.json
> > +++ b/qapi/block.json
> > @@ -545,31 +545,37 @@
> > #
> > # Since: 4.0
> > #
> > -# Example:
> > +# .. qmp-example::
> > +#:annotated:
> > #
> > -# Set new histograms for all io types with intervals
> > -# [0, 10), [10, 50), [50, 100), [100, +inf):
> > +#Set new histograms for all io types with intervals
> > +#[0, 10), [10, 50), [50, 100), [100, +inf)::
> > #
> > # -> { "execute": "block-latency-histogram-set",
> > # "arguments": { "id": "drive0",
> > # "boundaries": [10, 50, 100] } }
> > # <- { "return": {} }
> > #
> > -# Example:
> > +# .. qmp-example::
> > +#:annotated:
> > #
> > -# Set new histogram only for write, other histograms will remain
> > -# not changed (or not created):
> > +#Set new histogram only for write, other histograms will remain
> > +#not changed (or not created)::
> > #
> > # -> { "execute": "block-latency-histogram-set",
> > # "arguments": { "id": "drive0",
> > # "boundaries-write": [10, 50, 100] } }
> > # <- { "return": {} }
> > #
> > -# Example:
> > +# .. qmp-example::
> > +#:annotated:
> > #
> > -# Set new histograms with the following intervals:
> > -# read, flush: [0, 10), [10, 50), [50, 100), [100, +inf)
> > -# write: [0, 1000), [1000, 5000), [5000, +inf)
> > +#Set new histograms with the following intervals:
> > +#
> > +#- read, flush: [0, 10), [10, 50), [50, 100), [100, +inf)
> > +#- write: [0, 1000), [1000, 5000), [5000, +inf)
> > +#
> > +#::
> > #
> > # -> { "execute": "block-latency-histogram-set",
> > # "arguments": { "id": "drive0",
># "boundaries": [10, 50, 100],
># "boundaries-write": [1000, 5000] } }
># <- { "return": {} }
>#
># .. qmp-example::
>#:title: Remove all latency histograms
>#
># -> { "execute": "block-latency-histogram-set",
># "arguments": { "id": "drive0" } }
># <- { "return": {} }
>##
>
> I think using :annotated: for this one as well will look better.
>
Sure, why not. I tried to minimize more complex rewrites as a rule, but
it's no problem to change the styling to taste.
>
> [...]
>
> Reviewed-by: Markus Armbruster
>
>
Re: [PATCH 4/8] docs/sphinx: add CSS styling for qmp-example directive
On Tue, Jul 9, 2024 at 6:34 AM Markus Armbruster wrote: > John Snow writes: > > > From: Harmonie Snow > > > > Add CSS styling for qmp-example directives to increase readability and > > consistently style all example blocks. > > > > Signed-off-by: Harmonie Snow > > Signed-off-by: John Snow > > Same sadness as for the previous patch. > Should we do anything about that? In the long run, I don't expect anyone will actually need to care about what this directive looked like in some intermediate state before we ever used it. If you want to evaluate the directive in the in-between states, I recommend modifying a document and seeing what it does; but I didn't really intend for anyone to really see it that way. (Isn't it a bit overboard to write unit tests for intermediate tree states...?) > > Acked-by: Markus Armbruster > >
Re: [PATCH 3/8] docs/qapidoc: add QMP highlighting to annotated qmp-example blocks
On Tue, Jul 9, 2024 at 6:33 AM Markus Armbruster wrote:
> John Snow writes:
>
> > For any code literal blocks inside of a qmp-example directive, apply and
> > enforce the QMP lexer/highlighter to those blocks.
> >
> > This way, you won't need to write:
> >
> > ```
> > .. qmp-example::
> >:annotated:
> >
> >Blah blah
> >
> >.. code-block:: QMP
> >
> > -> { "lorem": "ipsum" }
> > ```
> >
> > But instead, simply:
> >
> > ```
> > .. qmp-example::
> >:annotated:
> >
> >Blah blah::
> >
> > -> { "lorem": "ipsum" }
> > ```
> >
> > Once the directive block is exited, whatever the previous default
> > highlight language was will be restored; localizing the forced QMP
> > lexing to exclusively this directive.
> >
> > Note, if the default language is *already* QMP, this directive will not
> > generate and restore redundant highlight configuration nodes. We may
> > well decide that the default language ought to be QMP for any QAPI
> > reference pages, but this way the directive behaves consistently no
> > matter where it is used.
> >
> > Signed-off-by: John Snow
>
> Sadly, the previous patch didn't add tests, so this patch's effect
> isn't as easy to observe as it could be.
>
Sorry O:-)
In truth, I never intended it to *not* have this feature, but thought it
was helpful to split out just the code responsible for this feature into
its own patch so its soul could be independently judged to see if it was
lighter than a feather.
(i.e., does the convenience justify the SLOC?)
>
> Acked-by: Markus Armbruster
>
Re: [PATCH 1/8] docs/qapidoc: factor out do_parse()
On Sat, Jul 6, 2024, 10:47 AM Markus Armbruster wrote: > John Snow writes: > > > Factor out the compatibility parser helper into a base class, so it can > > be shared by other directives. > > > > Signed-off-by: John Snow > > R-by stands. > Assuming true even if I rebase on top of the 3.x patches and do_parse() becomes quite a bit more trivial? --js >
Re: [PATCH 5/8] qapi: convert "Example" sections without titles
On Sat, Jul 6, 2024, 10:42 AM Markus Armbruster wrote:
> John Snow writes:
>
> > Use the no-option form of ".. qmp-example::" to convert any Examples
> > that do not have any form of caption or explanation whatsoever. Note
> > that in a few cases, example sections are split into two or more
> > separate example blocks. This is only done stylistically to create a
> > delineation between two or more logically independent examples.
> >
> > See commit-3: "docs/qapidoc: create qmp-example directive", for a
> > detailed explanation of this custom directive syntax.
> >
> > See commit+3: "qapi: remove "Example" doc section" for a detailed
> > explanation of why.
> >
> > Signed-off-by: John Snow
>
> [...]
>
> > diff --git a/qapi/run-state.json b/qapi/run-state.json
> > index 252d7d6afa7..718a3c958e9 100644
> > --- a/qapi/run-state.json
> > +++ b/qapi/run-state.json
>
> [...]
>
> > @@ -453,7 +453,7 @@
> > #
> > # Since: 5.0
> > #
> > -# Example:
> > +# .. qmp-example::
> > #
> > # <- { "event": "GUEST_CRASHLOADED",
> > # "data": { "action": "run" },
>
> Trivial semantic conflict, we need
>
Caught on rebase late Fri, already fixed locally and will be in v2 (which I
rebased on top of my sphinx 3.x patches, which change the do_parse() stuff
too.)
> @@ -469,7 +469,7 @@
>#
># Since: 9.1
>#
> -# Example:
> +# .. qmp-example::
>#
># <- { "event": "GUEST_PVSHUTDOWN",
># "timestamp": { "seconds": 1648245259, "microseconds": 893771
> } }
>
>
> > @@ -597,7 +597,7 @@
> > #
> > # Since: 5.2
> > #
> > -# Example:
> > +# .. qmp-example::
> > #
> > # <- { "event": "MEMORY_FAILURE",
> > # "data": { "recipient": "hypervisor",
>
> [...]
>
>
Re: [PATCH 0/4] Python: Add 3.13 support, play linter whackamole
On Wed, Jun 26, 2024, 7:22 PM John Snow wrote: > Fix some regressions in check-python-tox that have crept in since Pylint > 3.x, and add Python 3.13 support to the pipeline. > > GitLab pipeline (before I fixed the missing DCO, but let's be honest, it > can't possibly be worth re-running so many tests for just that): > https://gitlab.com/jsnow/qemu/-/pipelines/1349737188 > > John Snow (4): > python: linter changes for pylint 3.x > python: Do not use pylint 3.2.4 with python 3.8 > iotests: Change imports for Python 3.13 > python: enable testing for 3.13 > > python/qemu/machine/machine.py | 1 + > python/qemu/utils/qemu_ga_client.py| 2 +- > python/setup.cfg | 4 +++- > tests/docker/dockerfiles/python.docker | 1 + > tests/qemu-iotests/testenv.py | 7 ++- > tests/qemu-iotests/testrunner.py | 9 ++--- > 6 files changed, 18 insertions(+), 6 deletions(-) > > -- > 2.45.0 > Staging under my Python branch, since I need to get tests green to roll forward with some more substantial changes. --js >
[PATCH 5/8] qapi: convert "Example" sections without titles
Use the no-option form of ".. qmp-example::" to convert any Examples
that do not have any form of caption or explanation whatsoever. Note
that in a few cases, example sections are split into two or more
separate example blocks. This is only done stylistically to create a
delineation between two or more logically independent examples.
See commit-3: "docs/qapidoc: create qmp-example directive", for a
detailed explanation of this custom directive syntax.
See commit+3: "qapi: remove "Example" doc section" for a detailed
explanation of why.
Signed-off-by: John Snow
---
qapi/acpi.json | 4 +--
qapi/block-core.json | 64 +---
qapi/block.json | 18 ++-
qapi/char.json | 24 +--
qapi/control.json| 8 ++---
qapi/dump.json | 8 ++---
qapi/machine-target.json | 2 +-
qapi/machine.json| 38
qapi/migration.json | 58 ++--
qapi/misc-target.json| 22 +++---
qapi/misc.json | 32 ++--
qapi/net.json| 20 +++--
qapi/pci.json| 2 +-
qapi/qdev.json | 10 ---
qapi/qom.json| 8 ++---
qapi/replay.json | 8 ++---
qapi/rocker.json | 8 ++---
qapi/run-state.json | 30 +--
qapi/tpm.json| 6 ++--
qapi/trace.json | 4 +--
qapi/transaction.json| 2 +-
qapi/ui.json | 34 ++---
qapi/vfio.json | 2 +-
qapi/virtio.json | 2 +-
qapi/yank.json | 4 +--
25 files changed, 216 insertions(+), 202 deletions(-)
diff --git a/qapi/acpi.json b/qapi/acpi.json
index aa4dbe57943..045dab6228b 100644
--- a/qapi/acpi.json
+++ b/qapi/acpi.json
@@ -111,7 +111,7 @@
#
# Since: 2.1
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-acpi-ospm-status" }
# <- { "return": [ { "device": "d1", "slot": "0", "slot-type": "DIMM",
"source": 1, "status": 0},
@@ -131,7 +131,7 @@
#
# Since: 2.1
#
-# Example:
+# .. qmp-example::
#
# <- { "event": "ACPI_DEVICE_OST",
# "data": { "info": { "device": "d1", "slot": "0",
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 9ef23ec02ae..4e0f0395146 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -764,7 +764,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-block" }
# <- {
@@ -1168,7 +1168,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-blockstats" }
# <- {
@@ -1461,7 +1461,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "block_resize",
# "arguments": { "device": "scratch", "size": 1073741824 } }
@@ -1682,7 +1682,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "blockdev-snapshot-sync",
# "arguments": { "device": "ide-hd0",
@@ -1715,7 +1715,7 @@
#
# Since: 2.5
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "blockdev-add",
# "arguments": { "driver": "qcow2",
@@ -1861,7 +1861,7 @@
#
# Since: 1.3
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "block-commit",
# "arguments": { "device": "virtio0",
@@ -1899,7 +1899,7 @@
#
# Since: 1.6
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "drive-backup",
# "arguments": { "device": "drive0",
@@ -1925,7 +1925,7 @@
#
# Since: 2.3
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "blockdev-backup",
# "arguments": { "device": "src-id",
@@ -1949,7 +1949,7 @@
#
# Since: 2.0
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-named-block-nodes" }
# <- { "return": [ { "ro":false,
@@ -2130,7 +2130,7 @@
#
# Since: 1.3
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "drive-mirror",
# "arguments": { "device": "ide-hd0",
@@ -2307,7 +2307,7 @@
#
# Since: 2.4
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "block-dirty-bitmap-add",
# "arguments": { "node": "drive0", "name": "bitma
[PATCH 8/8] qapi: remove "Example" doc section
Fully eliminate the "Example" sections in QAPI doc blocks now that they
have all been converted to arbitrary rST syntax using the
".. qmp-example::" directive. Update tests to match.
Migrating to the new syntax
---
The old "Example:" or "Examples:" section syntax is now caught as an
error, but "Example::" is stil permitted as explicit rST syntax for an
un-lexed, generic preformatted text block.
('Example' is not special in this case, any sentence that ends with "::"
will start an indented code block in rST.)
Arbitrary rST for Examples is now possible, but it's strongly
recommended that documentation authors use the ".. qmp-example::"
directive for consistent visual formatting in rendered HTML docs. The
":title:" directive option may be used to add extra information into the
title bar for the example. The ":annotated:" option can be used to write
arbitrary rST instead, with nested "::" blocks applying QMP formatting
where desired.
Other choices available are ".. code-block:: QMP" which will not create
an "Example:" box, or the short-form "::" code-block syntax which will
not apply QMP highlighting when used outside of the qmp-example
directive.
Why?
This patch has several benefits:
1. Example sections can now be written more arbitrarily, mixing
explanatory paragraphs and code blocks however desired.
2. Example sections can now use fully arbitrary rST.
3. All code blocks are now lexed and validated as QMP; increasing
usability of the docs and ensuring validity of example snippets.
(To some extent - This patch only gaurantees it lexes correctly, not
that it's valid under the JSON or QMP grammars. It will catch most
small mistakes, however.)
4. Each qmp-example can be titled or annotated independently without
bypassing the QMP lexer/validator.
(i.e. code blocks are now for *code* only, so we don't have to
sacrifice exposition for having lexically valid examples.)
NOTE: As with the "Notes" conversion patch, this patch (and those
preceding) may change the rendering order for Examples in the
current generator. The forthcoming qapidoc rewrite will fix this
by always generating documentation in source order.
Signed-off-by: John Snow
---
docs/devel/qapi-code-gen.rst| 58 -
scripts/qapi/parser.py | 10 +-
tests/qapi-schema/doc-good.json | 19 +++
tests/qapi-schema/doc-good.out | 26 ++-
tests/qapi-schema/doc-good.txt | 23 ++---
5 files changed, 98 insertions(+), 38 deletions(-)
diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index ae97b335cbf..2e10a3cbd69 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -899,7 +899,7 @@ Documentation markup
Documentation comments can use most rST markup. In particular,
-a ``::`` literal block can be used for examples::
+a ``::`` literal block can be used for pre-formatted text::
# ::
#
@@ -995,8 +995,8 @@ line "Features:", like this::
# @feature: Description text
A tagged section begins with a paragraph that starts with one of the
-following words: "Since:", "Example:"/"Examples:", "Returns:",
-"Errors:", "TODO:". It ends with the start of a new section.
+following words: "Since:", "Returns:", "Errors:", "TODO:". It ends with
+the start of a new section.
The second and subsequent lines of tagged sections must be indented
like this::
@@ -1020,13 +1020,53 @@ detailing a relevant error condition. For example::
A "Since: x.y.z" tagged section lists the release that introduced the
definition.
-An "Example" or "Examples" section is rendered entirely
-as literal fixed-width text. "TODO" sections are not rendered at all
-(they are for developers, not users of QMP). In other sections, the
-text is formatted, and rST markup can be used.
+"TODO" sections are not rendered at all (they are for developers, not
+users of QMP). In other sections, the text is formatted, and rST markup
+can be used.
+
+QMP Examples can be added by using the ``.. qmp-example::``
+directive. In its simplest form, this can be used to contain a single
+QMP code block which accepts standard JSON syntax with additional server
+directionality indicators (``->`` and ``<-``), and elisions (``...``).
+
+Optionally, a plaintext title may be provided by using the ``:title:``
+directive option. If the title is omitted, the example title will
+default to "Example:".
+
+A simple QMP example::
+
+ # .. qmp-example::
+ #:title: Using query-block
+ #
+ #-> { "execute": "query-block" }
[PATCH 6/8] qapi: convert "Example" sections with titles
When an Example section has a brief explanation, convert it to a
qmp-example:: section using the :title: option.
Rule of thumb: If the title can fit on a single line and requires no rST
markup, it's a good candidate for using the :title: option of
qmp-example.
In this patch, trailing punctuation is removed from the title section
for consistent headline aesthetics. In just one case, specifics of the
example are removed to make the title read better.
See commit-4: "docs/qapidoc: create qmp-example directive", for a
detailed explanation of this custom directive syntax.
See commit+2: "qapi: remove "Example" doc section" for a detailed
explanation of why.
Signed-off-by: John Snow
---
qapi/block-core.json | 24
qapi/block.json | 13 ++---
qapi/migration.json | 25 ++---
qapi/qom.json| 8
qapi/ui.json | 11 ++-
qapi/virtio.json | 19 ++-
6 files changed, 52 insertions(+), 48 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 4e0f0395146..a371e3464d2 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -5885,9 +5885,8 @@
#
# Since: 2.7
#
-# Examples:
-#
-# 1. Add a new node to a quorum
+# .. qmp-example::
+#:title: Add a new node to a quorum
#
# -> { "execute": "blockdev-add",
# "arguments": {
@@ -5901,7 +5900,8 @@
# "node": "new_node" } }
# <- { "return": {} }
#
-# 2. Delete a quorum's node
+# .. qmp-example::
+#:title: Delete a quorum's node
#
# -> { "execute": "x-blockdev-change",
# "arguments": { "parent": "disk1",
@@ -5937,16 +5937,16 @@
#
# Since: 2.12
#
-# Examples:
-#
-# 1. Move a node into an IOThread
+# .. qmp-example::
+#:title: Move a node into an IOThread
#
# -> { "execute": "x-blockdev-set-iothread",
# "arguments": { "node-name": "disk1",
# "iothread": "iothread0" } }
# <- { "return": {} }
#
-# 2. Move a node into the main loop
+# .. qmp-example::
+#:title: Move a node into the main loop
#
# -> { "execute": "x-blockdev-set-iothread",
# "arguments": { "node-name": "disk1",
@@ -6022,16 +6022,16 @@
#
# Since: 2.0
#
-# Examples:
-#
-# 1. Read operation
+# .. qmp-example::
+#:title: Read operation
#
# <- { "event": "QUORUM_REPORT_BAD",
# "data": { "node-name": "node0", "sector-num": 345435,
"sectors-count": 5,
#"type": "read" },
# "timestamp": { "seconds": 1344522075, "microseconds": 745528 } }
#
-# 2. Flush operation
+# .. qmp-example::
+#:title: Flush operation
#
# <- { "event": "QUORUM_REPORT_BAD",
# "data": { "node-name": "node0", "sector-num": 0, "sectors-count":
2097120,
diff --git a/qapi/block.json b/qapi/block.json
index c8e52bc2d29..5ddd061e964 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -342,9 +342,8 @@
#
# Since: 2.5
#
-# Examples:
-#
-# 1. Change a removable medium
+# .. qmp-example::
+#:title: Change a removable medium
#
# -> { "execute": "blockdev-change-medium",
# "arguments": { "id": "ide0-1-0",
@@ -352,7 +351,8 @@
# "format": "raw" } }
# <- { "return": {} }
#
-# 2. Load a read-only medium into a writable drive
+# .. qmp-example::
+#:title: Load a read-only medium into a writable drive
#
# -> { "execute": "blockdev-change-medium",
# "arguments": { "id": "floppyA",
@@ -577,9 +577,8 @@
# "boundaries-write": [1000, 5000] } }
# <- { "return": {} }
#
-# Example:
-#
-# Remove all latency histograms:
+# .. qmp-example::
+#:title: Remove all latency histograms
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0" } }
diff --git a/qapi/migration.json b/qapi/migration.json
index a4391ea7e6f..37ce8afa380 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -287,14 +287,14 @@
#
# Since: 0.14
#
-# Examples:
-#
-# 1. Before the first migration
+# .. qmp-example::
+#:title: Before the first migration
#
# -> { "execute": "query-migrate" }
# <- { &quo
[PATCH 4/8] docs/sphinx: add CSS styling for qmp-example directive
From: Harmonie Snow
Add CSS styling for qmp-example directives to increase readability and
consistently style all example blocks.
Signed-off-by: Harmonie Snow
Signed-off-by: John Snow
---
docs/sphinx-static/theme_overrides.css | 49 ++
1 file changed, 49 insertions(+)
diff --git a/docs/sphinx-static/theme_overrides.css
b/docs/sphinx-static/theme_overrides.css
index c70ef951286..965ecac54fd 100644
--- a/docs/sphinx-static/theme_overrides.css
+++ b/docs/sphinx-static/theme_overrides.css
@@ -87,6 +87,55 @@ div[class^="highlight"] pre {
padding-bottom: 1px;
}
+/* qmp-example directive styling */
+
+.rst-content .admonition-example {
+/* do not apply the standard admonition background */
+background-color: transparent;
+border: solid #ffd2ed 1px;
+}
+
+.rst-content .admonition-example > .admonition-title:before {
+content: "▷";
+}
+
+.rst-content .admonition-example > .admonition-title {
+background-color: #5980a6;
+}
+
+.rst-content .admonition-example > div[class^="highlight"] {
+/* make code boxes take up the full width of the admonition w/o margin */
+margin-left: -12px;
+margin-right: -12px;
+
+border-top: 1px solid #ffd2ed;
+border-bottom: 1px solid #ffd2ed;
+border-left: 0px;
+border-right: 0px;
+}
+
+.rst-content .admonition-example > div[class^="highlight"]:nth-child(2) {
+/* If a code box is the second element in an example admonition,
+ * it is the first child after the title. let it sit flush against
+ * the title. */
+margin-top: -12px;
+border-top: 0px;
+}
+
+.rst-content .admonition-example > div[class^="highlight"]:last-child {
+/* If a code box is the final element in an example admonition, don't
+ * render margin below it; let it sit flush with the end of the
+ * admonition box */
+margin-bottom: -12px;
+border-bottom: 0px;
+}
+
+.rst-content .admonition-example .highlight {
+background-color: #fffafd;
+}
+
+/* end qmp-example styling */
+
@media screen {
/* content column
--
2.45.0
[PATCH 7/8] qapi: convert "Example" sections with longer prose
These examples require longer explanations or have explanations that
require markup to look reasonable when rendered and so use the longer
form of the ".. qmp-example::" directive.
By using the :annotated: option, the content in the example block is
assumed *not* to be a code block literal and is instead parsed as normal
rST - with the exception that any code literal blocks after `::` will
assumed to be a QMP code literal block.
Note: There's one title-less conversion in this patch that comes along
for the ride because it's part of a larger "Examples" block that was
better to convert all at once.
See commit-5: "docs/qapidoc: create qmp-example directive", for a
detailed explanation of this custom directive syntax.
See commit+1: "qapi: remove "Example" doc section" for a detailed
explanation of why.
Signed-off-by: John Snow
---
qapi/block.json | 26 --
qapi/machine.json | 30 --
qapi/migration.json | 7 +--
qapi/virtio.json| 24 ++--
4 files changed, 59 insertions(+), 28 deletions(-)
diff --git a/qapi/block.json b/qapi/block.json
index 5ddd061e964..d95e9fd8140 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -545,31 +545,37 @@
#
# Since: 4.0
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# Set new histograms for all io types with intervals
-# [0, 10), [10, 50), [50, 100), [100, +inf):
+#Set new histograms for all io types with intervals
+#[0, 10), [10, 50), [50, 100), [100, +inf)::
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0",
# "boundaries": [10, 50, 100] } }
# <- { "return": {} }
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# Set new histogram only for write, other histograms will remain
-# not changed (or not created):
+#Set new histogram only for write, other histograms will remain
+#not changed (or not created)::
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0",
# "boundaries-write": [10, 50, 100] } }
# <- { "return": {} }
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# Set new histograms with the following intervals:
-# read, flush: [0, 10), [10, 50), [50, 100), [100, +inf)
-# write: [0, 1000), [1000, 5000), [5000, +inf)
+#Set new histograms with the following intervals:
+#
+#- read, flush: [0, 10), [10, 50), [50, 100), [100, +inf)
+#- write: [0, 1000), [1000, 5000), [5000, +inf)
+#
+#::
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0",
diff --git a/qapi/machine.json b/qapi/machine.json
index 83f60b319c7..0a5ffe652b7 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -1047,10 +1047,11 @@
#
# Since: 2.7
#
-# Examples:
+# .. qmp-example::
+#:annotated:
#
-# For pseries machine type started with -smp 2,cores=2,maxcpus=4
-# -cpu POWER8:
+#For pseries machine type started with
+#``-smp 2,cores=2,maxcpus=4 -cpu POWER8``::
#
# -> { "execute": "query-hotpluggable-cpus" }
# <- {"return": [
@@ -1060,7 +1061,10 @@
#"vcpus-count": 1, "qom-path": "/machine/unattached/device[0]"}
#]}
#
-# For pc machine type started with -smp 1,maxcpus=2:
+# .. qmp-example::
+#:annotated:
+#
+#For pc machine type started with ``-smp 1,maxcpus=2``::
#
# -> { "execute": "query-hotpluggable-cpus" }
# <- {"return": [
@@ -1075,8 +1079,11 @@
# }
#]}
#
-# For s390x-virtio-ccw machine type started with -smp 1,maxcpus=2
-# -cpu qemu (Since: 2.11):
+# .. qmp-example::
+#:annotated:
+#
+#For s390x-virtio-ccw machine type started with
+#``-smp 1,maxcpus=2 -cpu qemu`` (Since: 2.11)::
#
# -> { "execute": "query-hotpluggable-cpus" }
# <- {"return": [
@@ -1130,12 +1137,15 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# -> { "execute": "balloon", "arguments": { "value": 536870912 } }
-# <- { "return": {} }
+#::
#
-# With a 2.5GiB guest this command inflated the ballon to 3GiB.
+# -> { "execute": "balloon", "arguments": { "value": 536870912 } }
+# <- { "return": {} }
+#
+#With a 2.5GiB guest this command inflated the ballon to 3GiB.
##
{ 'command': 'balloon', 'data': {'valu
[PATCH 3/8] docs/qapidoc: add QMP highlighting to annotated qmp-example blocks
For any code literal blocks inside of a qmp-example directive, apply and
enforce the QMP lexer/highlighter to those blocks.
This way, you won't need to write:
```
.. qmp-example::
:annotated:
Blah blah
.. code-block:: QMP
-> { "lorem": "ipsum" }
```
But instead, simply:
```
.. qmp-example::
:annotated:
Blah blah::
-> { "lorem": "ipsum" }
```
Once the directive block is exited, whatever the previous default
highlight language was will be restored; localizing the forced QMP
lexing to exclusively this directive.
Note, if the default language is *already* QMP, this directive will not
generate and restore redundant highlight configuration nodes. We may
well decide that the default language ought to be QMP for any QAPI
reference pages, but this way the directive behaves consistently no
matter where it is used.
Signed-off-by: John Snow
---
docs/sphinx/qapidoc.py | 53 ++
1 file changed, 49 insertions(+), 4 deletions(-)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index b0f3917dc5b..fb2b23698a0 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -26,6 +26,7 @@
import os
import re
+import sys
import textwrap
from typing import List
@@ -37,6 +38,7 @@
from qapi.schema import QAPISchema
import sphinx
+from sphinx import addnodes
from sphinx.directives.code import CodeBlock
from sphinx.errors import ExtensionError
from sphinx.util.nodes import nested_parse_with_titles
@@ -574,10 +576,10 @@ class QMPExample(CodeBlock, NestedDirective):
Custom admonition for QMP code examples.
When the :annotated: option is present, the body of this directive
-is parsed as normal rST instead. Code blocks must be explicitly
-written by the user, but this allows for intermingling explanatory
-paragraphs with arbitrary rST syntax and code blocks for more
-involved examples.
+is parsed as normal rST, but with any '::' code blocks set to use
+the QMP lexer. Code blocks must be explicitly written by the user,
+but this allows for intermingling explanatory paragraphs with
+arbitrary rST syntax and code blocks for more involved examples.
When :annotated: is absent, the directive body is treated as a
simple standalone QMP code block literal.
@@ -591,6 +593,33 @@ class QMPExample(CodeBlock, NestedDirective):
"title": directives.unchanged,
}
+def _highlightlang(self) -> addnodes.highlightlang:
+"""Return the current highlightlang setting for the document"""
+node = None
+doc = self.state.document
+
+if hasattr(doc, "findall"):
+# docutils >= 0.18.1
+for node in doc.findall(addnodes.highlightlang):
+pass
+else:
+for elem in doc.traverse():
+if isinstance(elem, addnodes.highlightlang):
+node = elem
+
+if node:
+return node
+
+# No explicit directive found, use defaults
+node = addnodes.highlightlang(
+lang=self.env.config.highlight_language,
+force=False,
+# Yes, Sphinx uses this value to effectively disable line
+# numbers and not 0 or None or -1 or something. ¯\_(ツ)_/¯
+linenothreshold=sys.maxsize,
+)
+return node
+
def admonition_wrap(self, *content) -> List[nodes.Node]:
title = "Example:"
if "title" in self.options:
@@ -605,8 +634,24 @@ def admonition_wrap(self, *content) -> List[nodes.Node]:
return [admon]
def run_annotated(self) -> List[nodes.Node]:
+lang_node = self._highlightlang()
+
content_node: nodes.Element = nodes.section()
+
+# Configure QMP highlighting for "::" blocks, if needed
+if lang_node["lang"] != "QMP":
+content_node += addnodes.highlightlang(
+lang="QMP",
+force=False, # "True" ignores lexing errors
+linenothreshold=lang_node["linenothreshold"],
+)
+
self.do_parse(self.content, content_node)
+
+# Restore prior language highlighting, if needed
+if lang_node["lang"] != "QMP":
+content_node += addnodes.highlightlang(**lang_node.attributes)
+
return content_node.children
def run(self) -> List[nodes.Node]:
--
2.45.0
[PATCH 2/8] docs/qapidoc: create qmp-example directive
This is a directive that creates a syntactic sugar for creating
"Example" boxes very similar to the ones already used in the bitmaps.rst
document, please see e.g.
https://www.qemu.org/docs/master/interop/bitmaps.html#creation-block-dirty-bitmap-add
In its simplest form, when a custom title is not needed or wanted, and
the example body is *solely* a QMP example:
```
.. qmp-example::
{body}
```
is syntactic sugar for:
```
.. admonition:: Example:
.. code-block:: QMP
{body}
```
When a custom, plaintext title that describes the example is desired,
this form:
```
.. qmp-example::
:title: Defrobnification
{body}
```
Is syntactic sugar for:
```
.. admonition:: Example: Defrobnification
.. code-block:: QMP
{body}
```
Lastly, when Examples are multi-step processes that require non-QMP
exposition, have lengthy titles, or otherwise involve prose with rST
markup (lists, cross-references, etc), the most complex form:
```
.. qmp-example::
:annotated:
This example shows how to use `foo-command`::
{body}
For more information, please see `frobnozz`.
```
Is desugared to:
```
.. admonition:: Example:
This example shows how to use `foo-command`::
{body}
For more information, please see `frobnozz`.
```
Note that :annotated: and :title: options can be combined together, if
desired.
The primary benefit here being documentation source consistently using
the same directive for all forms of examples to ensure consistent visual
styling, and ensuring all relevant prose is visually grouped alongside
the code literal block.
Note that as of this commit, the code-block rST syntax "::" does not
apply QMP highlighting; you would need to use ".. code-block:: QMP". The
very next commit changes this behavior to assume all "::" code blocks
within this directive are QMP blocks.
Signed-off-by: John Snow
---
docs/sphinx/qapidoc.py | 55 ++
1 file changed, 55 insertions(+)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index 43dd99e21e6..b0f3917dc5b 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -27,6 +27,7 @@
import os
import re
import textwrap
+from typing import List
from docutils import nodes
from docutils.parsers.rst import Directive, directives
@@ -36,6 +37,7 @@
from qapi.schema import QAPISchema
import sphinx
+from sphinx.directives.code import CodeBlock
from sphinx.errors import ExtensionError
from sphinx.util.nodes import nested_parse_with_titles
@@ -567,10 +569,63 @@ def run(self):
raise ExtensionError(str(err)) from err
+class QMPExample(CodeBlock, NestedDirective):
+"""
+Custom admonition for QMP code examples.
+
+When the :annotated: option is present, the body of this directive
+is parsed as normal rST instead. Code blocks must be explicitly
+written by the user, but this allows for intermingling explanatory
+paragraphs with arbitrary rST syntax and code blocks for more
+involved examples.
+
+When :annotated: is absent, the directive body is treated as a
+simple standalone QMP code block literal.
+"""
+
+required_argument = 0
+optional_arguments = 0
+has_content = True
+option_spec = {
+"annotated": directives.flag,
+"title": directives.unchanged,
+}
+
+def admonition_wrap(self, *content) -> List[nodes.Node]:
+title = "Example:"
+if "title" in self.options:
+title = f"{title} {self.options['title']}"
+
+admon = nodes.admonition(
+"",
+nodes.title("", title),
+*content,
+classes=["admonition", "admonition-example"],
+)
+return [admon]
+
+def run_annotated(self) -> List[nodes.Node]:
+content_node: nodes.Element = nodes.section()
+self.do_parse(self.content, content_node)
+return content_node.children
+
+def run(self) -> List[nodes.Node]:
+annotated = "annotated" in self.options
+
+if annotated:
+content_nodes = self.run_annotated()
+else:
+self.arguments = ["QMP"]
+content_nodes = super().run()
+
+return self.admonition_wrap(*content_nodes)
+
+
def setup(app):
"""Register qapi-doc directive with Sphinx"""
app.add_config_value("qapidoc_srctree", None, "env")
app.add_directive("qapi-doc", QAPIDocDirective)
+app.add_directive("qmp-example", QMPExample)
return {
"version": __version__,
--
2.45.0
[PATCH 0/8] qapi: convert example sections to qmp-example rST directives
GitLab: https://gitlab.com/jsnow/qemu/-/pipelines/1359714660 This patchset focuses on converting example sections to rST directives using a new `.. qmp-example::` directive. It is based on what I *assume* will be Markus' next pull request that covers note conversion. Pull these patches from GitLab directly if that's too annoying: https://gitlab.com/jsnow/qemu/-/commits/sphinx-domain-prereqs-examples It is also annoyingly the case that both Markus' next pull request and this series conflicts with a separate series I sent out, "docs/python: bump minimum Sphinx version" - so it's extremely likely I'll need to rebase and respin this series depending on what goes in and in what order. Ah well... Changes since this was split out from the prior series: - Harmonie updated the CSS for the example block section. I think it's really tidy now! Thanks Harmonie! - Dependence on SphinxDirective was removed, but it will likely re-appear in the next series anyway. - qapi-code-gen.rst was updated with a section on how to write examples. - Various minor tweaks to comments, commit messages, docs, etc. Harmonie Snow (1): docs/sphinx: add CSS styling for qmp-example directive John Snow (7): docs/qapidoc: factor out do_parse() docs/qapidoc: create qmp-example directive docs/qapidoc: add QMP highlighting to annotated qmp-example blocks qapi: convert "Example" sections without titles qapi: convert "Example" sections with titles qapi: convert "Example" sections with longer prose qapi: remove "Example" doc section docs/devel/qapi-code-gen.rst | 58 +++-- docs/sphinx-static/theme_overrides.css | 49 docs/sphinx/qapidoc.py | 156 + qapi/acpi.json | 4 +- qapi/block-core.json | 88 +++--- qapi/block.json| 57 + qapi/char.json | 24 ++-- qapi/control.json | 8 +- qapi/dump.json | 8 +- qapi/machine-target.json | 2 +- qapi/machine.json | 68 ++- qapi/migration.json| 90 +++--- qapi/misc-target.json | 22 ++-- qapi/misc.json | 32 ++--- qapi/net.json | 20 ++-- qapi/pci.json | 2 +- qapi/qdev.json | 10 +- qapi/qom.json | 16 +-- qapi/replay.json | 8 +- qapi/rocker.json | 8 +- qapi/run-state.json| 30 ++--- qapi/tpm.json | 6 +- qapi/trace.json| 4 +- qapi/transaction.json | 2 +- qapi/ui.json | 45 +++ qapi/vfio.json | 2 +- qapi/virtio.json | 45 --- qapi/yank.json | 4 +- scripts/qapi/parser.py | 10 +- tests/qapi-schema/doc-good.json| 19 +-- tests/qapi-schema/doc-good.out | 26 +++-- tests/qapi-schema/doc-good.txt | 23 ++-- 32 files changed, 605 insertions(+), 341 deletions(-) -- 2.45.0
[PATCH 1/8] docs/qapidoc: factor out do_parse()
Factor out the compatibility parser helper into a base class, so it can be shared by other directives. Signed-off-by: John Snow --- docs/sphinx/qapidoc.py | 64 +++--- 1 file changed, 35 insertions(+), 29 deletions(-) diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py index efcd84656fa..43dd99e21e6 100644 --- a/docs/sphinx/qapidoc.py +++ b/docs/sphinx/qapidoc.py @@ -494,7 +494,41 @@ def visit_module(self, name): super().visit_module(name) -class QAPIDocDirective(Directive): +class NestedDirective(Directive): +def run(self): +raise NotImplementedError + +def do_parse(self, rstlist, node): +""" +Parse rST source lines and add them to the specified node + +Take the list of rST source lines rstlist, parse them as +rST, and add the resulting docutils nodes as children of node. +The nodes are parsed in a way that allows them to include +subheadings (titles) without confusing the rendering of +anything else. +""" +# This is from kerneldoc.py -- it works around an API change in +# Sphinx between 1.6 and 1.7. Unlike kerneldoc.py, we use +# sphinx.util.nodes.nested_parse_with_titles() rather than the +# plain self.state.nested_parse(), and so we can drop the saving +# of title_styles and section_level that kerneldoc.py does, +# because nested_parse_with_titles() does that for us. +if USE_SSI: +with switch_source_input(self.state, rstlist): +nested_parse_with_titles(self.state, rstlist, node) +else: +save = self.state.memo.reporter +self.state.memo.reporter = AutodocReporter( +rstlist, self.state.memo.reporter +) +try: +nested_parse_with_titles(self.state, rstlist, node) +finally: +self.state.memo.reporter = save + + +class QAPIDocDirective(NestedDirective): """Extract documentation from the specified QAPI .json file""" required_argument = 1 @@ -532,34 +566,6 @@ def run(self): # so they are displayed nicely to the user raise ExtensionError(str(err)) from err -def do_parse(self, rstlist, node): -"""Parse rST source lines and add them to the specified node - -Take the list of rST source lines rstlist, parse them as -rST, and add the resulting docutils nodes as children of node. -The nodes are parsed in a way that allows them to include -subheadings (titles) without confusing the rendering of -anything else. -""" -# This is from kerneldoc.py -- it works around an API change in -# Sphinx between 1.6 and 1.7. Unlike kerneldoc.py, we use -# sphinx.util.nodes.nested_parse_with_titles() rather than the -# plain self.state.nested_parse(), and so we can drop the saving -# of title_styles and section_level that kerneldoc.py does, -# because nested_parse_with_titles() does that for us. -if USE_SSI: -with switch_source_input(self.state, rstlist): -nested_parse_with_titles(self.state, rstlist, node) -else: -save = self.state.memo.reporter -self.state.memo.reporter = AutodocReporter( -rstlist, self.state.memo.reporter -) -try: -nested_parse_with_titles(self.state, rstlist, node) -finally: -self.state.memo.reporter = save - def setup(app): """Register qapi-doc directive with Sphinx""" -- 2.45.0
Re: [PATCH 3/4] iotests: Change imports for Python 3.13
On Tue, Jul 2, 2024, 1:51 PM Nir Soffer wrote: > > On 2 Jul 2024, at 17:44, John Snow wrote: > > > > On Tue, Jul 2, 2024 at 7:52 AM Nir Soffer wrote: > >> On Thu, Jun 27, 2024 at 2:23 AM John Snow wrote: >> > >> > Python 3.13 isn't out yet, but it's in beta and Fedora is ramping up to >> > make it the default system interpreter for Fedora 41. >> > >> > They moved our cheese for where ContextManager lives; add a conditional >> > to locate it while we support both pre-3.9 and 3.13+. >> > >> > Signed-off-by: John Snow >> > --- >> > tests/qemu-iotests/testenv.py| 7 ++- >> > tests/qemu-iotests/testrunner.py | 9 ++--- >> > 2 files changed, 12 insertions(+), 4 deletions(-) >> > >> > diff --git a/tests/qemu-iotests/testenv.py >> b/tests/qemu-iotests/testenv.py >> > index 588f30a4f14..96d69e56963 100644 >> > --- a/tests/qemu-iotests/testenv.py >> > +++ b/tests/qemu-iotests/testenv.py >> > @@ -25,7 +25,12 @@ >> > import random >> > import subprocess >> > import glob >> > -from typing import List, Dict, Any, Optional, ContextManager >> > +from typing import List, Dict, Any, Optional >> > + >> > +if sys.version_info >= (3, 9): >> > +from contextlib import AbstractContextManager as ContextManager >> > +else: >> > +from typing import ContextManager >> >> It can be cleaner to add a compat module hiding the details so the >> entire project >> can have a single instance of this. Other code will just use: >> >> from compat import ContextManager >> > > If there were more than two uses, I'd consider it. As it stands, a > compat.py module with just one import conditional in it doesn't seem worth > the hassle. Are there more cases of compatibility goop inside iotests that > need to be factored out to make it worth it? > > > I don’t about other. For me even one instance is ugly enough :-) > I was going to add it to qemu/utils, but then I remembered the testenv/testrunner script here needs to operate without external dependencies because part of the function of these modules is to *locate* those dependencies. Ehh. I'm going to say that repeating the import scaffolding in just two places is fine enough for now and really not worth adding a compat.py for *just* this. Let's just get the tests green. --js
Re: [PATCH 3/4] iotests: Change imports for Python 3.13
On Tue, Jul 2, 2024 at 7:52 AM Nir Soffer wrote: > On Thu, Jun 27, 2024 at 2:23 AM John Snow wrote: > > > > Python 3.13 isn't out yet, but it's in beta and Fedora is ramping up to > > make it the default system interpreter for Fedora 41. > > > > They moved our cheese for where ContextManager lives; add a conditional > > to locate it while we support both pre-3.9 and 3.13+. > > > > Signed-off-by: John Snow > > --- > > tests/qemu-iotests/testenv.py| 7 ++- > > tests/qemu-iotests/testrunner.py | 9 ++--- > > 2 files changed, 12 insertions(+), 4 deletions(-) > > > > diff --git a/tests/qemu-iotests/testenv.py > b/tests/qemu-iotests/testenv.py > > index 588f30a4f14..96d69e56963 100644 > > --- a/tests/qemu-iotests/testenv.py > > +++ b/tests/qemu-iotests/testenv.py > > @@ -25,7 +25,12 @@ > > import random > > import subprocess > > import glob > > -from typing import List, Dict, Any, Optional, ContextManager > > +from typing import List, Dict, Any, Optional > > + > > +if sys.version_info >= (3, 9): > > +from contextlib import AbstractContextManager as ContextManager > > +else: > > +from typing import ContextManager > > It can be cleaner to add a compat module hiding the details so the > entire project > can have a single instance of this. Other code will just use: > > from compat import ContextManager > If there were more than two uses, I'd consider it. As it stands, a compat.py module with just one import conditional in it doesn't seem worth the hassle. Are there more cases of compatibility goop inside iotests that need to be factored out to make it worth it? --js
Re: [PATCH 3/4] iotests: Change imports for Python 3.13
Ping - happy to merge this series myself but didn't wanna change iotests without at least an ack from the lord of that castle. On Wed, Jun 26, 2024, 7:22 PM John Snow wrote: > Python 3.13 isn't out yet, but it's in beta and Fedora is ramping up to > make it the default system interpreter for Fedora 41. > > They moved our cheese for where ContextManager lives; add a conditional > to locate it while we support both pre-3.9 and 3.13+. > > Signed-off-by: John Snow > --- > tests/qemu-iotests/testenv.py| 7 ++- > tests/qemu-iotests/testrunner.py | 9 ++--- > 2 files changed, 12 insertions(+), 4 deletions(-) > > diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py > index 588f30a4f14..96d69e56963 100644 > --- a/tests/qemu-iotests/testenv.py > +++ b/tests/qemu-iotests/testenv.py > @@ -25,7 +25,12 @@ > import random > import subprocess > import glob > -from typing import List, Dict, Any, Optional, ContextManager > +from typing import List, Dict, Any, Optional > + > +if sys.version_info >= (3, 9): > +from contextlib import AbstractContextManager as ContextManager > +else: > +from typing import ContextManager > > DEF_GDB_OPTIONS = 'localhost:12345' > > diff --git a/tests/qemu-iotests/testrunner.py > b/tests/qemu-iotests/testrunner.py > index 7b322272e92..2e236c8fa39 100644 > --- a/tests/qemu-iotests/testrunner.py > +++ b/tests/qemu-iotests/testrunner.py > @@ -27,11 +27,14 @@ > import shutil > import sys > from multiprocessing import Pool > -from typing import List, Optional, Any, Sequence, Dict, \ > -ContextManager > - > +from typing import List, Optional, Any, Sequence, Dict > from testenv import TestEnv > > +if sys.version_info >= (3, 9): > +from contextlib import AbstractContextManager as ContextManager > +else: > +from typing import ContextManager > + > > def silent_unlink(path: Path) -> None: > try: > -- > 2.45.0 > >
Re: [PATCH v2 10/21] qapi: convert "Note" sections to plain rST
On Fri, Jun 28, 2024, 5:52 AM Markus Armbruster wrote: > John Snow writes: > > > We do not need a dedicated section for notes. By eliminating a specially > > parsed section, these notes can be treated as normal rST paragraphs in > > the new QMP reference manual, and can be placed and styled much more > > flexibly. > > > > Convert all existing "Note" and "Notes" sections to pure rST. As part of > > the conversion, capitalize the first letter of each sentence and add > > trailing punctuation where appropriate to ensure notes look sensible and > > consistent in rendered HTML documentation. Markup is also re-aligned to > > the de-facto standard of 3 spaces for directives. > > > > Update docs/devel/qapi-code-gen.rst to reflect the new paradigm, and > > update the QAPI parser to prohibit "Note" sections while suggesting a > > new syntax. The exact formatting to use is a matter of taste, but a good > > candidate is simply: > > > > .. note:: lorem ipsum ... > >... dolor sit amet ... > >... consectetur adipiscing elit ... > > > > ... but there are other choices, too. The Sphinx readthedocs theme > > offers theming for the following forms (capitalization unimportant); all > > are adorned with a (!) symbol () in the title bar for rendered HTML > > docs. > > > > See > > > https://sphinx-rtd-theme.readthedocs.io/en/stable/demo/demo.html#admonitions > > for examples of each directive/admonition in use. > > > > These are rendered in orange: > > > > .. Attention:: ... > > .. Caution:: ... > > .. WARNING:: ... > > > > These are rendered in red: > > > > .. DANGER:: ... > > .. Error:: ... > > > > These are rendered in green: > > > > .. Hint:: ... > > .. Important:: ... > > .. Tip:: ... > > > > These are rendered in blue: > > > > .. Note:: ... > > .. admonition:: custom title > > > >admonition body text > > > > This patch uses ".. note::" almost everywhere, with just two "caution" > > directives. Several instances of "Notes:" have been converted to merely > > ".. note::" where appropriate, but ".. admonition:: notes" is used in a > > few places where we had an ordered list of multiple notes that would not > > make sense as standalone/separate admonitions. > > I looked for hunks that don't 1:1 replace "Note:" or "Notes:" by > ".. note::." Findings: > > * Two hunks replace by ".. caution::" instead. Commit message got it. > Good. > > * Four hunks replace by ".. admonition:: notes", one of them as a test. > Commit message got it. Good. > > * Three hunks split "Notes:" into multiple ".. note::". Good, but could > be mentioned in commit message. > I meant to imply it when discussing when admonition was used, but yeah. > * Two hunks drop "Note:", changing it into paragraph. The paragraph > merges into the preceding "Example" section. Good, but should be > mentioned in the commit message, or turned into a separate patch. > Eh. we got enough commits. I think it's helpful to keep the whole conversion in one giant bang so that the diff is helpful in illustrating all of the different types of conversions. (In fact, even though I split out Example conversion for your sake in review, I think it'd be helpful to squash them together on merge for the same exact reason.) Let's just amend the commit message. > * One hunk adjusts a test case for the removal of the "Note:" tag. > Good, but could be mentioned in the commit message. > > Perhaps tweak the paragraph above: > > This patch uses ".. note::" almost everywhere, with just two "caution" > directives. Several instances of "Notes:" have been converted to > merely ".. note::", or multiple ".. note::" where appropriate. > ".. admonition:: notes" is used in a few places where we had an > ordered list of multiple notes that would not make sense as > standalone/separate admonitions. Two "Note:" following "Example:" > have been turned into ordinary paragraphs within the example. > > Okay? > Yep, suits me fine. > > NOTE: Because qapidoc.py does not attempt to preserve source ordering of > > sections, the conversion of Notes from a "tagged section" to an > > "untagged section" means that rendering order for some notes *may > > change* as a result of this patch. The forthcoming qapidoc.py rewrite > > strictly preserves source ordering in the rendered documentation, so > > this issue will be rectified in the new generator. > > > > Signed-off-by: John Snow > > Acked-by: Stefan Hajnoczi [for block*.json] > > I dislike the indentation changes, and may revert them in my tree. > 😢 Would you take a patch adjusting the indent later, or will you then tell me it's not worth the git blame fuzz? :) > Reviewed-by: Markus Armbruster > >
Re: [PATCH v2 07/21] docs/qapidoc: fix nested parsing under untagged sections
On Fri, Jun 28, 2024, 11:10 AM John Snow wrote: > > > On Fri, Jun 28, 2024, 3:55 AM Markus Armbruster wrote: > >> John Snow writes: >> >> > Sphinx does not like sections without titles, because it wants to >> > convert every section into a reference. When there is no title, it >> > struggles to do this and transforms the tree inproperly. >> > >> > Depending on the rST used, this may result in an assertion error deep in >> > the docutils HTMLWriter. >> > >> > (Observed when using ".. admonition:: Notes" under such a section - When >> > this is transformed with its own element, Sphinx is fooled into >> > believing this title belongs to the section and incorrect mutates the >> > docutils tree, leading to errors during rendering time.) >> > >> > When parsing an untagged section (free paragraphs), skip making a hollow >> > section and instead append the parse results to the prior section. >> > >> > Many Bothans died to bring us this information. >> > >> > Signed-off-by: John Snow >> > Acked-by: Markus Armbruster >> >> Generated HTML changes, but the diff is hard to review due to id >> attribute changes all over the place. >> >> Generated qemu-ga-ref.7 also changes: >> >> diff -rup old/qemu-ga-ref.7 new/qemu-ga-ref.7 >> --- old/qemu-ga-ref.7 2024-06-27 10:42:21.466096276 +0200 >> +++ new/qemu-ga-ref.7 2024-06-27 10:45:36.502414099 +0200 >> @@ -397,6 +397,7 @@ shutdown request, with no guarantee of s >> .B \fBmode\fP: \fBstring\fP (optional) >> \(dqhalt\(dq, \(dqpowerdown\(dq (default), or \(dqreboot\(dq >> .UNINDENT >> +.sp >> This command does NOT return a response on success. Success >> condition is indicated by the VM exiting with a zero exit status or, >> when running with \-\-no\-shutdown, by issuing the query\-status QMP >> @@ -1348,6 +1349,7 @@ the new password entry string, base64 en >> .B \fBcrypted\fP: \fBboolean\fP >> true if password is already crypt()d, false if raw >> .UNINDENT >> +.sp >> If the \fBcrypted\fP flag is true, it is the caller\(aqs >> responsibility to >> ensure the correct crypt() encryption scheme is used. This command >> does not attempt to interpret or report on the encryption scheme. >> >> We add vertical space. Visible when viewed with man. Looks like an >> improvement to me. >> >> Here's the first of these two spots in HTML: >> >> - >> -> class="docutils literal notranslate">> class="pre">guest-shutdown (Command)> href="#qapidoc-31" title="Permalink to this heading"> >> + >> +> class="docutils literal notranslate">> class="pre">guest-shutdown (Command)> href="#qapidoc-30" title="Permalink to this heading"> >> Initiate guest-activated shutdown. Note: this is an asynchronous >> shutdown request, with no guarantee of successful shutdown. >> >> @@ -502,22 +502,20 @@ shutdown request, with no guarantee of s >> >> >> >> - >> This command does NOT return a response on success. Success >> condition is indicated by the VM exiting with a zero exit status or, >> when running with –no-shutdown, by issuing the query-status QMP >> command to confirm the VM status is “shutdown”. >> - >> - >> -Since >> + >> +Since >> 0.15.0 >> >> >> >> The id changes muddy the waters. With them manually removed: >> >> >> > class="docutils literal notranslate">> class="pre">guest-shutdown (Command)> href="#qapidoc-31" title="Permalink to this heading"> >> Initiate guest-activated shutdown. Note: this is an asynchronous >> shutdown request, with no guarantee of successful shutdown. >> >> @@ -502,22 +502,20 @@ shutdown request, with no guarantee of s >> >> >> >> - >> This command does NOT return a response on success. Success >> condition is indicated by the VM exiting with a zero exit status or, >> when running with –no-shutdown, by issuing the query-status QMP >> command to confirm the VM status is “shutdown”. >> - >> >> Since >> 0.15.0 >> >> >> >> Makes no visual difference in my browser. >> >> Do these differences match your expectations? >> > > Yep! > > It does change the output just a little, but Sphinx really doesn't like > title-less sections. > > I thought the change looked fine, and I'm still planning on removing this > old generator anyway, so... > Oh, pro tip: try using the xml builder before and after for a cleaner comparison. --js >
Re: [PATCH v2 15/21] docs/qapidoc: create qmp-example directive
On Fri, Jun 28, 2024, 9:24 AM Markus Armbruster wrote:
> John Snow writes:
>
> > This is a directive that creates a syntactic sugar for creating
> > "Example" boxes very similar to the ones already used in the bitmaps.rst
> > document, please see e.g.
> >
> https://www.qemu.org/docs/master/interop/bitmaps.html#creation-block-dirty-bitmap-add
> >
> > In its simplest form, when a custom title is not needed or wanted, and
> > the example body is *solely* a QMP example:
> >
> > ```
> > .. qmp-example::
> >
> >{body}
> > ```
> >
> > is syntactic sugar for:
> >
> > ```
> > .. admonition:: Example:
> >
> >.. code-block:: QMP
> >
> > {body}
> > ```
> >
> > When a custom, plaintext title that describes the example is desired,
> > this form:
> >
> > ```
> > .. qmp-example::
> >:title: Defrobnification
> >
> >{body}
> > ```
> >
> > Is syntactic sugar for:
> >
> > ```
> > .. admonition:: Example: Defrobnification
> >
> >.. code-block:: QMP
> >
> > {body}
> > ```
> >
> > Lastly, when Examples are multi-step processes that require non-QMP
> > exposition, have lengthy titles, or otherwise involve prose with rST
> > markup (lists, cross-references, etc), the most complex form:
> >
> > ```
> > .. qmp-example::
> >:annotated:
> >
> >This example shows how to use `foo-command`::
> >
> > {body}
> > ```
> >
> > Is desugared to:
> >
> > ```
> > .. admonition:: Example:
> >
> >This example shows how to use `foo-command`::
> >
> > {body}
> >
> >For more information, please see `frobnozz`.
> > ```
>
^ Whoops, added prose in the desugar block without modifying the original.
> Can we combine the latter two? Like this:
>
> .. qmp-example::
> :title: Defrobnification
> :annotated:
>
> This example shows how to use `foo-command`::
>
>{body}
>
Yes! I only didn't use that form in the series because splitting longer
Examples into title and prose felt like an editorial decision, but
absolutely you can use both.
> > The primary benefit here being documentation source consistently using
> > the same directive for all forms of examples to ensure consistent visual
> > styling, and ensuring all relevant prose is visually grouped alongside
> > the code literal block.
> >
> > Note that as of this commit, the code-block rST syntax "::" does not
> > apply QMP highlighting; you would need to use ".. code-block:: QMP". The
> > very next commit changes this behavior to assume all "::" code blocks
> > within this directive are QMP blocks.
> >
> > Signed-off-by: John Snow
> > ---
> > docs/sphinx/qapidoc.py | 60 --
> > 1 file changed, 58 insertions(+), 2 deletions(-)
>
> No tests? Hmm, I see you convert existing tests in PATCH 19-21. While
> that works, test coverage now would make it easier to see how each patch
> affects doc generator output.
>
Mmm. Do you want me to move the test changes up to this patch ... ?
> > diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> > index 43dd99e21e6..a2fa05fc491 100644
> > --- a/docs/sphinx/qapidoc.py
> > +++ b/docs/sphinx/qapidoc.py
> > @@ -27,16 +27,19 @@
> > import os
> > import re
> > import textwrap
> > +from typing import List
> >
> > from docutils import nodes
> > -from docutils.parsers.rst import Directive, directives
> > +from docutils.parsers.rst import directives
> > from docutils.statemachine import ViewList
> > from qapi.error import QAPIError, QAPISemError
> > from qapi.gen import QAPISchemaVisitor
> > from qapi.schema import QAPISchema
> >
> > import sphinx
> > +from sphinx.directives.code import CodeBlock
> > from sphinx.errors import ExtensionError
> > +from sphinx.util.docutils import SphinxDirective
> > from sphinx.util.nodes import nested_parse_with_titles
> >
> >
> > @@ -494,7 +497,7 @@ def visit_module(self, name):
> > super().visit_module(name)
> >
> >
> > -class NestedDirective(Directive):
> > +class NestedDirective(SphinxDirective):
>
> What is this about?
>
Hmm. Strictly it's for access to sphinx configuration which I use only in
the next patch, but practically I suspect if I don't change it *here* that
the multiple inheritance fr
Re: [PATCH v2 14/21] docs/qapidoc: factor out do_parse()
On Fri, Jun 28, 2024, 9:09 AM Markus Armbruster wrote: > John Snow writes: > > > Factor out the compatibility parser helper so it can be shared by other > > directives. > > Suggest "Factor out the compatibility parser helper into a base class, > so it can be shared by other directives." Sure. Haven't read the other mails yet. I'll make the change if you want a v3, otherwise feel free to edit. > > > > Signed-off-by: John Snow > > --- > > docs/sphinx/qapidoc.py | 64 +++--- > > 1 file changed, 35 insertions(+), 29 deletions(-) > > > > diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py > > index efcd84656fa..43dd99e21e6 100644 > > --- a/docs/sphinx/qapidoc.py > > +++ b/docs/sphinx/qapidoc.py > > @@ -494,7 +494,41 @@ def visit_module(self, name): > > super().visit_module(name) > > > > > > -class QAPIDocDirective(Directive): > > +class NestedDirective(Directive): > > +def run(self): > > +raise NotImplementedError > > Should this class be abstract? > It could be ... *sneezes* I plan to delete it by the end of the qapi-domain series anyway, or perhaps I could even delete it *before* with a dedicated "require sphinx >= 3.x" miniseries. Actually, that's probably a really good idea... > > + > > +def do_parse(self, rstlist, node): > > +""" > > +Parse rST source lines and add them to the specified node > > + > > +Take the list of rST source lines rstlist, parse them as > > +rST, and add the resulting docutils nodes as children of node. > > +The nodes are parsed in a way that allows them to include > > +subheadings (titles) without confusing the rendering of > > +anything else. > > +""" > > +# This is from kerneldoc.py -- it works around an API change in > > +# Sphinx between 1.6 and 1.7. Unlike kerneldoc.py, we use > > +# sphinx.util.nodes.nested_parse_with_titles() rather than the > > +# plain self.state.nested_parse(), and so we can drop the saving > > +# of title_styles and section_level that kerneldoc.py does, > > +# because nested_parse_with_titles() does that for us. > > +if USE_SSI: > > +with switch_source_input(self.state, rstlist): > > +nested_parse_with_titles(self.state, rstlist, node) > > +else: > > +save = self.state.memo.reporter > > +self.state.memo.reporter = AutodocReporter( > > +rstlist, self.state.memo.reporter > > +) > > +try: > > +nested_parse_with_titles(self.state, rstlist, node) > > +finally: > > +self.state.memo.reporter = save > > + > > + > > +class QAPIDocDirective(NestedDirective): > > """Extract documentation from the specified QAPI .json file""" > > > > required_argument = 1 > > @@ -532,34 +566,6 @@ def run(self): > > # so they are displayed nicely to the user > > raise ExtensionError(str(err)) from err > > > > -def do_parse(self, rstlist, node): > > -"""Parse rST source lines and add them to the specified node > > - > > -Take the list of rST source lines rstlist, parse them as > > -rST, and add the resulting docutils nodes as children of node. > > -The nodes are parsed in a way that allows them to include > > -subheadings (titles) without confusing the rendering of > > -anything else. > > -""" > > -# This is from kerneldoc.py -- it works around an API change in > > -# Sphinx between 1.6 and 1.7. Unlike kerneldoc.py, we use > > -# sphinx.util.nodes.nested_parse_with_titles() rather than the > > -# plain self.state.nested_parse(), and so we can drop the saving > > -# of title_styles and section_level that kerneldoc.py does, > > -# because nested_parse_with_titles() does that for us. > > -if USE_SSI: > > -with switch_source_input(self.state, rstlist): > > -nested_parse_with_titles(self.state, rstlist, node) > > -else: > > -save = self.state.memo.reporter > > -self.state.memo.reporter = AutodocReporter( > > -rstlist, self.state.memo.reporter > > -) > > -try: > > -nested_parse_with_titles(self.state, rstlist, node) > > -finally: > > -self.state.memo.reporter = save > > - > > > > def setup(app): > > """Register qapi-doc directive with Sphinx""" > > Reviewed-by: Markus Armbruster > >
Re: [PATCH v2 07/21] docs/qapidoc: fix nested parsing under untagged sections
On Fri, Jun 28, 2024, 3:55 AM Markus Armbruster wrote: > John Snow writes: > > > Sphinx does not like sections without titles, because it wants to > > convert every section into a reference. When there is no title, it > > struggles to do this and transforms the tree inproperly. > > > > Depending on the rST used, this may result in an assertion error deep in > > the docutils HTMLWriter. > > > > (Observed when using ".. admonition:: Notes" under such a section - When > > this is transformed with its own element, Sphinx is fooled into > > believing this title belongs to the section and incorrect mutates the > > docutils tree, leading to errors during rendering time.) > > > > When parsing an untagged section (free paragraphs), skip making a hollow > > section and instead append the parse results to the prior section. > > > > Many Bothans died to bring us this information. > > > > Signed-off-by: John Snow > > Acked-by: Markus Armbruster > > Generated HTML changes, but the diff is hard to review due to id > attribute changes all over the place. > > Generated qemu-ga-ref.7 also changes: > > diff -rup old/qemu-ga-ref.7 new/qemu-ga-ref.7 > --- old/qemu-ga-ref.7 2024-06-27 10:42:21.466096276 +0200 > +++ new/qemu-ga-ref.7 2024-06-27 10:45:36.502414099 +0200 > @@ -397,6 +397,7 @@ shutdown request, with no guarantee of s > .B \fBmode\fP: \fBstring\fP (optional) > \(dqhalt\(dq, \(dqpowerdown\(dq (default), or \(dqreboot\(dq > .UNINDENT > +.sp > This command does NOT return a response on success. Success > condition is indicated by the VM exiting with a zero exit status or, > when running with \-\-no\-shutdown, by issuing the query\-status QMP > @@ -1348,6 +1349,7 @@ the new password entry string, base64 en > .B \fBcrypted\fP: \fBboolean\fP > true if password is already crypt()d, false if raw > .UNINDENT > +.sp > If the \fBcrypted\fP flag is true, it is the caller\(aqs > responsibility to > ensure the correct crypt() encryption scheme is used. This command > does not attempt to interpret or report on the encryption scheme. > > We add vertical space. Visible when viewed with man. Looks like an > improvement to me. > > Here's the first of these two spots in HTML: > > - > - class="docutils literal notranslate"> class="pre">guest-shutdown (Command) href="#qapidoc-31" title="Permalink to this heading"> > + > + class="docutils literal notranslate"> class="pre">guest-shutdown (Command) href="#qapidoc-30" title="Permalink to this heading"> > Initiate guest-activated shutdown. Note: this is an asynchronous > shutdown request, with no guarantee of successful shutdown. > > @@ -502,22 +502,20 @@ shutdown request, with no guarantee of s > > > > - > This command does NOT return a response on success. Success > condition is indicated by the VM exiting with a zero exit status or, > when running with –no-shutdown, by issuing the query-status QMP > command to confirm the VM status is “shutdown”. > - > - > -Since > + > +Since > 0.15.0 > > > > The id changes muddy the waters. With them manually removed: > > > class="docutils literal notranslate"> class="pre">guest-shutdown (Command) href="#qapidoc-31" title="Permalink to this heading"> > Initiate guest-activated shutdown. Note: this is an asynchronous > shutdown request, with no guarantee of successful shutdown. > > @@ -502,22 +502,20 @@ shutdown request, with no guarantee of s > > > > - > This command does NOT return a response on success. Success > condition is indicated by the VM exiting with a zero exit status or, > when running with –no-shutdown, by issuing the query-status QMP > command to confirm the VM status is “shutdown”. > - > > Since > 0.15.0 > > > > Makes no visual difference in my browser. > > Do these differences match your expectations? > Yep! It does change the output just a little, but Sphinx really doesn't like title-less sections. I thought the change looked fine, and I'm still planning on removing this old generator anyway, so... >
Re: [PATCH v2 04/21] docs/qapidoc: delint a tiny portion of the module
On Fri, Jun 28, 2024, 3:29 AM Markus Armbruster wrote: > John Snow writes: > > > In a forthcoming series that adds a new QMP documentation generator, it > > will be helpful to have a linting baseline. However, there's no need to > > shuffle around the deck chairs too much, because most of this code will > > be removed once that new qapidoc generator (the "transmogrifier") is in > > place. > > > > To ease my pain: just turn off the black auto-formatter for most, but > > not all, of qapidoc.py. This will help ensure that *new* code follows a > > coding standard without bothering too much with cleaning up the existing > > code. > > > > Code that I intend to keep is still subject to the delinting beam. > > > > Signed-off-by: John Snow > > Reviewed-by: Markus Armbruster > > Not an objection, just so you know: I still see a few C0411 like 'third > party import "import sphinx" should be placed before ...' > > R-by stands. > Yeah, I think it depends on precisely where you run the script. I think because the folder is named "sphinx" that it confuses the tools in certain contexts. I'm not worried about it because we don't have an enforcement paradigm yet - I stick to my little self-test script just to make sure I'm being self-consistent, but I figured I'd worry about broader compatibility later when I reshuffle the deck chairs for qapi.
Re: [PATCH v2 05/21] qapi/parser: preserve indentation in QAPIDoc sections
On Thu, Jun 27, 2024, 2:25 AM Markus Armbruster wrote: > John Snow writes: > > > Change get_doc_indented() to preserve indentation on all subsequent text > > lines, and create a compatibility dedent() function for qapidoc.py that > > removes indentation the same way get_doc_indented() did. > > > > This is being done for the benefit of a new qapidoc generator which > > requires that indentation in argument and features sections are > > preserved. > > > > Prior to this patch, a section like this: > > > > ``` > > @name: lorem ipsum > >dolor sit amet > > consectetur adipiscing elit > > ``` > > > > would have its body text be parsed into: > > > > ``` > > lorem ipsum > > dolor sit amet > > consectetur adipiscing elit > > ``` > > > > We want to preserve the indentation for even the first body line so that > > the entire block can be parsed directly as rST. This patch would now > > parse that segment into: > > > > ``` > > lorem ipsum > >dolor sit amet > > consectetur adipiscing elit > > ``` > > > > This is helpful for formatting arguments and features as field lists in > > rST, where the new generator will format this information as: > > > > ``` > > :arg type name: lorem ipsum > >dolor sit amet > > consectetur apidiscing elit > > ``` > > > > ...and can be formed by the simple concatenation of the field list > > construct and the body text. The indents help preserve the continuation > > of a block-level element, and further allow the use of additional rST > > block-level constructs such as code blocks, lists, and other such > > markup. > > > > This understandably breaks the existing qapidoc.py; so a new function is > > added there to dedent the text for compatibility. Once the new generator > > is merged, this function will not be needed any longer and can be > > dropped. > > I'll restore this paragraph if you don't mind: > > I verified this patch changes absolutely nothing by comparing the > md5sums of the QMP ref html pages both before and after the change, so > it's certified inert. QAPI test output has been updated to reflect the > new strategy of preserving indents for rST. > Nope, misunderstood the instruction. Go right ahead. > > Signed-off-by: John Snow > > [Edited commit message and code comments per review --js] > > And I'll drop this line. > Vestigial remnants of a more optimistic time. > > Reviewed-by: Markus Armbruster > >
Re: [PATCH v2 03/21] docs/qapidoc: remove unused intersperse function
On Thu, Jun 27, 2024, 2:02 AM Markus Armbruster wrote: > John Snow writes: > > > This function has been unused since fd62bff901b. > > > > Signed-off-by: John Snow > > Reviewed-by: Markus Armbruster > > I assume you won't mind me adding a bit of polish: "since commit > fd62bff901b (sphinx/qapidoc: Drop code to generate doc for simple union > tag)". > Please be my guest! --js (Looking at it, that's a very weird phrase, isn't it?) >
[PATCH 0/4] Python: Add 3.13 support, play linter whackamole
Fix some regressions in check-python-tox that have crept in since Pylint 3.x, and add Python 3.13 support to the pipeline. GitLab pipeline (before I fixed the missing DCO, but let's be honest, it can't possibly be worth re-running so many tests for just that): https://gitlab.com/jsnow/qemu/-/pipelines/1349737188 John Snow (4): python: linter changes for pylint 3.x python: Do not use pylint 3.2.4 with python 3.8 iotests: Change imports for Python 3.13 python: enable testing for 3.13 python/qemu/machine/machine.py | 1 + python/qemu/utils/qemu_ga_client.py| 2 +- python/setup.cfg | 4 +++- tests/docker/dockerfiles/python.docker | 1 + tests/qemu-iotests/testenv.py | 7 ++- tests/qemu-iotests/testrunner.py | 9 ++--- 6 files changed, 18 insertions(+), 6 deletions(-) -- 2.45.0
[PATCH 1/4] python: linter changes for pylint 3.x
New bleeding edge versions, new nits to iron out. This addresses the 'check-python-tox' optional GitLab test, while 'check-python-minreqs' saw no regressions, since it's frozen on an older version of pylint. Fixes: qemu/machine/machine.py:345:52: E0606: Possibly using variable 'sock' before assignment (possibly-used-before-assignment) qemu/utils/qemu_ga_client.py:168:4: R1711: Useless return at end of function or method (useless-return) Signed-off-by: John Snow --- python/qemu/machine/machine.py | 1 + python/qemu/utils/qemu_ga_client.py | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index f648f6af451..ebb58d5b68c 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -335,6 +335,7 @@ def binary(self) -> str: def _pre_launch(self) -> None: if self._qmp_set: +sock = None if self._monitor_address is None: self._sock_pair = socket.socketpair() os.set_inheritable(self._sock_pair[0].fileno(), True) diff --git a/python/qemu/utils/qemu_ga_client.py b/python/qemu/utils/qemu_ga_client.py index 9a665e6e990..cf0fcf9a8bb 100644 --- a/python/qemu/utils/qemu_ga_client.py +++ b/python/qemu/utils/qemu_ga_client.py @@ -174,7 +174,7 @@ def suspend(self, mode: str) -> None: # On error exception will raise except asyncio.TimeoutError: # On success command will timed out -return +pass def shutdown(self, mode: str = 'powerdown') -> None: if mode not in ['powerdown', 'halt', 'reboot']: -- 2.45.0
[PATCH 2/4] python: Do not use pylint 3.2.4 with python 3.8
There is a bug in this version, see: https://github.com/pylint-dev/pylint/issues/9751 Signed-off-by: John Snow --- python/setup.cfg | 1 + 1 file changed, 1 insertion(+) diff --git a/python/setup.cfg b/python/setup.cfg index 48668609d3e..8ebd345d7ed 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -41,6 +41,7 @@ devel = isort >= 5.1.2 mypy >= 1.4.0 pylint >= 2.17.3 +pylint != 3.2.4; python_version<"3.9" tox >= 3.18.0 urwid >= 2.1.2 urwid-readline >= 0.13 -- 2.45.0
[PATCH 4/4] python: enable testing for 3.13
Python 3.13 is in beta and Fedora 41 is preparing to make it the default system interpreter; enable testing for it. (In the event problems develop prior to release, it should only impact the check-python-tox job, which is not run by default and is allowed to fail.) Signed-off-by: John Snow --- python/setup.cfg | 3 ++- tests/docker/dockerfiles/python.docker | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/python/setup.cfg b/python/setup.cfg index 8ebd345d7ed..3b4e2cc5501 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -19,6 +19,7 @@ classifiers = Programming Language :: Python :: 3.10 Programming Language :: Python :: 3.11 Programming Language :: Python :: 3.12 +Programming Language :: Python :: 3.13 Typing :: Typed [options] @@ -184,7 +185,7 @@ multi_line_output=3 # of python available on your system to run this test. [tox:tox] -envlist = py38, py39, py310, py311, py312 +envlist = py38, py39, py310, py311, py312, py313 skip_missing_interpreters = true [testenv] diff --git a/tests/docker/dockerfiles/python.docker b/tests/docker/dockerfiles/python.docker index a3c1321190c..8f0af9ef25f 100644 --- a/tests/docker/dockerfiles/python.docker +++ b/tests/docker/dockerfiles/python.docker @@ -14,6 +14,7 @@ ENV PACKAGES \ python3.10 \ python3.11 \ python3.12 \ +python3.13 \ python3.8 \ python3.9 -- 2.45.0
[PATCH 3/4] iotests: Change imports for Python 3.13
Python 3.13 isn't out yet, but it's in beta and Fedora is ramping up to make it the default system interpreter for Fedora 41. They moved our cheese for where ContextManager lives; add a conditional to locate it while we support both pre-3.9 and 3.13+. Signed-off-by: John Snow --- tests/qemu-iotests/testenv.py| 7 ++- tests/qemu-iotests/testrunner.py | 9 ++--- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py index 588f30a4f14..96d69e56963 100644 --- a/tests/qemu-iotests/testenv.py +++ b/tests/qemu-iotests/testenv.py @@ -25,7 +25,12 @@ import random import subprocess import glob -from typing import List, Dict, Any, Optional, ContextManager +from typing import List, Dict, Any, Optional + +if sys.version_info >= (3, 9): +from contextlib import AbstractContextManager as ContextManager +else: +from typing import ContextManager DEF_GDB_OPTIONS = 'localhost:12345' diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py index 7b322272e92..2e236c8fa39 100644 --- a/tests/qemu-iotests/testrunner.py +++ b/tests/qemu-iotests/testrunner.py @@ -27,11 +27,14 @@ import shutil import sys from multiprocessing import Pool -from typing import List, Optional, Any, Sequence, Dict, \ -ContextManager - +from typing import List, Optional, Any, Sequence, Dict from testenv import TestEnv +if sys.version_info >= (3, 9): +from contextlib import AbstractContextManager as ContextManager +else: +from typing import ContextManager + def silent_unlink(path: Path) -> None: try: -- 2.45.0
[PATCH v2 15/21] docs/qapidoc: create qmp-example directive
This is a directive that creates a syntactic sugar for creating
"Example" boxes very similar to the ones already used in the bitmaps.rst
document, please see e.g.
https://www.qemu.org/docs/master/interop/bitmaps.html#creation-block-dirty-bitmap-add
In its simplest form, when a custom title is not needed or wanted, and
the example body is *solely* a QMP example:
```
.. qmp-example::
{body}
```
is syntactic sugar for:
```
.. admonition:: Example:
.. code-block:: QMP
{body}
```
When a custom, plaintext title that describes the example is desired,
this form:
```
.. qmp-example::
:title: Defrobnification
{body}
```
Is syntactic sugar for:
```
.. admonition:: Example: Defrobnification
.. code-block:: QMP
{body}
```
Lastly, when Examples are multi-step processes that require non-QMP
exposition, have lengthy titles, or otherwise involve prose with rST
markup (lists, cross-references, etc), the most complex form:
```
.. qmp-example::
:annotated:
This example shows how to use `foo-command`::
{body}
```
Is desugared to:
```
.. admonition:: Example:
This example shows how to use `foo-command`::
{body}
For more information, please see `frobnozz`.
```
The primary benefit here being documentation source consistently using
the same directive for all forms of examples to ensure consistent visual
styling, and ensuring all relevant prose is visually grouped alongside
the code literal block.
Note that as of this commit, the code-block rST syntax "::" does not
apply QMP highlighting; you would need to use ".. code-block:: QMP". The
very next commit changes this behavior to assume all "::" code blocks
within this directive are QMP blocks.
Signed-off-by: John Snow
---
docs/sphinx/qapidoc.py | 60 --
1 file changed, 58 insertions(+), 2 deletions(-)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index 43dd99e21e6..a2fa05fc491 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -27,16 +27,19 @@
import os
import re
import textwrap
+from typing import List
from docutils import nodes
-from docutils.parsers.rst import Directive, directives
+from docutils.parsers.rst import directives
from docutils.statemachine import ViewList
from qapi.error import QAPIError, QAPISemError
from qapi.gen import QAPISchemaVisitor
from qapi.schema import QAPISchema
import sphinx
+from sphinx.directives.code import CodeBlock
from sphinx.errors import ExtensionError
+from sphinx.util.docutils import SphinxDirective
from sphinx.util.nodes import nested_parse_with_titles
@@ -494,7 +497,7 @@ def visit_module(self, name):
super().visit_module(name)
-class NestedDirective(Directive):
+class NestedDirective(SphinxDirective):
def run(self):
raise NotImplementedError
@@ -567,10 +570,63 @@ def run(self):
raise ExtensionError(str(err)) from err
+class QMPExample(CodeBlock, NestedDirective):
+"""
+Custom admonition for QMP code examples.
+
+When the :annotated: option is present, the body of this directive
+is parsed as normal rST instead. Code blocks must be explicitly
+written by the user, but this allows for intermingling explanatory
+paragraphs with arbitrary rST syntax and code blocks for more
+involved examples.
+
+When :annotated: is absent, the directive body is treated as a
+simple standalone QMP code block literal.
+"""
+
+required_argument = 0
+optional_arguments = 0
+has_content = True
+option_spec = {
+"annotated": directives.flag,
+"title": directives.unchanged,
+}
+
+def admonition_wrap(self, *content) -> List[nodes.Node]:
+title = "Example:"
+if "title" in self.options:
+title = f"{title} {self.options['title']}"
+
+admon = nodes.admonition(
+"",
+nodes.title("", title),
+*content,
+classes=["admonition", "admonition-example"],
+)
+return [admon]
+
+def run_annotated(self) -> List[nodes.Node]:
+content_node: nodes.Element = nodes.section()
+self.do_parse(self.content, content_node)
+return content_node.children
+
+def run(self) -> List[nodes.Node]:
+annotated = "annotated" in self.options
+
+if annotated:
+content_nodes = self.run_annotated()
+else:
+self.arguments = ["QMP"]
+content_nodes = super().run()
+
+return self.admonition_wrap(*content_nodes)
+
+
def setup(app):
"""Register qapi-doc directive with Sphinx"""
app.add_config_value("qapidoc_srctree", None, "env")
app.add_directive("qapi-doc", QAPIDocDirective)
+app.add_directive("qmp-example", QMPExample)
return {
"version": __version__,
--
2.45.0
[PATCH v2 21/21] qapi: remove "Example" doc section
Fully eliminate the "Example" sections in QAPI doc blocks now that they
have all been converted to arbitrary rST syntax using the
".. qmp-example::" directive. Update tests to match.
Migrating to the new syntax
---
The old "Example:" or "Examples:" section syntax is now caught as an
error, but "Example::" is stil permitted as explicit rST syntax for an
un-lexed, generic preformatted text block.
('Example' is not special in this case, any sentence that ends with "::"
will start an indented code block in rST.)
Arbitrary rST for Examples are now possible, but it's strongly
recommended that documentation authors use the ".. qmp-example::"
directive for consistent visual formatting in rendered HTML docs. The
":title:" directive option may be used to add extra information into the
title bar for the example. The ":annotated:" option can be used to write
arbitrary rST instead, with nested "::" blocks applying QMP formatting
where desired.
Other choices available are ".. code-block:: QMP" which will not create
an "Example:" box, or the short-form "::" code-block syntax which will
not apply QMP highlighting when used outside of the qmp-example
directive.
Why?
This patch has several benefits:
1. Example sections can now be written more arbitrarily, mixing
explanatory paragraphs and code blocks however desired.
2. Example sections can now use fully arbitrary rST.
3. All code blocks are now lexed and validated as QMP; increasing
usability of the docs and ensuring validity of example snippets.
(To some extent - This patch only gaurantees it lexes correctly, not
that it's valid under the JSON or QMP grammars. It will catch most
small mistakes, however.)
4. Each qmp-example can be titled or annotated independently without
bypassing the QMP lexer/validator.
(i.e. code blocks are now for *code* only, so we don't have to
sacrifice exposition for having lexically valid examples.)
NOTE: As with the "Notes" conversion patch, this patch (and those
preceding) may change the rendering order for Examples in the
current generator. The forthcoming qapidoc rewrite will fix this
by always generating documentation in source order.
Signed-off-by: John Snow
---
docs/devel/qapi-code-gen.rst| 15 +++
scripts/qapi/parser.py | 10 +-
tests/qapi-schema/doc-good.json | 19 ---
tests/qapi-schema/doc-good.out | 26 ++
tests/qapi-schema/doc-good.txt | 23 ++-
5 files changed, 56 insertions(+), 37 deletions(-)
diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index ae97b335cbf..493e2b8f8df 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -995,8 +995,8 @@ line "Features:", like this::
# @feature: Description text
A tagged section begins with a paragraph that starts with one of the
-following words: "Since:", "Example:"/"Examples:", "Returns:",
-"Errors:", "TODO:". It ends with the start of a new section.
+following words: "Since:", "Returns:", "Errors:", "TODO:". It ends with
+the start of a new section.
The second and subsequent lines of tagged sections must be indented
like this::
@@ -1020,10 +1020,9 @@ detailing a relevant error condition. For example::
A "Since: x.y.z" tagged section lists the release that introduced the
definition.
-An "Example" or "Examples" section is rendered entirely
-as literal fixed-width text. "TODO" sections are not rendered at all
-(they are for developers, not users of QMP). In other sections, the
-text is formatted, and rST markup can be used.
+"TODO" sections are not rendered at all (they are for developers, not
+users of QMP). In other sections, the text is formatted, and rST markup
+can be used.
For example::
@@ -1058,11 +1057,11 @@ For example::
#
# Since: 0.14
#
- # Example:
+ # .. qmp-example::
#
# -> { "execute": "query-blockstats" }
# <- {
- # ... lots of output ...
+ # ...
#}
##
{ 'command': 'query-blockstats',
diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index 6ad5663e545..adc85b5b394 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -553,7 +553,7 @@ def get_doc(self) -> 'QAPIDoc':
# Note: "sections" with two colons are left alone as
# rST markup and not interpreted as a section heading.
-# TODO: Remove this error sometime in 2025 or so
+# TODO: Remove these errors sometime in 2025 or so
[PATCH v2 07/21] docs/qapidoc: fix nested parsing under untagged sections
Sphinx does not like sections without titles, because it wants to
convert every section into a reference. When there is no title, it
struggles to do this and transforms the tree inproperly.
Depending on the rST used, this may result in an assertion error deep in
the docutils HTMLWriter.
(Observed when using ".. admonition:: Notes" under such a section - When
this is transformed with its own element, Sphinx is fooled into
believing this title belongs to the section and incorrect mutates the
docutils tree, leading to errors during rendering time.)
When parsing an untagged section (free paragraphs), skip making a hollow
section and instead append the parse results to the prior section.
Many Bothans died to bring us this information.
Signed-off-by: John Snow
Acked-by: Markus Armbruster
---
docs/sphinx/qapidoc.py | 16 +++-
1 file changed, 11 insertions(+), 5 deletions(-)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index f9683444b14..efcd84656fa 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -272,14 +272,20 @@ def _nodes_for_sections(self, doc):
if section.tag and section.tag == 'TODO':
# Hide TODO: sections
continue
+
+if not section.tag:
+# Sphinx cannot handle sectionless titles;
+# Instead, just append the results to the prior section.
+container = nodes.container()
+self._parse_text_into_node(section.text, container)
+nodelist += container.children
+continue
+
snode = self._make_section(section.tag)
-if section.tag and section.tag.startswith('Example'):
+if section.tag.startswith('Example'):
snode += self._nodes_for_example(dedent(section.text))
else:
-self._parse_text_into_node(
-dedent(section.text) if section.tag else section.text,
-snode,
-)
+self._parse_text_into_node(dedent(section.text), snode)
nodelist.append(snode)
return nodelist
--
2.45.0
[PATCH v2 20/21] qapi: convert "Example" sections with longer prose
These examples require longer explanations or have explanations that
require markup to look reasonable when rendered and so use the longer
form of the ".. qmp-example::" directive.
By using the :annotated: option, the content in the example block is
assumed *not* to be a code block literal and is instead parsed as normal
rST - with the exception that any code literal blocks after `::` will
assumed to be a QMP code literal block.
Note: There's one title-less conversion in this patch that comes along
for the ride because it's part of a larger "Examples" block that was
better to convert all at once.
See commit-5: "docs/qapidoc: create qmp-example directive", for a
detailed explanation of this custom directive syntax.
See commit+1: "qapi: remove "Example" doc section" for a detailed
explanation of why.
Signed-off-by: John Snow
---
qapi/block.json | 26 --
qapi/machine.json | 30 --
qapi/migration.json | 7 +--
qapi/virtio.json| 24 ++--
4 files changed, 59 insertions(+), 28 deletions(-)
diff --git a/qapi/block.json b/qapi/block.json
index 5ddd061e964..d95e9fd8140 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -545,31 +545,37 @@
#
# Since: 4.0
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# Set new histograms for all io types with intervals
-# [0, 10), [10, 50), [50, 100), [100, +inf):
+#Set new histograms for all io types with intervals
+#[0, 10), [10, 50), [50, 100), [100, +inf)::
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0",
# "boundaries": [10, 50, 100] } }
# <- { "return": {} }
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# Set new histogram only for write, other histograms will remain
-# not changed (or not created):
+#Set new histogram only for write, other histograms will remain
+#not changed (or not created)::
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0",
# "boundaries-write": [10, 50, 100] } }
# <- { "return": {} }
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# Set new histograms with the following intervals:
-# read, flush: [0, 10), [10, 50), [50, 100), [100, +inf)
-# write: [0, 1000), [1000, 5000), [5000, +inf)
+#Set new histograms with the following intervals:
+#
+#- read, flush: [0, 10), [10, 50), [50, 100), [100, +inf)
+#- write: [0, 1000), [1000, 5000), [5000, +inf)
+#
+#::
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0",
diff --git a/qapi/machine.json b/qapi/machine.json
index 83f60b319c7..0a5ffe652b7 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -1047,10 +1047,11 @@
#
# Since: 2.7
#
-# Examples:
+# .. qmp-example::
+#:annotated:
#
-# For pseries machine type started with -smp 2,cores=2,maxcpus=4
-# -cpu POWER8:
+#For pseries machine type started with
+#``-smp 2,cores=2,maxcpus=4 -cpu POWER8``::
#
# -> { "execute": "query-hotpluggable-cpus" }
# <- {"return": [
@@ -1060,7 +1061,10 @@
#"vcpus-count": 1, "qom-path": "/machine/unattached/device[0]"}
#]}
#
-# For pc machine type started with -smp 1,maxcpus=2:
+# .. qmp-example::
+#:annotated:
+#
+#For pc machine type started with ``-smp 1,maxcpus=2``::
#
# -> { "execute": "query-hotpluggable-cpus" }
# <- {"return": [
@@ -1075,8 +1079,11 @@
# }
#]}
#
-# For s390x-virtio-ccw machine type started with -smp 1,maxcpus=2
-# -cpu qemu (Since: 2.11):
+# .. qmp-example::
+#:annotated:
+#
+#For s390x-virtio-ccw machine type started with
+#``-smp 1,maxcpus=2 -cpu qemu`` (Since: 2.11)::
#
# -> { "execute": "query-hotpluggable-cpus" }
# <- {"return": [
@@ -1130,12 +1137,15 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# -> { "execute": "balloon", "arguments": { "value": 536870912 } }
-# <- { "return": {} }
+#::
#
-# With a 2.5GiB guest this command inflated the ballon to 3GiB.
+# -> { "execute": "balloon", "arguments": { "value": 536870912 } }
+# <- { "return": {} }
+#
+#With a 2.5GiB guest this command inflated the ballon to 3GiB.
##
{ 'command': 'balloon', 'data': {'valu
[PATCH v2 10/21] qapi: convert "Note" sections to plain rST
We do not need a dedicated section for notes. By eliminating a specially parsed section, these notes can be treated as normal rST paragraphs in the new QMP reference manual, and can be placed and styled much more flexibly. Convert all existing "Note" and "Notes" sections to pure rST. As part of the conversion, capitalize the first letter of each sentence and add trailing punctuation where appropriate to ensure notes look sensible and consistent in rendered HTML documentation. Markup is also re-aligned to the de-facto standard of 3 spaces for directives. Update docs/devel/qapi-code-gen.rst to reflect the new paradigm, and update the QAPI parser to prohibit "Note" sections while suggesting a new syntax. The exact formatting to use is a matter of taste, but a good candidate is simply: .. note:: lorem ipsum ... ... dolor sit amet ... ... consectetur adipiscing elit ... ... but there are other choices, too. The Sphinx readthedocs theme offers theming for the following forms (capitalization unimportant); all are adorned with a (!) symbol () in the title bar for rendered HTML docs. See https://sphinx-rtd-theme.readthedocs.io/en/stable/demo/demo.html#admonitions for examples of each directive/admonition in use. These are rendered in orange: .. Attention:: ... .. Caution:: ... .. WARNING:: ... These are rendered in red: .. DANGER:: ... .. Error:: ... These are rendered in green: .. Hint:: ... .. Important:: ... .. Tip:: ... These are rendered in blue: .. Note:: ... .. admonition:: custom title admonition body text This patch uses ".. note::" almost everywhere, with just two "caution" directives. Several instances of "Notes:" have been converted to merely ".. note::" where appropriate, but ".. admonition:: notes" is used in a few places where we had an ordered list of multiple notes that would not make sense as standalone/separate admonitions. NOTE: Because qapidoc.py does not attempt to preserve source ordering of sections, the conversion of Notes from a "tagged section" to an "untagged section" means that rendering order for some notes *may change* as a result of this patch. The forthcoming qapidoc.py rewrite strictly preserves source ordering in the rendered documentation, so this issue will be rectified in the new generator. Signed-off-by: John Snow Acked-by: Stefan Hajnoczi [for block*.json] --- docs/devel/qapi-code-gen.rst | 7 +- qapi/block-core.json | 28 +++--- qapi/block.json | 2 +- qapi/char.json| 12 +-- qapi/control.json | 17 ++-- qapi/dump.json| 2 +- qapi/introspect.json | 6 +- qapi/machine-target.json | 26 +++--- qapi/machine.json | 47 +- qapi/migration.json | 12 +-- qapi/misc.json| 88 +-- qapi/net.json | 6 +- qapi/pci.json | 8 +- qapi/qdev.json| 28 +++--- qapi/qom.json | 17 ++-- qapi/rocker.json | 16 ++-- qapi/run-state.json | 18 ++-- qapi/sockets.json | 10 +-- qapi/stats.json | 22 ++--- qapi/transaction.json | 8 +- qapi/ui.json | 29 +++--- qapi/virtio.json | 12 +-- qga/qapi-schema.json | 48 +- scripts/qapi/parser.py| 15 tests/qapi-schema/doc-empty-section.err | 2 +- tests/qapi-schema/doc-empty-section.json | 2 +- tests/qapi-schema/doc-good.json | 4 +- tests/qapi-schema/doc-good.out| 8 +- tests/qapi-schema/doc-good.txt| 10 +-- .../qapi-schema/doc-interleaved-section.json | 2 +- 30 files changed, 260 insertions(+), 252 deletions(-) diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst index cee43222f19..ae97b335cbf 100644 --- a/docs/devel/qapi-code-gen.rst +++ b/docs/devel/qapi-code-gen.rst @@ -995,14 +995,13 @@ line "Features:", like this:: # @feature: Description text A tagged section begins with a paragraph that starts with one of the -following words: "Note:"/"Notes:", "Since:", "Example:"/"Examples:", -"Returns:", "Errors:", "TODO:". It ends with the start of a new -section. +following words: "Since:", "Example:"/"Examples:", "Returns:", +"Errors:", "TODO:". It ends with the start of a new sectio
[PATCH v2 02/21] qapi: linter fixups
Fix minor irritants to pylint/flake8 et al.
(Yes, these need to be guarded by the Python tests. That's a work in
progress, a series that's quite likely to follow once I finish this
Sphinx project. Please pardon the temporary irritation.)
Signed-off-by: John Snow
Reviewed-by: Markus Armbruster
---
scripts/qapi/introspect.py | 8
scripts/qapi/schema.py | 6 +++---
scripts/qapi/visit.py | 5 +++--
3 files changed, 10 insertions(+), 9 deletions(-)
diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
index 86c075a6ad2..ac14b20f308 100644
--- a/scripts/qapi/introspect.py
+++ b/scripts/qapi/introspect.py
@@ -27,8 +27,8 @@
from .schema import (
QAPISchema,
QAPISchemaAlternatives,
-QAPISchemaBranches,
QAPISchemaArrayType,
+QAPISchemaBranches,
QAPISchemaBuiltinType,
QAPISchemaEntity,
QAPISchemaEnumMember,
@@ -233,9 +233,9 @@ def _use_type(self, typ: QAPISchemaType) -> str:
typ = type_int
elif (isinstance(typ, QAPISchemaArrayType) and
typ.element_type.json_type() == 'int'):
-type_intList = self._schema.lookup_type('intList')
-assert type_intList
-typ = type_intList
+type_intlist = self._schema.lookup_type('intList')
+assert type_intlist
+typ = type_intlist
# Add type to work queue if new
if typ not in self._used_types:
self._used_types.append(typ)
diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
index 721c470d2b8..d65c35f6ee6 100644
--- a/scripts/qapi/schema.py
+++ b/scripts/qapi/schema.py
@@ -730,6 +730,7 @@ def set_defined_in(self, name: str) -> None:
for v in self.variants:
v.set_defined_in(name)
+# pylint: disable=unused-argument
def check(
self, schema: QAPISchema, seen: Dict[str, QAPISchemaMember]
) -> None:
@@ -1166,7 +1167,7 @@ def _def_definition(self, defn: QAPISchemaDefinition) ->
None:
defn.info, "%s is already defined" % other_defn.describe())
self._entity_dict[defn.name] = defn
-def lookup_entity(self,name: str) -> Optional[QAPISchemaEntity]:
+def lookup_entity(self, name: str) -> Optional[QAPISchemaEntity]:
return self._entity_dict.get(name)
def lookup_type(self, name: str) -> Optional[QAPISchemaType]:
@@ -1302,11 +1303,10 @@ def _make_implicit_object_type(
name = 'q_obj_%s-%s' % (name, role)
typ = self.lookup_entity(name)
if typ:
-assert(isinstance(typ, QAPISchemaObjectType))
+assert isinstance(typ, QAPISchemaObjectType)
# The implicit object type has multiple users. This can
# only be a duplicate definition, which will be flagged
# later.
-pass
else:
self._def_definition(QAPISchemaObjectType(
name, info, None, ifcond, None, None, members, None))
diff --git a/scripts/qapi/visit.py b/scripts/qapi/visit.py
index e766acaac92..12f92e429f6 100644
--- a/scripts/qapi/visit.py
+++ b/scripts/qapi/visit.py
@@ -280,8 +280,9 @@ def gen_visit_alternate(name: str,
abort();
default:
assert(visit_is_input(v));
-error_setg(errp, "Invalid parameter type for '%%s', expected:
%(name)s",
- name ? name : "null");
+error_setg(errp,
+ "Invalid parameter type for '%%s', expected: %(name)s",
+ name ? name : "null");
/* Avoid passing invalid *obj to qapi_free_%(c_name)s() */
g_free(*obj);
*obj = NULL;
--
2.45.0
[PATCH v2 13/21] qapi/parser: don't parse rST markup as section headers
The double-colon synax is rST formatting that precedes a literal code
block. We do not want to capture these as QAPI-specific sections.
Coerce blocks that start with e.g. "Example::" to be parsed as untagged
paragraphs instead of special tagged sections.
Signed-off-by: John Snow
---
scripts/qapi/parser.py | 9 +++--
tests/qapi-schema/doc-good.json | 3 +++
tests/qapi-schema/doc-good.out | 3 +++
tests/qapi-schema/doc-good.txt | 3 +++
4 files changed, 16 insertions(+), 2 deletions(-)
diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index 0a13f0f541a..6ad5663e545 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -544,10 +544,15 @@ def get_doc(self) -> 'QAPIDoc':
line = self.get_doc_indented(doc)
no_more_args = True
elif match := re.match(
-r'(Returns|Errors|Since|Notes?|Examples?|TODO): *',
-line):
+r'(Returns|Errors|Since|Notes?|Examples?|TODO)'
+r'(?!::): *',
+line,
+):
# tagged section
+# Note: "sections" with two colons are left alone as
+# rST markup and not interpreted as a section heading.
+
# TODO: Remove this error sometime in 2025 or so
# after we've fully transitioned to the new qapidoc
# generator.
diff --git a/tests/qapi-schema/doc-good.json b/tests/qapi-schema/doc-good.json
index 32ff910b4f8..107123f8a8d 100644
--- a/tests/qapi-schema/doc-good.json
+++ b/tests/qapi-schema/doc-good.json
@@ -181,6 +181,9 @@
# - *verbatim*
# - {braces}
#
+# Note::
+#Ceci n'est pas une note
+#
# Since: 2.10
##
{ 'command': 'cmd',
diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out
index 631dc9f8dad..bd876b6542d 100644
--- a/tests/qapi-schema/doc-good.out
+++ b/tests/qapi-schema/doc-good.out
@@ -190,6 +190,9 @@ frobnicate
section=Examples
- *verbatim*
- {braces}
+section=None
+Note::
+ Ceci n'est pas une note
section=Since
2.10
doc symbol=cmd-boxed
diff --git a/tests/qapi-schema/doc-good.txt b/tests/qapi-schema/doc-good.txt
index d8bfa742c2f..30d457e5488 100644
--- a/tests/qapi-schema/doc-good.txt
+++ b/tests/qapi-schema/doc-good.txt
@@ -231,6 +231,9 @@ Examples
- *verbatim*
- {braces}
+Note::
+ Ceci n'est pas une note
+
Since
~
--
2.45.0
[PATCH v2 18/21] qapi: convert "Example" sections without titles
Use the no-option form of ".. qmp-example::" to convert any Examples
that do not have any form of caption or explanation whatsoever.
See commit-3: "docs/qapidoc: create qmp-example directive", for a
detailed explanation of this custom directive syntax.
See commit+3: "qapi: remove "Example" doc section" for a detailed
explanation of why.
Signed-off-by: John Snow
---
qapi/acpi.json | 4 +--
qapi/block-core.json | 64 +---
qapi/block.json | 18 ++-
qapi/char.json | 24 +--
qapi/control.json| 8 ++---
qapi/dump.json | 8 ++---
qapi/machine-target.json | 2 +-
qapi/machine.json| 38
qapi/migration.json | 58 ++--
qapi/misc-target.json| 22 +++---
qapi/misc.json | 32 ++--
qapi/net.json| 20 +++--
qapi/pci.json| 2 +-
qapi/qdev.json | 10 ---
qapi/qom.json| 8 ++---
qapi/replay.json | 8 ++---
qapi/rocker.json | 8 ++---
qapi/run-state.json | 30 +--
qapi/tpm.json| 6 ++--
qapi/trace.json | 4 +--
qapi/transaction.json| 2 +-
qapi/ui.json | 34 ++---
qapi/vfio.json | 2 +-
qapi/virtio.json | 2 +-
qapi/yank.json | 4 +--
25 files changed, 216 insertions(+), 202 deletions(-)
diff --git a/qapi/acpi.json b/qapi/acpi.json
index aa4dbe57943..045dab6228b 100644
--- a/qapi/acpi.json
+++ b/qapi/acpi.json
@@ -111,7 +111,7 @@
#
# Since: 2.1
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-acpi-ospm-status" }
# <- { "return": [ { "device": "d1", "slot": "0", "slot-type": "DIMM",
"source": 1, "status": 0},
@@ -131,7 +131,7 @@
#
# Since: 2.1
#
-# Example:
+# .. qmp-example::
#
# <- { "event": "ACPI_DEVICE_OST",
# "data": { "info": { "device": "d1", "slot": "0",
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 9ef23ec02ae..4e0f0395146 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -764,7 +764,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-block" }
# <- {
@@ -1168,7 +1168,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-blockstats" }
# <- {
@@ -1461,7 +1461,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "block_resize",
# "arguments": { "device": "scratch", "size": 1073741824 } }
@@ -1682,7 +1682,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "blockdev-snapshot-sync",
# "arguments": { "device": "ide-hd0",
@@ -1715,7 +1715,7 @@
#
# Since: 2.5
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "blockdev-add",
# "arguments": { "driver": "qcow2",
@@ -1861,7 +1861,7 @@
#
# Since: 1.3
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "block-commit",
# "arguments": { "device": "virtio0",
@@ -1899,7 +1899,7 @@
#
# Since: 1.6
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "drive-backup",
# "arguments": { "device": "drive0",
@@ -1925,7 +1925,7 @@
#
# Since: 2.3
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "blockdev-backup",
# "arguments": { "device": "src-id",
@@ -1949,7 +1949,7 @@
#
# Since: 2.0
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-named-block-nodes" }
# <- { "return": [ { "ro":false,
@@ -2130,7 +2130,7 @@
#
# Since: 1.3
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "drive-mirror",
# "arguments": { "device": "ide-hd0",
@@ -2307,7 +2307,7 @@
#
# Since: 2.4
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "block-dirty-bitmap-add",
# "arguments": { "node": "drive0", "name": "bitmap0" } }
@@ -2331,7 +2331,7 @@
#
# Since: 2.4
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "block-dirty-bitmap-remove",
# "arguments": { "no
[PATCH v2 14/21] docs/qapidoc: factor out do_parse()
Factor out the compatibility parser helper so it can be shared by other directives. Signed-off-by: John Snow --- docs/sphinx/qapidoc.py | 64 +++--- 1 file changed, 35 insertions(+), 29 deletions(-) diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py index efcd84656fa..43dd99e21e6 100644 --- a/docs/sphinx/qapidoc.py +++ b/docs/sphinx/qapidoc.py @@ -494,7 +494,41 @@ def visit_module(self, name): super().visit_module(name) -class QAPIDocDirective(Directive): +class NestedDirective(Directive): +def run(self): +raise NotImplementedError + +def do_parse(self, rstlist, node): +""" +Parse rST source lines and add them to the specified node + +Take the list of rST source lines rstlist, parse them as +rST, and add the resulting docutils nodes as children of node. +The nodes are parsed in a way that allows them to include +subheadings (titles) without confusing the rendering of +anything else. +""" +# This is from kerneldoc.py -- it works around an API change in +# Sphinx between 1.6 and 1.7. Unlike kerneldoc.py, we use +# sphinx.util.nodes.nested_parse_with_titles() rather than the +# plain self.state.nested_parse(), and so we can drop the saving +# of title_styles and section_level that kerneldoc.py does, +# because nested_parse_with_titles() does that for us. +if USE_SSI: +with switch_source_input(self.state, rstlist): +nested_parse_with_titles(self.state, rstlist, node) +else: +save = self.state.memo.reporter +self.state.memo.reporter = AutodocReporter( +rstlist, self.state.memo.reporter +) +try: +nested_parse_with_titles(self.state, rstlist, node) +finally: +self.state.memo.reporter = save + + +class QAPIDocDirective(NestedDirective): """Extract documentation from the specified QAPI .json file""" required_argument = 1 @@ -532,34 +566,6 @@ def run(self): # so they are displayed nicely to the user raise ExtensionError(str(err)) from err -def do_parse(self, rstlist, node): -"""Parse rST source lines and add them to the specified node - -Take the list of rST source lines rstlist, parse them as -rST, and add the resulting docutils nodes as children of node. -The nodes are parsed in a way that allows them to include -subheadings (titles) without confusing the rendering of -anything else. -""" -# This is from kerneldoc.py -- it works around an API change in -# Sphinx between 1.6 and 1.7. Unlike kerneldoc.py, we use -# sphinx.util.nodes.nested_parse_with_titles() rather than the -# plain self.state.nested_parse(), and so we can drop the saving -# of title_styles and section_level that kerneldoc.py does, -# because nested_parse_with_titles() does that for us. -if USE_SSI: -with switch_source_input(self.state, rstlist): -nested_parse_with_titles(self.state, rstlist, node) -else: -save = self.state.memo.reporter -self.state.memo.reporter = AutodocReporter( -rstlist, self.state.memo.reporter -) -try: -nested_parse_with_titles(self.state, rstlist, node) -finally: -self.state.memo.reporter = save - def setup(app): """Register qapi-doc directive with Sphinx""" -- 2.45.0
[PATCH v2 16/21] docs/qapidoc: add QMP highlighting to annotated qmp-example blocks
For any code literal blocks inside of a qmp-example directive, apply and
enforce the QMP lexer/highlighter to those blocks.
This way, you won't need to write:
```
.. qmp-example::
:annotated:
Blah blah
.. code-block:: QMP
-> { "lorem": "ipsum" }
```
But instead, simply:
```
.. qmp-example::
:annotated:
Blah blah::
-> { "lorem": "ipsum" }
```
Once the directive block is exited, whatever the previous default
highlight language was will be restored; localizing the forced QMP
lexing to exclusively this directive.
Signed-off-by: John Snow
---
docs/sphinx/qapidoc.py | 53 ++
1 file changed, 49 insertions(+), 4 deletions(-)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index a2fa05fc491..c8c404ad1b0 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -26,6 +26,7 @@
import os
import re
+import sys
import textwrap
from typing import List
@@ -37,6 +38,7 @@
from qapi.schema import QAPISchema
import sphinx
+from sphinx import addnodes
from sphinx.directives.code import CodeBlock
from sphinx.errors import ExtensionError
from sphinx.util.docutils import SphinxDirective
@@ -575,10 +577,10 @@ class QMPExample(CodeBlock, NestedDirective):
Custom admonition for QMP code examples.
When the :annotated: option is present, the body of this directive
-is parsed as normal rST instead. Code blocks must be explicitly
-written by the user, but this allows for intermingling explanatory
-paragraphs with arbitrary rST syntax and code blocks for more
-involved examples.
+is parsed as normal rST, but with any '::' code blocks set to use
+the QMP lexer. Code blocks must be explicitly written by the user,
+but this allows for intermingling explanatory paragraphs with
+arbitrary rST syntax and code blocks for more involved examples.
When :annotated: is absent, the directive body is treated as a
simple standalone QMP code block literal.
@@ -592,6 +594,33 @@ class QMPExample(CodeBlock, NestedDirective):
"title": directives.unchanged,
}
+def _highlightlang(self) -> addnodes.highlightlang:
+"""Return the current highlightlang setting for the document"""
+node = None
+doc = self.state.document
+
+if hasattr(doc, "findall"):
+# docutils >= 0.18.1
+for node in doc.findall(addnodes.highlightlang):
+pass
+else:
+for elem in doc.traverse():
+if isinstance(elem, addnodes.highlightlang):
+node = elem
+
+if node:
+return node
+
+# No explicit directive found, use defaults
+node = addnodes.highlightlang(
+lang=self.env.config.highlight_language,
+force=False,
+# Yes, Sphinx uses this value to effectively disable line
+# numbers and not 0 or None or -1 or something. ¯\_(ツ)_/¯
+linenothreshold=sys.maxsize,
+)
+return node
+
def admonition_wrap(self, *content) -> List[nodes.Node]:
title = "Example:"
if "title" in self.options:
@@ -606,8 +635,24 @@ def admonition_wrap(self, *content) -> List[nodes.Node]:
return [admon]
def run_annotated(self) -> List[nodes.Node]:
+lang_node = self._highlightlang()
+
content_node: nodes.Element = nodes.section()
+
+# Configure QMP highlighting for "::" blocks, if needed
+if lang_node["lang"] != "QMP":
+content_node += addnodes.highlightlang(
+lang="QMP",
+force=False, # "True" ignores lexing errors
+linenothreshold=lang_node["linenothreshold"],
+)
+
self.do_parse(self.content, content_node)
+
+# Restore prior language highlighting, if needed
+if lang_node["lang"] != "QMP":
+content_node += addnodes.highlightlang(**lang_node.attributes)
+
return content_node.children
def run(self) -> List[nodes.Node]:
--
2.45.0
[PATCH v2 17/21] docs/sphinx: add CSS styling for qmp-example directive
From: Harmonie Snow
Add CSS styling for qmp-example directives to increase readability and
consistently style all example blocks.
Signed-off-by: Harmonie Snow
Signed-off-by: John Snow
---
docs/sphinx-static/theme_overrides.css | 46 ++
1 file changed, 46 insertions(+)
diff --git a/docs/sphinx-static/theme_overrides.css
b/docs/sphinx-static/theme_overrides.css
index c70ef951286..f3223af62b2 100644
--- a/docs/sphinx-static/theme_overrides.css
+++ b/docs/sphinx-static/theme_overrides.css
@@ -87,6 +87,52 @@ div[class^="highlight"] pre {
padding-bottom: 1px;
}
+/* qmp-example directive styling */
+
+.rst-content .admonition-example {
+background-color: #fcfcfc;
+padding: 0px;
+}
+
+.rst-content .admonition-example > .admonition-title {
+background-color: #338254;
+margin: 0px;
+}
+
+.rst-content .admonition-example > div[class^=highlight] {
+border-top: 1px solid #d7f0d7;
+border-bottom: 1px solid #d7f0d7;
+border-left: 3px solid #d7f0d7;
+border-right: 3px solid #d7f0d7;
+}
+
+.rst-content .admonition-example .highlight {
+background: linear-gradient(#d9f1d9 1%, #ecf8ec 10%,
+#ecf8ec 90%, #d9f1d9 99%);
+}
+
+.rst-content .admonition-example > .admonition-title:before {
+content: "🧩";
+}
+
+.rst-content .admonition-example > .admonition-title p {
+margin: 0px;
+}
+
+.rst-content .admonition-example p {
+padding: 15px 10px;
+margin: 0px;
+background-color: #fafafa;
+border-left: 1px solid #ededed;
+border-right: 1px solid #ededed;
+}
+
+.rst-content .admonition-example > .admonition {
+background-color: #fcfcfc;
+}
+
+/* end qmp-example styling */
+
@media screen {
/* content column
--
2.45.0
[PATCH v2 01/21] [DO-NOT-MERGE]: Add some ad-hoc linting helpers.
These aren't ready for upstream inclusion, because they do not properly manage version dependencies, execution environment and so on. These are just the tools I use in my Own Special Environment :tm: for testing and debugging. They've been tested only on Fedora 38 for right now, which means: Python 3.11.4-1.fc38 pylint 2.17.4-2.fc38 mypy 1.4.0-1.fc38 isort 5.12.0-1.fc38 flake8 5.0.3-2.fc38 "Soon" :tm: I'll move the qapi generator code under the python/ directory to take advantage of the more robust linting infrastructure there, but that's going to happen after this series. Sorry about that! Signed-off-by: John Snow --- scripts/qapi-lint.sh | 61 +++ scripts/qapi/Makefile | 5 2 files changed, 66 insertions(+) create mode 100755 scripts/qapi-lint.sh create mode 100644 scripts/qapi/Makefile diff --git a/scripts/qapi-lint.sh b/scripts/qapi-lint.sh new file mode 100755 index 000..732716f2444 --- /dev/null +++ b/scripts/qapi-lint.sh @@ -0,0 +1,61 @@ +#!/usr/bin/env bash +set -e + +if [[ -f qapi/.flake8 ]]; then +echo "flake8 --config=qapi/.flake8 qapi/" +flake8 --config=qapi/.flake8 qapi/ +fi +if [[ -f qapi/pylintrc ]]; then +echo "pylint --rcfile=qapi/pylintrc qapi/" +pylint --rcfile=qapi/pylintrc qapi/ +fi +if [[ -f qapi/mypy.ini ]]; then +echo "mypy --config-file=qapi/mypy.ini qapi/" +mypy --config-file=qapi/mypy.ini qapi/ +fi + +if [[ -f qapi/.isort.cfg ]]; then +pushd qapi +echo "isort -c ." +isort -c . +popd +fi + +if [[ -f ../docs/sphinx/qapi-domain.py ]]; then +pushd ../docs/sphinx + +echo "mypy --strict qapi-domain.py" +mypy --strict qapi-domain.py +echo "flake8 qapi-domain.py --max-line-length=99" +flake8 qapi-domain.py --max-line-length=99 +echo "isort qapi-domain.py" +isort qapi-domain.py +echo "black --check qapi-domain.py" +black --check qapi-domain.py + +popd +fi + +if [[ -f ../docs/sphinx/qapidoc.py ]]; then +pushd ../docs/sphinx + +echo "pylint --rc-file ../../scripts/qapi/pylintrc qapidoc.py" +PYTHONPATH=../scripts/ pylint \ + --rc-file ../../scripts/qapi/pylintrc \ + qapidoc.py +echo "flake8 qapidoc.py --max-line-length=80" +flake8 qapidoc.py --max-line-length=80 +echo "isort qapidoc.py" +isort qapidoc.py +echo "black --line-length 80 --check qapidoc.py" +black --line-length 80 --check qapidoc.py + +popd +fi + +pushd ../build +make -j13 +make check-qapi-schema +make docs +make sphinxdocs +popd diff --git a/scripts/qapi/Makefile b/scripts/qapi/Makefile new file mode 100644 index 000..314e8a5505e --- /dev/null +++ b/scripts/qapi/Makefile @@ -0,0 +1,5 @@ +check: + isort -c . + flake8 . + cd .. && pylint --rcfile=qapi/pylintrc qapi + cd .. && mypy -p qapi --config-file=qapi/mypy.ini -- 2.45.0
[PATCH v2 12/21] qapi: add markup to note blocks
Generally, surround command-line options with ``literal`` markup to help
it stand out from prose in rendered HTML, and add cross-references to
replace "see also" messages.
References to types, values, and other QAPI definitions are not yet
adjusted here; they will be converted en masse in a subsequent patch
after the new QAPI doc generator is merged.
Signed-off-by: John Snow
Reviewed-by: Markus Armbruster
---
qapi/control.json | 4 ++--
qapi/misc.json | 8
qapi/qdev.json | 2 +-
qapi/run-state.json | 2 +-
qapi/sockets.json | 2 +-
qapi/ui.json| 2 +-
6 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/qapi/control.json b/qapi/control.json
index 59d5e00c151..fe2af45120b 100644
--- a/qapi/control.json
+++ b/qapi/control.json
@@ -24,8 +24,8 @@
#
# .. note:: This command is valid exactly when first connecting: it must
#be issued before any other command will be accepted, and will fail
-#once the monitor is accepting other commands. (see qemu
-#docs/interop/qmp-spec.rst)
+#once the monitor is accepting other commands.
+#(see :doc:`/interop/qmp-spec`)
#
# .. note:: The QMP client needs to explicitly enable QMP capabilities,
#otherwise all the QMP capabilities will be turned off by default.
diff --git a/qapi/misc.json b/qapi/misc.json
index 13ea82f5254..b04efbadec6 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -104,7 +104,7 @@
# Returns a list of information about each iothread.
#
# .. note:: This list excludes the QEMU main loop thread, which is not
-#declared using the -object iothread command-line option. It is
+#declared using the ``-object iothread`` command-line option. It is
#always the main thread of the process.
#
# Returns: a list of @IOThreadInfo for each iothread
@@ -138,8 +138,8 @@
#
# .. note:: This function will succeed even if the guest is already in
#the stopped state. In "inmigrate" state, it will ensure that the
-#guest remains paused once migration finishes, as if the -S option
-#was passed on the command line.
+#guest remains paused once migration finishes, as if the ``-S``
+#option was passed on the command line.
#
#In the "suspended" state, it will completely stop the VM and cause
#a transition to the "paused" state. (Since 9.0)
@@ -161,7 +161,7 @@
# .. note:: This command will succeed if the guest is currently running.
#It will also succeed if the guest is in the "inmigrate" state; in
#this case, the effect of the command is to make sure the guest
-#starts once migration finishes, removing the effect of the -S
+#starts once migration finishes, removing the effect of the ``-S``
#command line option if it was passed.
#
#If the VM was previously suspended, and not been reset or woken,
diff --git a/qapi/qdev.json b/qapi/qdev.json
index f5b35a814fe..d031fc3590d 100644
--- a/qapi/qdev.json
+++ b/qapi/qdev.json
@@ -59,7 +59,7 @@
#the 'docs/qdev-device-use.txt' file.
#
# 3. It's possible to list device properties by running QEMU with
-#the "-device DEVICE,help" command-line argument, where DEVICE
+#the ``-device DEVICE,help`` command-line argument, where DEVICE
#is the device's name.
#
# Example:
diff --git a/qapi/run-state.json b/qapi/run-state.json
index dc524234ace..252d7d6afa7 100644
--- a/qapi/run-state.json
+++ b/qapi/run-state.json
@@ -146,7 +146,7 @@
# @reason: The @ShutdownCause which resulted in the SHUTDOWN.
# (since 4.0)
#
-# .. note:: If the command-line option "-no-shutdown" has been
+# .. note:: If the command-line option ``-no-shutdown`` has been
#specified, qemu will not exit, and a STOP event will eventually
#follow the SHUTDOWN event.
#
diff --git a/qapi/sockets.json b/qapi/sockets.json
index 3970118bf4e..4d78d2ccb77 100644
--- a/qapi/sockets.json
+++ b/qapi/sockets.json
@@ -181,7 +181,7 @@
#
# .. note:: This type is deprecated in favor of SocketAddress. The
#difference between SocketAddressLegacy and SocketAddress is that
-#the latter has fewer {} on the wire.
+#the latter has fewer ``{}`` on the wire.
#
# Since: 1.3
##
diff --git a/qapi/ui.json b/qapi/ui.json
index a165e44..5bcccbfc930 100644
--- a/qapi/ui.json
+++ b/qapi/ui.json
@@ -1268,7 +1268,7 @@
# Since: 2.6
#
# .. note:: The consoles are visible in the qom tree, under
-#/backend/console[$index]. They have a device link and head
+#``/backend/console[$index]``. They have a device link and head
#property, so it is possible to map which console belongs to which
#device and display.
#
--
2.45.0
[PATCH v2 19/21] qapi: convert "Example" sections with titles
When an Example section has a brief explanation, convert it to a
qmp-example:: section using the :title: option.
Rule of thumb: If the title can fit on a single line and requires no rST
markup, it's a good candidate for using the :title: option of
qmp-example.
In this patch, trailing punctuation is removed from the title section
for consistent headline aesthetics. In just one case, specifics of the
example are removed to make the title read better.
See commit-4: "docs/qapidoc: create qmp-example directive", for a
detailed explanation of this custom directive syntax.
See commit+2: "qapi: remove "Example" doc section" for a detailed
explanation of why.
Signed-off-by: John Snow
---
qapi/block-core.json | 24
qapi/block.json | 13 ++---
qapi/migration.json | 25 ++---
qapi/qom.json| 8
qapi/ui.json | 11 ++-
qapi/virtio.json | 19 ++-
6 files changed, 52 insertions(+), 48 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 4e0f0395146..a371e3464d2 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -5885,9 +5885,8 @@
#
# Since: 2.7
#
-# Examples:
-#
-# 1. Add a new node to a quorum
+# .. qmp-example::
+#:title: Add a new node to a quorum
#
# -> { "execute": "blockdev-add",
# "arguments": {
@@ -5901,7 +5900,8 @@
# "node": "new_node" } }
# <- { "return": {} }
#
-# 2. Delete a quorum's node
+# .. qmp-example::
+#:title: Delete a quorum's node
#
# -> { "execute": "x-blockdev-change",
# "arguments": { "parent": "disk1",
@@ -5937,16 +5937,16 @@
#
# Since: 2.12
#
-# Examples:
-#
-# 1. Move a node into an IOThread
+# .. qmp-example::
+#:title: Move a node into an IOThread
#
# -> { "execute": "x-blockdev-set-iothread",
# "arguments": { "node-name": "disk1",
# "iothread": "iothread0" } }
# <- { "return": {} }
#
-# 2. Move a node into the main loop
+# .. qmp-example::
+#:title: Move a node into the main loop
#
# -> { "execute": "x-blockdev-set-iothread",
# "arguments": { "node-name": "disk1",
@@ -6022,16 +6022,16 @@
#
# Since: 2.0
#
-# Examples:
-#
-# 1. Read operation
+# .. qmp-example::
+#:title: Read operation
#
# <- { "event": "QUORUM_REPORT_BAD",
# "data": { "node-name": "node0", "sector-num": 345435,
"sectors-count": 5,
#"type": "read" },
# "timestamp": { "seconds": 1344522075, "microseconds": 745528 } }
#
-# 2. Flush operation
+# .. qmp-example::
+#:title: Flush operation
#
# <- { "event": "QUORUM_REPORT_BAD",
# "data": { "node-name": "node0", "sector-num": 0, "sectors-count":
2097120,
diff --git a/qapi/block.json b/qapi/block.json
index c8e52bc2d29..5ddd061e964 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -342,9 +342,8 @@
#
# Since: 2.5
#
-# Examples:
-#
-# 1. Change a removable medium
+# .. qmp-example::
+#:title: Change a removable medium
#
# -> { "execute": "blockdev-change-medium",
# "arguments": { "id": "ide0-1-0",
@@ -352,7 +351,8 @@
# "format": "raw" } }
# <- { "return": {} }
#
-# 2. Load a read-only medium into a writable drive
+# .. qmp-example::
+#:title: Load a read-only medium into a writable drive
#
# -> { "execute": "blockdev-change-medium",
# "arguments": { "id": "floppyA",
@@ -577,9 +577,8 @@
# "boundaries-write": [1000, 5000] } }
# <- { "return": {} }
#
-# Example:
-#
-# Remove all latency histograms:
+# .. qmp-example::
+#:title: Remove all latency histograms
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0" } }
diff --git a/qapi/migration.json b/qapi/migration.json
index a4391ea7e6f..37ce8afa380 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -287,14 +287,14 @@
#
# Since: 0.14
#
-# Examples:
-#
-# 1. Before the first migration
+# .. qmp-example::
+#:title: Before the first migration
#
# -> { "execute": "query-migrate" }
# <- { &quo
[PATCH v2 04/21] docs/qapidoc: delint a tiny portion of the module
In a forthcoming series that adds a new QMP documentation generator, it
will be helpful to have a linting baseline. However, there's no need to
shuffle around the deck chairs too much, because most of this code will
be removed once that new qapidoc generator (the "transmogrifier") is in
place.
To ease my pain: just turn off the black auto-formatter for most, but
not all, of qapidoc.py. This will help ensure that *new* code follows a
coding standard without bothering too much with cleaning up the existing
code.
Code that I intend to keep is still subject to the delinting beam.
Signed-off-by: John Snow
Reviewed-by: Markus Armbruster
---
docs/sphinx/qapidoc.py | 62 +-
1 file changed, 37 insertions(+), 25 deletions(-)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index 3c0565d0ceb..659e507353a 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -28,26 +28,33 @@
import re
from docutils import nodes
+from docutils.parsers.rst import Directive, directives
from docutils.statemachine import ViewList
-from docutils.parsers.rst import directives, Directive
-from sphinx.errors import ExtensionError
-from sphinx.util.nodes import nested_parse_with_titles
-import sphinx
-from qapi.gen import QAPISchemaVisitor
from qapi.error import QAPIError, QAPISemError
+from qapi.gen import QAPISchemaVisitor
from qapi.schema import QAPISchema
+import sphinx
+from sphinx.errors import ExtensionError
+from sphinx.util.nodes import nested_parse_with_titles
+
# Sphinx up to 1.6 uses AutodocReporter; 1.7 and later
# use switch_source_input. Check borrowed from kerneldoc.py.
-Use_SSI = sphinx.__version__[:3] >= '1.7'
-if Use_SSI:
+USE_SSI = sphinx.__version__[:3] >= "1.7"
+if USE_SSI:
from sphinx.util.docutils import switch_source_input
else:
-from sphinx.ext.autodoc import AutodocReporter
+from sphinx.ext.autodoc import ( # pylint: disable=no-name-in-module
+AutodocReporter,
+)
-__version__ = '1.0'
+__version__ = "1.0"
+
+
+# Disable black auto-formatter until re-enabled:
+# fmt: off
class QAPISchemaGenRSTVisitor(QAPISchemaVisitor):
@@ -441,6 +448,10 @@ def get_document_nodes(self):
return self._top_node.children
+# Turn the black formatter on for the rest of the file.
+# fmt: on
+
+
class QAPISchemaGenDepVisitor(QAPISchemaVisitor):
"""A QAPI schema visitor which adds Sphinx dependencies each module
@@ -448,34 +459,34 @@ class QAPISchemaGenDepVisitor(QAPISchemaVisitor):
that the generated documentation output depends on the input
schema file associated with each module in the QAPI input.
"""
+
def __init__(self, env, qapidir):
self._env = env
self._qapidir = qapidir
def visit_module(self, name):
if name != "./builtin":
-qapifile = self._qapidir + '/' + name
+qapifile = self._qapidir + "/" + name
self._env.note_dependency(os.path.abspath(qapifile))
super().visit_module(name)
class QAPIDocDirective(Directive):
"""Extract documentation from the specified QAPI .json file"""
+
required_argument = 1
optional_arguments = 1
-option_spec = {
-'qapifile': directives.unchanged_required
-}
+option_spec = {"qapifile": directives.unchanged_required}
has_content = False
def new_serialno(self):
"""Return a unique new ID string suitable for use as a node's ID"""
env = self.state.document.settings.env
-return 'qapidoc-%d' % env.new_serialno('qapidoc')
+return "qapidoc-%d" % env.new_serialno("qapidoc")
def run(self):
env = self.state.document.settings.env
-qapifile = env.config.qapidoc_srctree + '/' + self.arguments[0]
+qapifile = env.config.qapidoc_srctree + "/" + self.arguments[0]
qapidir = os.path.dirname(qapifile)
try:
@@ -513,13 +524,14 @@ def do_parse(self, rstlist, node):
# plain self.state.nested_parse(), and so we can drop the saving
# of title_styles and section_level that kerneldoc.py does,
# because nested_parse_with_titles() does that for us.
-if Use_SSI:
+if USE_SSI:
with switch_source_input(self.state, rstlist):
nested_parse_with_titles(self.state, rstlist, node)
else:
save = self.state.memo.reporter
self.state.memo.reporter = AutodocReporter(
-rstlist, self.state.memo.reporter)
+rstlist, self.state.memo.reporter
+)
try:
nested_parse_with_titles(self.state, rstlist, node)
finally:
@@ -527,12 +539,12 @@
[PATCH v2 11/21] qapi: update prose in note blocks
Where I've noticed, rephrase the note to read more fluently. Signed-off-by: John Snow Reviewed-by: Markus Armbruster --- qapi/block-core.json | 4 ++-- qga/qapi-schema.json | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index cacedfb771c..9ef23ec02ae 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -6048,9 +6048,9 @@ # # @name: the name of the internal snapshot to be created # -# .. note:: In transaction, if @name is empty, or any snapshot matching +# .. note:: In a transaction, if @name is empty or any snapshot matching #@name exists, the operation will fail. Only some image formats -#support it, for example, qcow2, and rbd. +#support it; for example, qcow2, and rbd. # # Since: 1.7 ## diff --git a/qga/qapi-schema.json b/qga/qapi-schema.json index 57598331c5c..1273d85bb5f 100644 --- a/qga/qapi-schema.json +++ b/qga/qapi-schema.json @@ -480,7 +480,7 @@ # # Returns: Number of file systems thawed by this call # -# .. note:: If return value does not match the previous call to +# .. note:: If the return value does not match the previous call to #guest-fsfreeze-freeze, this likely means some freezable filesystems #were unfrozen before this call, and that the filesystem state may #have changed before issuing this command. -- 2.45.0
[PATCH v2 06/21] qapi/parser: fix comment parsing immediately following a doc block
If a comment immediately follows a doc block, the parser doesn't ignore
that token appropriately. Fix that.
e.g.
> ##
> # = Hello World!
> ##
>
> # I'm a comment!
will break the parser, because it does not properly ignore the comment
token if it immediately follows a doc block.
Fixes: 3d035cd2cca6 (qapi: Rewrite doc comment parser)
Signed-off-by: John Snow
Reviewed-by: Markus Armbruster
---
scripts/qapi/parser.py | 2 +-
tests/qapi-schema/doc-good.json | 2 ++
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index 1ef1f85b028..c3d20cc01bc 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -583,7 +583,7 @@ def get_doc(self) -> 'QAPIDoc':
line = self.get_doc_line()
first = False
-self.accept(False)
+self.accept()
doc.end()
return doc
diff --git a/tests/qapi-schema/doc-good.json b/tests/qapi-schema/doc-good.json
index de38a386e8f..8b39eb946af 100644
--- a/tests/qapi-schema/doc-good.json
+++ b/tests/qapi-schema/doc-good.json
@@ -55,6 +55,8 @@
# - {braces}
##
+# Not a doc comment
+
##
# @Enum:
#
--
2.45.0
[PATCH v2 08/21] qapi: fix non-compliant JSON examples
The new QMP documentation generator wants to parse all examples as
"QMP". We have an existing QMP lexer in docs/sphinx/qmp_lexer.py (Seen
in-use here: https://qemu-project.gitlab.io/qemu/interop/bitmaps.html)
that allows the use of "->", "<-" and "..." tokens to denote QMP
protocol flow with elisions, but otherwise defers to the JSON lexer.
To utilize this lexer for the existing QAPI documentation, we need them
to conform to a standard so that they lex and render correctly. Once the
QMP lexer is active for examples, errant QMP/JSON will produce warning
messages and fail the build.
Fix any invalid JSON found in QAPI documentation (identified by
attempting to lex all examples as QMP; see subsequent
commits). Additionally, elisions must be standardized for the QMP lexer;
they must be represented as the value "...", so three examples have been
adjusted to support that format here.
Signed-off-by: John Snow
Reviewed-by: Markus Armbruster
---
qapi/control.json | 3 ++-
qapi/machine.json | 2 +-
qapi/migration.json | 2 +-
qapi/misc.json | 3 ++-
qapi/net.json | 6 +++---
qapi/rocker.json| 2 +-
qapi/ui.json| 2 +-
7 files changed, 11 insertions(+), 9 deletions(-)
diff --git a/qapi/control.json b/qapi/control.json
index 6bdbf077c2e..10c906fa0e7 100644
--- a/qapi/control.json
+++ b/qapi/control.json
@@ -145,7 +145,8 @@
# },
# {
#"name":"system_powerdown"
-# }
+# },
+# ...
# ]
#}
#
diff --git a/qapi/machine.json b/qapi/machine.json
index 2fd3e9c3d5d..a982c945035 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -1057,7 +1057,7 @@
#"vcpus-count": 1 },
# { "props": { "core-id": 0 }, "type": "POWER8-spapr-cpu-core",
#"vcpus-count": 1, "qom-path": "/machine/unattached/device[0]"}
-#]}'
+#]}
#
# For pc machine type started with -smp 1,maxcpus=2:
#
diff --git a/qapi/migration.json b/qapi/migration.json
index 0f24206bce4..9ec9ef36c47 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -2106,7 +2106,7 @@
# Example:
#
# -> {"execute": "calc-dirty-rate", "arguments": {"calc-time": 1,
-# 'sample-pages': 512} }
+# "sample-pages": 512} }
# <- { "return": {} }
#
# Measure dirty rate using dirty bitmap for 500 milliseconds:
diff --git a/qapi/misc.json b/qapi/misc.json
index ec30e5c570a..4b41e15dcd4 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -287,7 +287,8 @@
#
# Example:
#
-# -> { "execute": "get-win32-socket", "arguments": { "info": "abcd123..",
fdname": "skclient" } }
+# -> { "execute": "get-win32-socket",
+# "arguments": { "info": "abcd123..", "fdname": "skclient" } }
# <- { "return": {} }
##
{ 'command': 'get-win32-socket', 'data': {'info': 'str', 'fdname': 'str'},
'if': 'CONFIG_WIN32' }
diff --git a/qapi/net.json b/qapi/net.json
index 0f5a259475e..c19df435a53 100644
--- a/qapi/net.json
+++ b/qapi/net.json
@@ -1003,9 +1003,9 @@
#
# Example:
#
-# <- { 'event': 'NETDEV_STREAM_DISCONNECTED',
-# 'data': {'netdev-id': 'netdev0'},
-# 'timestamp': {'seconds': 1663330937, 'microseconds': 526695} }
+# <- { "event": "NETDEV_STREAM_DISCONNECTED",
+# "data": {"netdev-id": "netdev0"},
+# "timestamp": {"seconds": 1663330937, "microseconds": 526695} }
##
{ 'event': 'NETDEV_STREAM_DISCONNECTED',
'data': { 'netdev-id': 'str' } }
diff --git a/qapi/rocker.json b/qapi/rocker.json
index 5635cf174fd..f5225eb62cc 100644
--- a/qapi/rocker.json
+++ b/qapi/rocker.json
@@ -250,7 +250,7 @@
# "action": {"goto-tbl": 10},
# "mask": {"in-pport": 4294901760}
# },
-# {...more...},
+# {...},
#]}
##
{ 'command': 'query-rocker-of-dpa-flows',
diff --git a/qapi/ui.json b/qapi/ui.json
index f610bce118a..c12f5292571 100644
--- a/qapi/ui.json
+++ b/qapi/ui.json
@@ -361,7 +361,7 @@
#"channel-id": 0,
#"tls": false
# },
-# [ ... more channels follow ... ]
+# ...
# ]
# }
#}
--
2.45.0
[PATCH v2 09/21] qapi: nail down convention that Errors sections are lists
By unstated convention, Errors sections are rST lists. Document the convention, and make the one exception conform. Signed-off-by: John Snow Reviewed-by: Markus Armbruster --- docs/devel/qapi-code-gen.rst | 7 +++ qapi/transaction.json| 2 +- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst index f453bd35465..cee43222f19 100644 --- a/docs/devel/qapi-code-gen.rst +++ b/docs/devel/qapi-code-gen.rst @@ -1011,6 +1011,13 @@ like this:: "Returns" and "Errors" sections are only valid for commands. They document the success and the error response, respectively. +"Errors" sections should be formatted as an rST list, each entry +detailing a relevant error condition. For example:: + + # Errors: + # - If @device does not exist, DeviceNotFound + # - Any other error returns a GenericError. + A "Since: x.y.z" tagged section lists the release that introduced the definition. diff --git a/qapi/transaction.json b/qapi/transaction.json index 5749c133d4a..07afc269d54 100644 --- a/qapi/transaction.json +++ b/qapi/transaction.json @@ -235,7 +235,7 @@ # additional detail. # # Errors: -# Any errors from commands in the transaction +# - Any errors from commands in the transaction # # Note: The transaction aborts on the first failure. Therefore, there # will be information on only one failed operation returned in an -- 2.45.0
[PATCH v2 05/21] qapi/parser: preserve indentation in QAPIDoc sections
Change get_doc_indented() to preserve indentation on all subsequent text
lines, and create a compatibility dedent() function for qapidoc.py that
removes indentation the same way get_doc_indented() did.
This is being done for the benefit of a new qapidoc generator which
requires that indentation in argument and features sections are
preserved.
Prior to this patch, a section like this:
```
@name: lorem ipsum
dolor sit amet
consectetur adipiscing elit
```
would have its body text be parsed into:
```
lorem ipsum
dolor sit amet
consectetur adipiscing elit
```
We want to preserve the indentation for even the first body line so that
the entire block can be parsed directly as rST. This patch would now
parse that segment into:
```
lorem ipsum
dolor sit amet
consectetur adipiscing elit
```
This is helpful for formatting arguments and features as field lists in
rST, where the new generator will format this information as:
```
:arg type name: lorem ipsum
dolor sit amet
consectetur apidiscing elit
```
...and can be formed by the simple concatenation of the field list
construct and the body text. The indents help preserve the continuation
of a block-level element, and further allow the use of additional rST
block-level constructs such as code blocks, lists, and other such
markup.
This understandably breaks the existing qapidoc.py; so a new function is
added there to dedent the text for compatibility. Once the new generator
is merged, this function will not be needed any longer and can be
dropped.
Signed-off-by: John Snow
[Edited commit message and code comments per review --js]
Reviewed-by: Markus Armbruster
---
docs/sphinx/qapidoc.py | 27 ++-
scripts/qapi/parser.py | 4 ++--
tests/qapi-schema/doc-good.out | 32
3 files changed, 40 insertions(+), 23 deletions(-)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index 659e507353a..f9683444b14 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -26,6 +26,7 @@
import os
import re
+import textwrap
from docutils import nodes
from docutils.parsers.rst import Directive, directives
@@ -53,6 +54,19 @@
__version__ = "1.0"
+def dedent(text: str) -> str:
+# Adjust indentation to make description text parse as paragraph.
+
+lines = text.splitlines(True)
+if re.match(r"\s+", lines[0]):
+# First line is indented; description started on the line after
+# the name. dedent the whole block.
+return textwrap.dedent(text)
+
+# Descr started on same line. Dedent line 2+.
+return lines[0] + textwrap.dedent("".join(lines[1:]))
+
+
# Disable black auto-formatter until re-enabled:
# fmt: off
@@ -164,7 +178,7 @@ def _nodes_for_members(self, doc, what, base=None,
branches=None):
term = self._nodes_for_one_member(section.member)
# TODO drop fallbacks when undocumented members are outlawed
if section.text:
-defn = section.text
+defn = dedent(section.text)
else:
defn = [nodes.Text('Not documented')]
@@ -202,7 +216,7 @@ def _nodes_for_enum_values(self, doc):
termtext.extend(self._nodes_for_ifcond(section.member.ifcond))
# TODO drop fallbacks when undocumented members are outlawed
if section.text:
-defn = section.text
+defn = dedent(section.text)
else:
defn = [nodes.Text('Not documented')]
@@ -237,7 +251,7 @@ def _nodes_for_features(self, doc):
dlnode = nodes.definition_list()
for section in doc.features.values():
dlnode += self._make_dlitem(
-[nodes.literal('', section.member.name)], section.text)
+[nodes.literal('', section.member.name)], dedent(section.text))
seen_item = True
if not seen_item:
@@ -260,9 +274,12 @@ def _nodes_for_sections(self, doc):
continue
snode = self._make_section(section.tag)
if section.tag and section.tag.startswith('Example'):
-snode += self._nodes_for_example(section.text)
+snode += self._nodes_for_example(dedent(section.text))
else:
-self._parse_text_into_node(section.text, snode)
+self._parse_text_into_node(
+dedent(section.text) if section.tag else section.text,
+snode,
+)
nodelist.append(snode)
return nodelist
diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index 7b13a583ac1..1ef1f85b028 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -448,7 +448,7 @@ def get_doc_indented(self, doc: 'QAPIDoc') -> Optional[str]:
indent = must_match(r'\s
[PATCH v2 00/21] qapi: convert "Note" and "Example" sections to rST
This series focuses primarily on converting our existing QAPI/QMP
documentation to remove special "Note" and "Example" sections in favor
of rST markup for the same.
This is being done primarily to reduce the number of specially parsed
QAPI sections we have in favor of allowing fully arbitrary rST markup
for greater flexibility and freedom in styling the rendered HTML
documentation.
(A tangible side benefit is that the new qapidoc generator has fewer
sections to reason about when it splices inherited documentation
together for the QMP reference manual; docs largely preserve the order
of documentation "as written" instead of needing to splice multiple
separate sections together. A goal of the new generator is to eventually
remove all tagged sections except for "members" and "features".)
V2:
- Lotsa stuff :{
- Added new example box styling and sphinx directive macro
- Addressed... most? list feedback. (But not the indent split.)
Known issues:
- The Example boxes are styled a little wonkily if prose follows the
code-block instead of precedes it, fixup will be tomorrow.
--js
Harmonie Snow (1):
docs/sphinx: add CSS styling for qmp-example directive
John Snow (20):
[DO-NOT-MERGE]: Add some ad-hoc linting helpers.
qapi: linter fixups
docs/qapidoc: remove unused intersperse function
docs/qapidoc: delint a tiny portion of the module
qapi/parser: preserve indentation in QAPIDoc sections
qapi/parser: fix comment parsing immediately following a doc block
docs/qapidoc: fix nested parsing under untagged sections
qapi: fix non-compliant JSON examples
qapi: nail down convention that Errors sections are lists
qapi: convert "Note" sections to plain rST
qapi: update prose in note blocks
qapi: add markup to note blocks
qapi/parser: don't parse rST markup as section headers
docs/qapidoc: factor out do_parse()
docs/qapidoc: create qmp-example directive
docs/qapidoc: add QMP highlighting to annotated qmp-example blocks
qapi: convert "Example" sections without titles
qapi: convert "Example" sections with titles
qapi: convert "Example" sections with longer prose
qapi: remove "Example" doc section
docs/devel/qapi-code-gen.rst | 25 +-
docs/sphinx-static/theme_overrides.css| 46
docs/sphinx/qapidoc.py| 258 +-
qapi/acpi.json| 4 +-
qapi/block-core.json | 116
qapi/block.json | 59 ++--
qapi/char.json| 36 ++-
qapi/control.json | 28 +-
qapi/dump.json| 10 +-
qapi/introspect.json | 6 +-
qapi/machine-target.json | 28 +-
qapi/machine.json | 117
qapi/migration.json | 104 +++
qapi/misc-target.json | 22 +-
qapi/misc.json| 123 +
qapi/net.json | 32 ++-
qapi/pci.json | 10 +-
qapi/qdev.json| 40 +--
qapi/qom.json | 33 ++-
qapi/replay.json | 8 +-
qapi/rocker.json | 26 +-
qapi/run-state.json | 48 ++--
qapi/sockets.json | 10 +-
qapi/stats.json | 22 +-
qapi/tpm.json | 6 +-
qapi/trace.json | 4 +-
qapi/transaction.json | 12 +-
qapi/ui.json | 76 +++---
qapi/vfio.json| 2 +-
qapi/virtio.json | 57 ++--
qapi/yank.json| 4 +-
qga/qapi-schema.json | 48 ++--
scripts/qapi-lint.sh | 61 +
scripts/qapi/Makefile | 5 +
scripts/qapi/introspect.py| 8 +-
scripts/qapi/parser.py| 38 ++-
scripts/qapi/schema.py| 6 +-
scripts/qapi/visit.py | 5 +-
tests/qapi-schema/doc-empty-section.err | 2 +-
tests/qapi-schema/doc-empty-section.json | 2 +-
tests/qapi-schema/doc-good.json | 28 +-
tests/qapi-schema/doc-good.out| 61 +++--
tests/qapi-schema/doc-good.txt| 36 ++-
.../qapi-schema/doc-interleaved-section.json | 2 +-
44 files changed, 1010 insertions(+), 664 deletions(-)
create mode 100755 scripts/qapi-lint.sh
create mode 100644 scripts/qapi/Makefile
--
2.45.0
[PATCH v2 03/21] docs/qapidoc: remove unused intersperse function
This function has been unused since fd62bff901b. Signed-off-by: John Snow --- docs/sphinx/qapidoc.py | 10 -- 1 file changed, 10 deletions(-) diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py index f270b494f01..3c0565d0ceb 100644 --- a/docs/sphinx/qapidoc.py +++ b/docs/sphinx/qapidoc.py @@ -50,16 +50,6 @@ __version__ = '1.0' -# Function borrowed from pydash, which is under the MIT license -def intersperse(iterable, separator): -"""Yield the members of *iterable* interspersed with *separator*.""" -iterable = iter(iterable) -yield next(iterable) -for item in iterable: -yield separator -yield item - - class QAPISchemaGenRSTVisitor(QAPISchemaVisitor): """A QAPI schema visitor which generates docutils/Sphinx nodes -- 2.45.0
Re: [PATCH 13/13] qapi: convert "Example" sections to rST
On Wed, Jun 26, 2024, 1:18 AM Markus Armbruster wrote:
> John Snow writes:
>
> > Eliminate the "Example" sections in QAPI doc blocks, converting them
> > into QMP example code blocks. This is generally done in this patch by
> > converting "Example:" or "Examples:" lines into ".. code-block:: QMP"
> > lines.
>
> [...]
>
> > diff --git a/qapi/migration.json b/qapi/migration.json
> > index 85a14bb4308..849358b6387 100644
> > --- a/qapi/migration.json
> > +++ b/qapi/migration.json
>
> [...]
>
> > @@ -336,7 +338,35 @@
> > # }
> > #}
> > #
> > -# 5. Migration is being performed and XBZRLE is active:
> > +# .. code-block:: QMP
> > +#:caption: Example: Migration is being performed and XBZRLE is
> active
> > +#
> > +# -> { "execute": "query-migrate" }
> > +# <- {
> > +# "return":{
> > +# "status":"active",
> > +# "total-time":12345,
> > +# "setup-time":12345,
> > +# "expected-downtime":12345,
> > +# "ram":{
> > +# "total":1057024,
> > +# "remaining":1053304,
> > +# "transferred":3720,
> > +# "duplicate":123,
> > +# "normal":123,
> > +# "normal-bytes":123456,
> > +# "dirty-sync-count":15
> > +# },
> > +# "disk":{
> > +# "total":20971520,
> > +# "remaining":20880384,
> > +# "transferred":91136
> > +# }
> > +# }
> > +#}
> > +#
> > +# .. code-block:: QMP
> > +#:caption: Example: Migration is being performed and XBZRLE is
> active
> > #
> > # -> { "execute": "query-migrate" }
> > # <- {
>
> Example accidentally duplicated.
>
Fixed this yesterday, oopsie. I think this was a rebase goof.
>
> [...]
>
> > diff --git a/tests/qapi-schema/doc-good.json
> b/tests/qapi-schema/doc-good.json
> > index 4b338cc0186..2774a7ce14d 100644
> > --- a/tests/qapi-schema/doc-good.json
> > +++ b/tests/qapi-schema/doc-good.json
> > @@ -46,11 +46,13 @@
> > #
> > # Duis aute irure dolor
> > #
> > -# Example:
> > +# .. code-block:: QMP
> > +#:caption: Example:
>
> See [*] below.
>
> > #
> > # -> in
> > # <- out
> > -# Examples:
> > +# .. code-block::
> > +#
>
> Likewise.
>
> > # - *verbatim*
> > # - {braces}
> > ##
> > @@ -172,12 +174,13 @@
> > #
> > # Duis aute irure dolor
> > #
> > -# Example:
> > +# .. code-block::
> > #
> > # -> in
> > # <- out
> > #
> > -# Examples:
> > +# .. code-block::
> > +#
> > # - *verbatim*
> > # - {braces}
> > #
> > @@ -196,7 +199,7 @@
> > # @cmd-feat1: a feature
> > # @cmd-feat2: another feature
> > #
> > -# Example:
> > +# .. code-block::
> > #
> > # -> in
> > #
> > diff --git a/tests/qapi-schema/doc-good.out
> b/tests/qapi-schema/doc-good.out
> > index 2c9b4e419cb..347b9cb7134 100644
> > --- a/tests/qapi-schema/doc-good.out
> > +++ b/tests/qapi-schema/doc-good.out
> > @@ -93,11 +93,13 @@ Notes:
> >
> > Duis aute irure dolor
> >
> > -Example:
> > +.. code-block:: QMP
> > + :caption: Example:
>
> [*] This demonstrates the "Example: ..." is *not* recognized as a
> "Example" section before the patch (compare to the change we get for
> recognized sections below).
>
> I pointed out the same issue for "Note" in review of PATCH 9, and
> suggested ways to resolve it. Pick a way there, and use it here as well
>
ACK
> >
> > -> in
> > <- out
> > -Examples:
> > +.. code-block::
> > +
> > - *verbatim*
> > - {braces}
> > doc symbol=Enum
> > @@ -184,10 +186,14 @@ frobnicate
> > - Ut enim ad minim veniam
> >
> > Duis aute irure dolor
> > -section=Example
> > +
> > +.. code-block::
> > +
> > -> in
> > <- out
> > -section=Examples
> > +
> > +.. code-block::
> >
Re: [PATCH 05/20] qapi/parser: adjust info location for doc body section
On Mon, May 27, 2024 at 7:58 AM Markus Armbruster wrote:
> John Snow writes:
>
> > On Thu, May 16, 2024, 1:58 AM Markus Armbruster
> wrote:
> >
> >> John Snow writes:
> >>
> >> > Instead of using the info object for the doc block as a whole, update
> >> > the info pointer for each call to ensure_untagged_section when the
> >> > existing section is otherwise empty. This way, Sphinx error
> information
> >> > will match precisely to where the text actually starts.
> >> >
> >> > Signed-off-by: John Snow
> >> > ---
> >> > scripts/qapi/parser.py | 9 +++--
> >> > 1 file changed, 7 insertions(+), 2 deletions(-)
> >> >
> >> > diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
> >> > index 8cdd5334ec6..41b9319e5cb 100644
> >> > --- a/scripts/qapi/parser.py
> >> > +++ b/scripts/qapi/parser.py
> >> > @@ -662,8 +662,13 @@ def end(self) -> None:
> >> >
> >> > def ensure_untagged_section(self, info: QAPISourceInfo) -> None:
> >> > if self.all_sections and not self.all_sections[-1].tag:
> >> > -# extend current section
> >> > -self.all_sections[-1].text += '\n'
> >>
> >> Before, we always append a newline.
> >>
> >> > +section = self.all_sections[-1]
> >> > +# Section is empty so far; update info to start *here*.
> >> > +if not section.text:
> >> > +section.info = info
> >> > +else:
> >> > +# extend current section
> >> > +self.all_sections[-1].text += '\n'
> >>
> >> Afterwards, we append it only when the section already has some text.
> >>
> >> The commit message claims the patch only adjusts section.info. That's
> a
> >> lie :)
> >>
> >
> > Well. It wasn't intentional, so it wasn't a lie... it was just wrong :)
> >
> >
> >> I believe the change makes no difference because .end() strips leading
> >> and trailing newline.
> >>
> >> > return
> >> > # start new section
> >> > section = self.Section(info)
> >>
> >> You could fix the commit message, but I think backing out the
> >> no-difference change is easier. The appended patch works in my testing.
> >>
> >> Next one. Your patch changes the meaning of section.info. Here's its
> >> initialization:
> >>
> >> class Section:
> >> # pylint: disable=too-few-public-methods
> >> def __init__(self, info: QAPISourceInfo,
> >> tag: Optional[str] = None):
> >> ---># section source info, i.e. where it begins
> >> self.info = info
> >> # section tag, if any ('Returns', '@name', ...)
> >> self.tag = tag
> >> # section text without tag
> >> self.text = ''
> >>
> >> The comment is now wrong. Calls for a thorough review of .info's uses.
> >>
> >
> > Hmm... Did I really change its meaning? I guess it's debatable what
> "where
> > it begins" means. Does the tagless section start...
> >
> > ## <-- Here?
> > # Hello! <-- Or here?
> > ##
> >
> > I assert the *section* starts wherever the first line of text it contains
> > starts. Nothing else makes any sense.
> >
> > There is value in recording where the doc block starts, but that's not a
> > task for the *section* info.
> >
> > I don't think I understand your feedback.
>
> This was before my vacation, and my memory is foggy, ... I may have
> gotten confused back then. Let me have a fresh look now.
>
> self.info gets initialized in Section.__init__() to whatever info it
> gets passed.
>
> Your patch makes .ensure_untagged_section() overwrite this Section.info
> when it extends an untagged section that is still empty. Hmmm. I'd
> prefer .info to remain constant after initialization.
>
but, we don't have the right info when we initialize the entire QAPIDoc
object, because the section hasn't truly actually started yet, so I don't
think I can actually achieve your preference.
>
> I figure this overwrite can happen only when extenting the body section
> Q
Re: [PATCH 09/13] qapi: convert "Note" sections to plain rST
On Fri, Jun 21, 2024 at 8:23 AM Markus Armbruster wrote: > John Snow writes: > > > On Thu, Jun 20, 2024 at 11:46 AM John Snow wrote: > > > >> > >> > >> On Thu, Jun 20, 2024, 9:35 AM Markus Armbruster > wrote: > >> > >>> Markus Armbruster writes: > >>> > >>> > John Snow writes: > >>> > >>> [...] > >>> > >>> >> diff --git a/qga/qapi-schema.json b/qga/qapi-schema.json > >>> >> index b3de1fb6b3a..57598331c5c 100644 > >>> >> --- a/qga/qapi-schema.json > >>> >> +++ b/qga/qapi-schema.json > >>> > >>> [...] > >>> > >>> >> @@ -631,8 +632,8 @@ > >>> >> # Errors: > >>> >> # - If hybrid suspend is not supported, Unsupported > >>> >> # > >>> >> -# Notes: It's strongly recommended to issue the guest-sync command > >>> >> -# before sending commands when the guest resumes > >>> >> +# .. note:: It's strongly recommended to issue the guest-sync > command > >>> >> +#before sending commands when the guest resumes. > >>> >> # > >>> >> # Since: 1.1 > >>> >> ## > >>> >> @@ -1461,16 +1462,15 @@ > >>> >> # * POSIX: as defined by os-release(5) > >>> >> # * Windows: contains string "server" or "client" > >>> >> # > >>> >> -# Notes: On POSIX systems the fields @id, @name, @pretty-name, > >>> >> -# @version, @version-id, @variant and @variant-id follow the > >>> >> -# definition specified in os-release(5). Refer to the manual > page > >>> >> -# for exact description of the fields. Their values are taken > >>> >> -# from the os-release file. If the file is not present in the > >>> >> -# system, or the values are not present in the file, the fields > >>> >> -# are not included. > >>> >> +# .. note:: On POSIX systems the fields @id, @name, @pretty-name, > >>> >> +#@version, @version-id, @variant and @variant-id follow the > >>> >> +#definition specified in os-release(5). Refer to the manual > page > >>> for > >>> >> +#exact description of the fields. Their values are taken from > the > >>> >> +#os-release file. If the file is not present in the system, or > >>> the > >>> >> +#values are not present in the file, the fields are not > included. > >>> >> # > >>> >> -# On Windows the values are filled from information gathered > from > >>> >> -# the system. > >>> >> +#On Windows the values are filled from information gathered > from > >>> >> +#the system. > >>> > > >>> > Please don't change the indentation here. I get the same output with > >>> > > >>> > @@ -1461,7 +1462,7 @@ > >>> ># * POSIX: as defined by os-release(5) > >>> ># * Windows: contains string "server" or "client" > >>> ># > >>> > -# Notes: On POSIX systems the fields @id, @name, @pretty-name, > >>> > +# .. note:: On POSIX systems the fields @id, @name, @pretty-name, > >>> ># @version, @version-id, @variant and @variant-id follow the > >>> ># definition specified in os-release(5). Refer to the manual > page > >>> ># for exact description of the fields. Their values are taken > >>> > >>> I'm blind. Actually, you change indentation of subsequent lines from 4 > >>> to 3 *everywhere*. I guess you do that to make subsequent lines line > up > >>> with the directive, here "note". > >>> > >>> Everywhere else, we indent such lines by 4. Hmm. How terrible would > it > >>> be not to mess with the alignment? > >>> > >>> If we want to use 3 for directives, is it worth pointing out in the > >>> commit message? > >>> > >>> [...] > >>> > >> > >> Let me look up some conventions and see what's the most prominent... as > >> well as testing what emacs does by default (or if emacs can be > configured > >> to do what we want with in-t
Re: [PATCH 09/13] qapi: convert "Note" sections to plain rST
On Fri, Jun 21, 2024 at 8:08 AM Markus Armbruster wrote:
> John Snow writes:
>
> > We do not need a dedicated section for notes. By eliminating a specially
> > parsed section, these notes can be treated as normal rST paragraphs in
> > the new QMP reference manual, and can be placed and styled much more
> > flexibly.
> >
> > Convert all existing "Note" and "Notes" sections to pure rST. As part of
> > the conversion, capitalize the first letter of each sentence and add
> > trailing punctuation where appropriate to ensure notes look sensible and
> > consistent in rendered HTML documentation.
> >
> > Update docs/devel/qapi-code-gen.rst to reflect the new paradigm, and ...
> >
> > ... Update the QAPI parser to prohibit "Note" sections while suggesting
> > a new syntax. The exact formatting to use is a matter of taste, but a
> > good candidate is simply:
> >
> > .. note:: lorem ipsum ...
> >
> > ... but there are other choices, too. The Sphinx readthedocs theme
> > offers theming for the following forms (capitalization unimportant); all
> > are adorned with a (!) symbol in the title bar for rendered HTML docs.
> >
> > See
> >
> https://sphinx-rtd-theme.readthedocs.io/en/stable/demo/demo.html#admonitions
> > for examples of each directive/admonition in use.
> >
> > These are rendered in orange:
> >
> > .. Attention:: ...
> > .. Caution:: ...
> > .. WARNING:: ...
> >
> > These are rendered in red:
> >
> > .. DANGER:: ...
> > .. Error:: ...
> >
> > These are rendered in green:
> >
> > .. Hint:: ...
> > .. Important:: ...
> > .. Tip:: ...
> >
> > These are rendered in blue:
> >
> > .. Note:: ...
> > .. admonition:: custom title
> >
> >admonition body text
> >
> > This patch uses ".. note::" almost everywhere, with just two "caution"
> > directives. ".. admonition:: notes" is used in a few places where we had
> > an ordered list of multiple notes that would not make sense as
> > standalone/separate admonitions.
> >
> > Signed-off-by: John Snow
> > Acked-by: Stefan Hajnoczi [for block*.json]
>
> [...]
>
> > diff --git a/qapi/qom.json b/qapi/qom.json
> > index 8bd299265e3..5bfa0ded42c 100644
> > --- a/qapi/qom.json
> > +++ b/qapi/qom.json
> > @@ -195,12 +195,12 @@
> > #
> > # @typename: the type name of an object
> > #
> > -# Note: objects can create properties at runtime, for example to
> > -# describe links between different devices and/or objects. These
> > -# properties are not included in the output of this command.
> > -#
> > # Returns: a list of ObjectPropertyInfo describing object properties
> > #
> > +# .. note:: Objects can create properties at runtime, for example to
> > +#describe links between different devices and/or objects. These
> > +#properties are not included in the output of this command.
> > +#
> > # Since: 2.12
> > ##
>
> You move the note. Commit message doesn't tell why.
>
> > { 'command': 'qom-list-properties',
>
> [...]
>
"v2" of this series now declines to move the note in this patch and instead
moves it in a separate patch that also enforces source order more strictly
so that the move can be explained in detail.
Rendering order diverges from source order briefly as a result; I will
mention that in the commit message instead.
(I don't think it's easy or worth doing to re-order the patches such that
source and render order never diverge; too much engineering for so
temporary a minor issue. Not to mention the source and render order is
already divergent in many places, so I don't think it's a regression so
much as it is a temporary lateralgression?)
Re: [PATCH 04/13] qapi/parser: preserve indentation in QAPIDoc sections
On Fri, Jun 21, 2024 at 2:38 AM Markus Armbruster wrote: > John Snow writes: > > > On Thu, Jun 20, 2024, 11:07 AM Markus Armbruster > wrote: > > > >> John Snow writes: > >> > >> > On Wed, Jun 19, 2024, 8:03 AM Markus Armbruster > wrote: > >> > > >> >> John Snow writes: > >> >> > >> >> > Change get_doc_indented() to preserve indentation on all > subsequent text > >> >> > lines, and create a compatibility dedent() function for qapidoc.py > to > >> >> > remove that indentation. This is being done for the benefit of a > new > >> >> > >> >> Suggest "remove indentation the same way get_doc_indented() did." > >> >> > >> > > >> > Aight. > >> > > >> > > >> >> > qapidoc generator which requires that indentation in argument and > >> >> > features sections are preserved. > >> >> > > >> >> > Prior to this patch, a section like this: > >> >> > > >> >> > ``` > >> >> > @name: lorem ipsum > >> >> >dolor sit amet > >> >> > consectetur adipiscing elit > >> >> > ``` > >> >> > > >> >> > would have its body text be parsed as: > >> >> > >> >> Suggest "parsed into". > >> >> > >> > > >> > Why? (I mean, I'll do it, but I don't see the semantic difference > >> > personally) > >> > > >> > >> "Parse as " vs. "Parse into ". > >> > >> >> > (first and final newline only for presentation) > >> >> > > >> >> > ``` > >> >> > lorem ipsum > >> >> > dolor sit amet > >> >> > consectetur adipiscing elit > >> >> > ``` > >> >> > > >> >> > We want to preserve the indentation for even the first body line > so that > >> >> > the entire block can be parsed directly as rST. This patch would > now > >> >> > parse that segment as: > >> >> > >> >> If you change "parsed as" to "parsed into" above, then do it here, > too. > >> >> > >> >> > > >> >> > ``` > >> >> > lorem ipsum > >> >> >dolor sit amet > >> >> > consectetur adipiscing elit > >> >> > ``` > >> >> > > >> >> > This is helpful for formatting arguments and features as field > lists in > >> >> > rST, where the new generator will format this information as: > >> >> > > >> >> > ``` > >> >> > :arg type name: lorem ipsum > >> >> >dolor sit amet > >> >> > consectetur apidiscing elit > >> >> > ``` > >> >> > > >> >> > ...and can be formed by the simple concatenation of the field list > >> >> > construct and the body text. The indents help preserve the > continuation > >> >> > of a block-level element, and further allow the use of additional > rST > >> >> > block-level constructs such as code blocks, lists, and other such > >> >> > markup. Avoiding reflowing the text conditionally also helps > preserve > >> >> > source line context for better rST error reporting from sphinx > through > >> >> > generated source, too. > >> >> > >> >> What do you mean by "reflowing"? > >> >> > >> > > >> > Poorly phrased, was thinking about emacs too much. I mean munging the > text > >> > post-hoc for the doc generator such that newlines are added or > removed in > >> > the process of re-formatting text to get the proper indentation for > the new > >> > rST form. > >> > > >> > In prototyping, this got messy very quickly and was difficult to > correlate > >> > source line numbers across the transformation. > >> > > >> > It was easier to just not munge the text at all instead of munging it > and > >> > then un-munging it. > >> > > >> > (semantic satiation: munge munge munge munge.) > >> > >> Is this about a possible alternative solution you explored? Keeping > >> .get_doc_indented() as is, and then try to undo its damage? > >> > > > > precisamente. That solution was categorically worse. > > Since .get_doc_indented() removes N spaces of indentation, we'd want to > add back N spaces of indentation. But we can't know N, so I guess we'd > make do with an arbitrary number. Where would reflowing come it? > > I'd like you to express more clearly that you're talking about an > alternative you rejected. Perhaps like this: > > block-level constructs such as code blocks, lists, and other such > markup. > > The alternative would be to somehow undo .get_doc_indented()'s > indentation changes in the new generator. Much messier. > > Feel free to add more detail to the last paragraph. > Eh, I just deleted it. I recall running into troubles but I can't articulate the precise conditions because as you point out, it's a doomed strategy for other reasons - you can't reconstruct the proper indentation. This patch is still the correct way to go, so I don't have to explain my failures at length in the commit message ... I just like giving people clues for *why* I decided to implement things a certain way, because I often find that more instructive than the "how". In this case, the "why" is probably more properly summarized as "it's a total shitshow in that direction, trust me" --js
Re: [PATCH 09/13] qapi: convert "Note" sections to plain rST
On Thu, Jun 20, 2024 at 11:46 AM John Snow wrote: > > > On Thu, Jun 20, 2024, 9:35 AM Markus Armbruster wrote: > >> Markus Armbruster writes: >> >> > John Snow writes: >> >> [...] >> >> >> diff --git a/qga/qapi-schema.json b/qga/qapi-schema.json >> >> index b3de1fb6b3a..57598331c5c 100644 >> >> --- a/qga/qapi-schema.json >> >> +++ b/qga/qapi-schema.json >> >> [...] >> >> >> @@ -631,8 +632,8 @@ >> >> # Errors: >> >> # - If hybrid suspend is not supported, Unsupported >> >> # >> >> -# Notes: It's strongly recommended to issue the guest-sync command >> >> -# before sending commands when the guest resumes >> >> +# .. note:: It's strongly recommended to issue the guest-sync command >> >> +#before sending commands when the guest resumes. >> >> # >> >> # Since: 1.1 >> >> ## >> >> @@ -1461,16 +1462,15 @@ >> >> # * POSIX: as defined by os-release(5) >> >> # * Windows: contains string "server" or "client" >> >> # >> >> -# Notes: On POSIX systems the fields @id, @name, @pretty-name, >> >> -# @version, @version-id, @variant and @variant-id follow the >> >> -# definition specified in os-release(5). Refer to the manual page >> >> -# for exact description of the fields. Their values are taken >> >> -# from the os-release file. If the file is not present in the >> >> -# system, or the values are not present in the file, the fields >> >> -# are not included. >> >> +# .. note:: On POSIX systems the fields @id, @name, @pretty-name, >> >> +#@version, @version-id, @variant and @variant-id follow the >> >> +#definition specified in os-release(5). Refer to the manual page >> for >> >> +#exact description of the fields. Their values are taken from the >> >> +#os-release file. If the file is not present in the system, or >> the >> >> +#values are not present in the file, the fields are not included. >> >> # >> >> -# On Windows the values are filled from information gathered from >> >> -# the system. >> >> +#On Windows the values are filled from information gathered from >> >> +#the system. >> > >> > Please don't change the indentation here. I get the same output with >> > >> > @@ -1461,7 +1462,7 @@ >> ># * POSIX: as defined by os-release(5) >> ># * Windows: contains string "server" or "client" >> ># >> > -# Notes: On POSIX systems the fields @id, @name, @pretty-name, >> > +# .. note:: On POSIX systems the fields @id, @name, @pretty-name, >> ># @version, @version-id, @variant and @variant-id follow the >> ># definition specified in os-release(5). Refer to the manual page >> ># for exact description of the fields. Their values are taken >> >> I'm blind. Actually, you change indentation of subsequent lines from 4 >> to 3 *everywhere*. I guess you do that to make subsequent lines line up >> with the directive, here "note". >> >> Everywhere else, we indent such lines by 4. Hmm. How terrible would it >> be not to mess with the alignment? >> >> If we want to use 3 for directives, is it worth pointing out in the >> commit message? >> >> [...] >> > > Let me look up some conventions and see what's the most prominent... as > well as testing what emacs does by default (or if emacs can be configured > to do what we want with in-tree style config. Warning: I am functionally > inept at emacs lisp. Warning 2x: [neo]vi[m] users, you're entirely on your > own. I'm sorry.) > > I use three myself by force of habit and have some mild reluctance to > respin for that reason, but ... guess we ought to be consistent if we can. > > (No idea where my habit came from. Maybe it is just because it looks nice > to me and no other reason.) > > ((I have no plans, nor desire, to write any kind of checker to enforce > this, though - sorry.)) > Sphinx doc uses three spaces: https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#directives ... but it warns that it's arbitrary; but it seems common to align with the directive. * https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html#footnotes footnotes require "at least 3" spaces * https
Re: [PATCH 09/13] qapi: convert "Note" sections to plain rST
On Thu, Jun 20, 2024, 9:55 AM Markus Armbruster wrote: > John Snow writes: > > > We do not need a dedicated section for notes. By eliminating a specially > > parsed section, these notes can be treated as normal rST paragraphs in > > the new QMP reference manual, and can be placed and styled much more > > flexibly. > > > > Convert all existing "Note" and "Notes" sections to pure rST. As part of > > the conversion, capitalize the first letter of each sentence and add > > trailing punctuation where appropriate to ensure notes look sensible and > > consistent in rendered HTML documentation. > > > > Update docs/devel/qapi-code-gen.rst to reflect the new paradigm, and ... > > > > ... Update the QAPI parser to prohibit "Note" sections while suggesting > > a new syntax. The exact formatting to use is a matter of taste, but a > > good candidate is simply: > > > > .. note:: lorem ipsum ... > > > > ... but there are other choices, too. The Sphinx readthedocs theme > > offers theming for the following forms (capitalization unimportant); all > > are adorned with a (!) symbol in the title bar for rendered HTML docs. > > > > See > > > https://sphinx-rtd-theme.readthedocs.io/en/stable/demo/demo.html#admonitions > > for examples of each directive/admonition in use. > > > > These are rendered in orange: > > > > .. Attention:: ... > > .. Caution:: ... > > .. WARNING:: ... > > > > These are rendered in red: > > > > .. DANGER:: ... > > .. Error:: ... > > > > These are rendered in green: > > > > .. Hint:: ... > > .. Important:: ... > > .. Tip:: ... > > > > These are rendered in blue: > > > > .. Note:: ... > > .. admonition:: custom title > > > >admonition body text > > > > This patch uses ".. note::" almost everywhere, > > Not mentioned, and may or may not be worth mentioning: both "Note:" and > "Notes:" become ".. note::", which renders as "Note". One instance > quoted below. > > No objection to the change; you obviously double-checked it reads okay > that way. > Right, some become "Note" if it feels appropriate, while others (mentioned just below) retained their "Notes" phrasing with a custom admonition if it felt appropriate. I can mention it more explicitly that some (but not all) "Notes" became "Note". (I am beginning to have doubts anyone will ever read and find these detailed commits useful, but you're welcome to tell your manager how much you love my commit messages and that I deserve a raise for such heroic efforts. /s) > >with just two "caution" > > directives. ".. admonition:: notes" is used in a few places where we had > > an ordered list of multiple notes that would not make sense as > > standalone/separate admonitions. > > > > Signed-off-by: John Snow > > Acked-by: Stefan Hajnoczi [for block*.json] > > [...] > > > diff --git a/qapi/block-core.json b/qapi/block-core.json > > index df5e07debd2..cacedfb771c 100644 > > --- a/qapi/block-core.json > > +++ b/qapi/block-core.json > > [...] > > > @@ -6048,9 +6048,9 @@ > > # > > # @name: the name of the internal snapshot to be created > > # > > -# Notes: In transaction, if @name is empty, or any snapshot matching > > -# @name exists, the operation will fail. Only some image formats > > -# support it, for example, qcow2, and rbd. > > +# .. note:: In transaction, if @name is empty, or any snapshot matching > > +#@name exists, the operation will fail. Only some image formats > > +#support it, for example, qcow2, and rbd. > > # > > # Since: 1.7 > > ## > > [...] > >
Re: [PATCH 09/13] qapi: convert "Note" sections to plain rST
On Thu, Jun 20, 2024, 9:35 AM Markus Armbruster wrote: > Markus Armbruster writes: > > > John Snow writes: > > [...] > > >> diff --git a/qga/qapi-schema.json b/qga/qapi-schema.json > >> index b3de1fb6b3a..57598331c5c 100644 > >> --- a/qga/qapi-schema.json > >> +++ b/qga/qapi-schema.json > > [...] > > >> @@ -631,8 +632,8 @@ > >> # Errors: > >> # - If hybrid suspend is not supported, Unsupported > >> # > >> -# Notes: It's strongly recommended to issue the guest-sync command > >> -# before sending commands when the guest resumes > >> +# .. note:: It's strongly recommended to issue the guest-sync command > >> +#before sending commands when the guest resumes. > >> # > >> # Since: 1.1 > >> ## > >> @@ -1461,16 +1462,15 @@ > >> # * POSIX: as defined by os-release(5) > >> # * Windows: contains string "server" or "client" > >> # > >> -# Notes: On POSIX systems the fields @id, @name, @pretty-name, > >> -# @version, @version-id, @variant and @variant-id follow the > >> -# definition specified in os-release(5). Refer to the manual page > >> -# for exact description of the fields. Their values are taken > >> -# from the os-release file. If the file is not present in the > >> -# system, or the values are not present in the file, the fields > >> -# are not included. > >> +# .. note:: On POSIX systems the fields @id, @name, @pretty-name, > >> +#@version, @version-id, @variant and @variant-id follow the > >> +#definition specified in os-release(5). Refer to the manual page > for > >> +#exact description of the fields. Their values are taken from the > >> +#os-release file. If the file is not present in the system, or the > >> +#values are not present in the file, the fields are not included. > >> # > >> -# On Windows the values are filled from information gathered from > >> -# the system. > >> +#On Windows the values are filled from information gathered from > >> +#the system. > > > > Please don't change the indentation here. I get the same output with > > > > @@ -1461,7 +1462,7 @@ > ># * POSIX: as defined by os-release(5) > ># * Windows: contains string "server" or "client" > ># > > -# Notes: On POSIX systems the fields @id, @name, @pretty-name, > > +# .. note:: On POSIX systems the fields @id, @name, @pretty-name, > ># @version, @version-id, @variant and @variant-id follow the > ># definition specified in os-release(5). Refer to the manual page > ># for exact description of the fields. Their values are taken > > I'm blind. Actually, you change indentation of subsequent lines from 4 > to 3 *everywhere*. I guess you do that to make subsequent lines line up > with the directive, here "note". > > Everywhere else, we indent such lines by 4. Hmm. How terrible would it > be not to mess with the alignment? > > If we want to use 3 for directives, is it worth pointing out in the > commit message? > > [...] > Let me look up some conventions and see what's the most prominent... as well as testing what emacs does by default (or if emacs can be configured to do what we want with in-tree style config. Warning: I am functionally inept at emacs lisp. Warning 2x: [neo]vi[m] users, you're entirely on your own. I'm sorry.) I use three myself by force of habit and have some mild reluctance to respin for that reason, but ... guess we ought to be consistent if we can. (No idea where my habit came from. Maybe it is just because it looks nice to me and no other reason.) ((I have no plans, nor desire, to write any kind of checker to enforce this, though - sorry.))
Re: [PATCH 09/13] qapi: convert "Note" sections to plain rST
On Wed, Jun 19, 2024, 9:07 AM Markus Armbruster wrote:
> John Snow writes:
>
> > We do not need a dedicated section for notes. By eliminating a specially
> > parsed section, these notes can be treated as normal rST paragraphs in
> > the new QMP reference manual, and can be placed and styled much more
> > flexibly.
> >
> > Convert all existing "Note" and "Notes" sections to pure rST. As part of
> > the conversion, capitalize the first letter of each sentence and add
> > trailing punctuation where appropriate to ensure notes look sensible and
> > consistent in rendered HTML documentation.
> >
> > Update docs/devel/qapi-code-gen.rst to reflect the new paradigm, and ...
> >
> > ... Update the QAPI parser to prohibit "Note" sections while suggesting
> > a new syntax. The exact formatting to use is a matter of taste, but a
> > good candidate is simply:
> >
> > .. note:: lorem ipsum ...
> >
> > ... but there are other choices, too. The Sphinx readthedocs theme
> > offers theming for the following forms (capitalization unimportant); all
> > are adorned with a (!) symbol in the title bar for rendered HTML docs.
> >
> > See
> >
> https://sphinx-rtd-theme.readthedocs.io/en/stable/demo/demo.html#admonitions
> > for examples of each directive/admonition in use.
> >
> > These are rendered in orange:
> >
> > .. Attention:: ...
> > .. Caution:: ...
> > .. WARNING:: ...
> >
> > These are rendered in red:
> >
> > .. DANGER:: ...
> > .. Error:: ...
> >
> > These are rendered in green:
> >
> > .. Hint:: ...
> > .. Important:: ...
> > .. Tip:: ...
> >
> > These are rendered in blue:
> >
> > .. Note:: ...
> > .. admonition:: custom title
> >
> >admonition body text
> >
> > This patch uses ".. note::" almost everywhere, with just two "caution"
> > directives. ".. admonition:: notes" is used in a few places where we had
> > an ordered list of multiple notes that would not make sense as
> > standalone/separate admonitions.
> >
> > Signed-off-by: John Snow
> > Acked-by: Stefan Hajnoczi [for block*.json]
>
> [...]
>
> > diff --git a/qapi/control.json b/qapi/control.json
> > index 10c906fa0e7..59d5e00c151 100644
> > --- a/qapi/control.json
> > +++ b/qapi/control.json
> > @@ -22,14 +22,13 @@
> > # "arguments": { "enable": [ "oob" ] } }
> > # <- { "return": {} }
> > #
> > -# Notes: This command is valid exactly when first connecting: it must
> > -# be issued before any other command will be accepted, and will
> > -# fail once the monitor is accepting other commands. (see qemu
> > -# docs/interop/qmp-spec.rst)
> > +# .. note:: This command is valid exactly when first connecting: it must
> > +#be issued before any other command will be accepted, and will fail
> > +#once the monitor is accepting other commands. (see qemu
> > +#docs/interop/qmp-spec.rst)
> > #
> > -# The QMP client needs to explicitly enable QMP capabilities,
> > -# otherwise all the QMP capabilities will be turned off by
> > -# default.
> > +# .. note:: The QMP client needs to explicitly enable QMP capabilities,
> > +#otherwise all the QMP capabilities will be turned off by default.
> > #
> > # Since: 0.13
> > ##
> > @@ -150,8 +149,8 @@
> > # ]
> > #}
> > #
> > -# Note: This example has been shortened as the real response is too
> > -# long.
> > +# This example has been shortened as the real response is too long.
> > +#
>
> Here's one way to transform the elision note, ...
>
> > ##
> > { 'command': 'query-commands', 'returns': ['CommandInfo'],
> >'allow-preconfig': true }
>
> [...]
>
> > diff --git a/qapi/pci.json b/qapi/pci.json
> > index 08bf6958634..f51159a2c4c 100644
> > --- a/qapi/pci.json
> > +++ b/qapi/pci.json
> > @@ -146,8 +146,8 @@
> > #
> > # @regions: a list of the PCI I/O regions associated with the device
> > #
> > -# Notes: the contents of @class_info.desc are not stable and should
> > -# only be treated as informational.
> > +# .. note:: The contents of @class_info.desc are not stable and should
> > +#only be treated as informational.
> > #
> > # Since: 0.14
> > ##
> > @@ -311,7 +311,8 @@
> > # ]
> > #}
> > #
> > -# Note: This example has been shortened as the real response is too
> > +# Note: This example has been shortened as the real response is too
> > # long.
> > +#
>
> ... and here's another way. Same way would be better, wouldn't it?
> They actually are after PATCH 13:
>
> -# Note: This example has been shortened as the real response is too
> -# long.
> +# This example has been shortened as the real response is too long.
>
> Move that hunk here, please.
>
> > ##
> > { 'command': 'query-pci', 'returns': ['PciInfo'] }
>
> [...]
>
Apologies, I meant to do this but forgot there were two cases and only
nabbed one.
Fixing.
>
Re: [PATCH 09/13] qapi: convert "Note" sections to plain rST
On Wed, Jun 19, 2024, 8:49 AM Markus Armbruster wrote:
> John Snow writes:
>
> > We do not need a dedicated section for notes. By eliminating a specially
> > parsed section, these notes can be treated as normal rST paragraphs in
> > the new QMP reference manual, and can be placed and styled much more
> > flexibly.
> >
> > Convert all existing "Note" and "Notes" sections to pure rST. As part of
> > the conversion, capitalize the first letter of each sentence and add
> > trailing punctuation where appropriate to ensure notes look sensible and
> > consistent in rendered HTML documentation.
> >
> > Update docs/devel/qapi-code-gen.rst to reflect the new paradigm, and ...
> >
> > ... Update the QAPI parser to prohibit "Note" sections while suggesting
>
> Scratch "... ..." and downcase "Update"?
>
> > a new syntax. The exact formatting to use is a matter of taste, but a
> > good candidate is simply:
> >
> > .. note:: lorem ipsum ...
> >
> > ... but there are other choices, too. The Sphinx readthedocs theme
> > offers theming for the following forms (capitalization unimportant); all
> > are adorned with a (!) symbol in the title bar for rendered HTML docs.
> >
> > See
> >
> https://sphinx-rtd-theme.readthedocs.io/en/stable/demo/demo.html#admonitions
> > for examples of each directive/admonition in use.
> >
> > These are rendered in orange:
> >
> > .. Attention:: ...
> > .. Caution:: ...
> > .. WARNING:: ...
> >
> > These are rendered in red:
> >
> > .. DANGER:: ...
> > .. Error:: ...
> >
> > These are rendered in green:
> >
> > .. Hint:: ...
> > .. Important:: ...
> > .. Tip:: ...
> >
> > These are rendered in blue:
> >
> > .. Note:: ...
> > .. admonition:: custom title
> >
> >admonition body text
> >
> > This patch uses ".. note::" almost everywhere, with just two "caution"
> > directives. ".. admonition:: notes" is used in a few places where we had
> > an ordered list of multiple notes that would not make sense as
> > standalone/separate admonitions.
> >
> > Signed-off-by: John Snow
> > Acked-by: Stefan Hajnoczi [for block*.json]
>
> [...]
>
> > diff --git a/qga/qapi-schema.json b/qga/qapi-schema.json
> > index b3de1fb6b3a..57598331c5c 100644
> > --- a/qga/qapi-schema.json
> > +++ b/qga/qapi-schema.json
> > @@ -422,8 +422,9 @@
> > # Returns: GuestFsfreezeStatus ("thawed", "frozen", etc., as defined
> > # below)
> > #
> > -# Note: This may fail to properly report the current state as a result
> > -# of some other guest processes having issued an fs freeze/thaw.
> > +# .. note:: This may fail to properly report the current state as a
> > +#result of some other guest processes having issued an fs
> > +#freeze/thaw.
> > #
> > # Since: 0.15.0
> > ##
> > @@ -443,9 +444,9 @@
> > #
> > # Returns: Number of file systems currently frozen.
> > #
> > -# Note: On Windows, the command is implemented with the help of a
> > -# Volume Shadow-copy Service DLL helper. The frozen state is
> > -# limited for up to 10 seconds by VSS.
> > +# .. note:: On Windows, the command is implemented with the help of a
> > +#Volume Shadow-copy Service DLL helper. The frozen state is limited
> > +#for up to 10 seconds by VSS.
> > #
> > # Since: 0.15.0
> > ##
> > @@ -479,10 +480,10 @@
> > #
> > # Returns: Number of file systems thawed by this call
> > #
> > -# Note: if return value does not match the previous call to
> > -# guest-fsfreeze-freeze, this likely means some freezable
> > -# filesystems were unfrozen before this call, and that the
> > -# filesystem state may have changed before issuing this command.
> > +# .. note:: If return value does not match the previous call to
> > +#guest-fsfreeze-freeze, this likely means some freezable filesystems
> > +#were unfrozen before this call, and that the filesystem state may
> > +#have changed before issuing this command.
> > #
> > # Since: 0.15.0
> > ##
> > @@ -560,8 +561,8 @@
> > # Errors:
> > # - If suspend to disk is not supported, Unsupported
> > #
> > -# Notes: It's strongly recommended to issue the guest-sync command
> > -# before sending commands when the guest resumes
> > +# .. note:: It's strongly recommended to issue
Re: [PATCH 08/13] qapi: ensure all errors sections are uniformly typset
On Wed, Jun 19, 2024, 8:10 AM Markus Armbruster wrote: > John Snow writes: > > > Transactions have the only instance of an Errors section that isn't a > > rST list; turn it into one. > > > > Signed-off-by: John Snow > > Let;s explain the "why" a bit more clearly. Maybe > > qapi: Nail down convention that Errors sections are lists > > By unstated convention, Errors sections are rST lists. Document the > convention, and make the one exception conform. > > > --- > > docs/devel/qapi-code-gen.rst | 7 +++ > > qapi/transaction.json| 2 +- > > 2 files changed, 8 insertions(+), 1 deletion(-) > > > > diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst > > index f453bd35465..cee43222f19 100644 > > --- a/docs/devel/qapi-code-gen.rst > > +++ b/docs/devel/qapi-code-gen.rst > > @@ -1011,6 +1011,13 @@ like this:: > > "Returns" and "Errors" sections are only valid for commands. They > > document the success and the error response, respectively. > > > > +"Errors" sections should be formatted as an rST list, each entry > > +detailing a relevant error condition. For example:: > > + > > + # Errors: > > + # - If @device does not exist, DeviceNotFound > > + # - Any other error returns a GenericError. > > + > > A "Since: x.y.z" tagged section lists the release that introduced the > > definition. > > > > diff --git a/qapi/transaction.json b/qapi/transaction.json > > index 5749c133d4a..07afc269d54 100644 > > --- a/qapi/transaction.json > > +++ b/qapi/transaction.json > > @@ -235,7 +235,7 @@ > > # additional detail. > > # > > # Errors: > > -# Any errors from commands in the transaction > > +# - Any errors from commands in the transaction > > # > > # Note: The transaction aborts on the first failure. Therefore, there > > # will be information on only one failed operation returned in an > > Preferably with an improved commit message > Reviewed-by: Markus Armbruster > okie dokie. (Feel free to adjust the doc phrasing too, if you want. I promise I'm not offended by that.) >
Re: [PATCH 04/13] qapi/parser: preserve indentation in QAPIDoc sections
On Thu, Jun 20, 2024, 11:07 AM Markus Armbruster wrote: > John Snow writes: > > > On Wed, Jun 19, 2024, 8:03 AM Markus Armbruster > wrote: > > > >> John Snow writes: > >> > >> > Change get_doc_indented() to preserve indentation on all subsequent > text > >> > lines, and create a compatibility dedent() function for qapidoc.py to > >> > remove that indentation. This is being done for the benefit of a new > >> > >> Suggest "remove indentation the same way get_doc_indented() did." > >> > > > > Aight. > > > > > >> > qapidoc generator which requires that indentation in argument and > >> > features sections are preserved. > >> > > >> > Prior to this patch, a section like this: > >> > > >> > ``` > >> > @name: lorem ipsum > >> >dolor sit amet > >> > consectetur adipiscing elit > >> > ``` > >> > > >> > would have its body text be parsed as: > >> > >> Suggest "parsed into". > >> > > > > Why? (I mean, I'll do it, but I don't see the semantic difference > > personally) > > > > "Parse as " vs. "Parse into ". > > >> > (first and final newline only for presentation) > >> > > >> > ``` > >> > lorem ipsum > >> > dolor sit amet > >> > consectetur adipiscing elit > >> > ``` > >> > > >> > We want to preserve the indentation for even the first body line so > that > >> > the entire block can be parsed directly as rST. This patch would now > >> > parse that segment as: > >> > >> If you change "parsed as" to "parsed into" above, then do it here, too. > >> > >> > > >> > ``` > >> > lorem ipsum > >> >dolor sit amet > >> > consectetur adipiscing elit > >> > ``` > >> > > >> > This is helpful for formatting arguments and features as field lists > in > >> > rST, where the new generator will format this information as: > >> > > >> > ``` > >> > :arg type name: lorem ipsum > >> >dolor sit amet > >> > consectetur apidiscing elit > >> > ``` > >> > > >> > ...and can be formed by the simple concatenation of the field list > >> > construct and the body text. The indents help preserve the > continuation > >> > of a block-level element, and further allow the use of additional rST > >> > block-level constructs such as code blocks, lists, and other such > >> > markup. Avoiding reflowing the text conditionally also helps preserve > >> > source line context for better rST error reporting from sphinx through > >> > generated source, too. > >> > >> What do you mean by "reflowing"? > >> > > > > Poorly phrased, was thinking about emacs too much. I mean munging the > text > > post-hoc for the doc generator such that newlines are added or removed in > > the process of re-formatting text to get the proper indentation for the > new > > rST form. > > > > In prototyping, this got messy very quickly and was difficult to > correlate > > source line numbers across the transformation. > > > > It was easier to just not munge the text at all instead of munging it and > > then un-munging it. > > > > (semantic satiation: munge munge munge munge.) > > Is this about a possible alternative solution you explored? Keeping > .get_doc_indented() as is, and then try to undo its damage? > precisamente. That solution was categorically worse. > [...] > >
Re: [PATCH 04/13] qapi/parser: preserve indentation in QAPIDoc sections
On Wed, Jun 19, 2024, 8:03 AM Markus Armbruster wrote: > John Snow writes: > > > Change get_doc_indented() to preserve indentation on all subsequent text > > lines, and create a compatibility dedent() function for qapidoc.py to > > remove that indentation. This is being done for the benefit of a new > > Suggest "remove indentation the same way get_doc_indented() did." > Aight. > > qapidoc generator which requires that indentation in argument and > > features sections are preserved. > > > > Prior to this patch, a section like this: > > > > ``` > > @name: lorem ipsum > >dolor sit amet > > consectetur adipiscing elit > > ``` > > > > would have its body text be parsed as: > > Suggest "parsed into". > Why? (I mean, I'll do it, but I don't see the semantic difference personally) > > (first and final newline only for presentation) > > > > ``` > > lorem ipsum > > dolor sit amet > > consectetur adipiscing elit > > ``` > > > > We want to preserve the indentation for even the first body line so that > > the entire block can be parsed directly as rST. This patch would now > > parse that segment as: > > If you change "parsed as" to "parsed into" above, then do it here, too. > > > > > ``` > > lorem ipsum > >dolor sit amet > > consectetur adipiscing elit > > ``` > > > > This is helpful for formatting arguments and features as field lists in > > rST, where the new generator will format this information as: > > > > ``` > > :arg type name: lorem ipsum > >dolor sit amet > > consectetur apidiscing elit > > ``` > > > > ...and can be formed by the simple concatenation of the field list > > construct and the body text. The indents help preserve the continuation > > of a block-level element, and further allow the use of additional rST > > block-level constructs such as code blocks, lists, and other such > > markup. Avoiding reflowing the text conditionally also helps preserve > > source line context for better rST error reporting from sphinx through > > generated source, too. > > What do you mean by "reflowing"? > Poorly phrased, was thinking about emacs too much. I mean munging the text post-hoc for the doc generator such that newlines are added or removed in the process of re-formatting text to get the proper indentation for the new rST form. In prototyping, this got messy very quickly and was difficult to correlate source line numbers across the transformation. It was easier to just not munge the text at all instead of munging it and then un-munging it. (semantic satiation: munge munge munge munge.) > > This understandably breaks the existing qapidoc.py; so a new function is > > added there to dedent the text for compatibility. Once the new generator > > is merged, this function will not be needed any longer and can be > > dropped. > > > > I verified this patch changes absolutely nothing by comparing the > > md5sums of the QMP ref html pages both before and after the change, so > > it's certified inert. QAPI test output has been updated to reflect the > > new strategy of preserving indents for rST. > > I think the remainder is unnecessary detail. Drop? > As long as you're convinced it's safe, it's done its job and we thank it for its service 🫡 > > before: > > > > 69cde3d6f18b0f324badbb447d4381ce manual_before/interop/qemu-ga-ref.html > > 446e9381833def2adc779f1b90f2215f manual_before/interop/qemu-qmp-ref.html > > df0ad6c26cb4c28b85d663fe44609c12 > manual_before/interop/qemu-storage-daemon-qmp-ref.html > > > > after: > > > > 69cde3d6f18b0f324badbb447d4381ce manual/interop/qemu-ga-ref.html > > 446e9381833def2adc779f1b90f2215f manual/interop/qemu-qmp-ref.html > > df0ad6c26cb4c28b85d663fe44609c12 > manual/interop/qemu-storage-daemon-qmp-ref.html > > > > Signed-off-by: John Snow > > --- > > docs/sphinx/qapidoc.py | 29 - > > scripts/qapi/parser.py | 5 +++-- > > tests/qapi-schema/doc-good.out | 32 > > 3 files changed, 43 insertions(+), 23 deletions(-) > > > > diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py > > index e675966defa..f2f2005dd5f 100644 > > --- a/docs/sphinx/qapidoc.py > > +++ b/docs/sphinx/qapidoc.py > > @@ -26,6 +26,7 @@ > > > > import os > > import re > > +import textwrap > > > > from docutils import node
Re: [PATCH 13/13] qapi: convert "Example" sections to rST
On Wed, Jun 19, 2024, 9:20 AM Markus Armbruster wrote:
> John Snow writes:
>
> > Eliminate the "Example" sections in QAPI doc blocks, converting them
> > into QMP example code blocks. This is generally done in this patch by
> > converting "Example:" or "Examples:" lines into ".. code-block:: QMP"
> > lines.
> >
> > The old "Example:" or "Examples:" syntax is now caught as an error; but
> > with the previous commit, "Example::" is still permitted as explicit rST
> > syntax. ('Example' is not special in this case; any sentence that ends
> > with "::" will start an indented code block in rST.)
> >
> > The ".. code-block:: QMP" form explicitly applies the QMP lexer and will
> > loosely validate an example as valid QMP/JSON. The "::" form does not
> > apply any lexer in particular and will not emit any errors.
> >
> > It is possible to choose the QMP lexer with the "::" form by using the
> > Sphinx directive ".. highlight:: QMP" in the document above where the
> > example appears; but this changes the lexer for *all* subsequent "::"
> > style code-blocks in the document thereafter.
> >
> > This patch does not change the default lexer for the legacy qapidoc
> > generator documents; future patches for the new qapidoc generator *may*
> > change this default.
> >
> > This patch has several benefits:
> >
> > 1. Example sections can now be written more arbitrarily, mixing
> >explanatory paragraphs and code blocks however desired.
> >
> > 2. "Example sections" can now use fully arbitrary rST.
>
> Do the double-quotes signify something I'm missing?
>
They aren't *sections* (QAPIDoc terminology) anymore, but was at a loss for
more precise phrasing.
> >
> > 3. All code blocks are now lexed and validated as QMP; increasing
> >usability of the docs and ensuring validity of example snippets.
> >
> >(To some extent - This patch only gaurantees it lexes correctly, not
> >that it's valid under the JSON or QMP grammars. It will catch most
> >small mistakes, however.)
> >
> > 4. Each code-block can be captioned independently without bypassing the
> >QMP lexer/validator.
> >
> >(i.e. code blocks are now for *code* only, so we don't have to
> >sacrifice annotations/captions for having lexicographically correct
> >examples.)
> >
> > For any sections with more than one example, examples are split up into
> > multiple code-block regions. If annotations are present, those
> > annotations are converted into code-block captions instead, e.g.
> >
> > ```
> > Examples:
> >
> >1. Lorem Ipsum
> >
> >-> { "foo": "bar" }
> > ```
> >
> > Is rewritten as:
> >
> > ```
> > .. code-block:: QMP
> >:caption: Example: Lorem Ipsum
> >
> >-> { "foo": "bar" }
> > ```
> >
> > This process was only semi-automated:
> >
> > 1. Replace "Examples?:" sections with sed:
> >
> > sed -i 's|# Example:|# .. code-block:: QMP|' *.json
> > sed -i 's|# Examples:|# .. code-block:: QMP|' *.json
> >
> > 2. Identify sections that no longer parse successfully by attempting the
> >doc build, convert annotations into captions manually.
> >(Tedious, oh well.)
> >
> > 3. Add captions where still needed:
> >
> > sed -zi 's|# .. code-block:: QMP\n#\n|# .. code-block:: QMP\n#
> :caption: Example\n#\n|g' *.json
> >
> > Not fully ideal, but hopefully not something that has to be done very
> > often. (Or ever again.)
> >
> > Signed-off-by: John Snow
> > Acked-by: Stefan Hajnoczi [for block*.json]
>
> [...]
>
> > diff --git a/qapi/pci.json b/qapi/pci.json
> > index f51159a2c4c..9192212661b 100644
> > --- a/qapi/pci.json
> > +++ b/qapi/pci.json
> > @@ -182,7 +182,8 @@
> > #
> > # Since: 0.14
> > #
> > -# Example:
> > +# .. code-block:: QMP
> > +#:caption: Example
> > #
> > # -> { "execute": "query-pci" }
> > # <- { "return": [
> > @@ -311,8 +312,7 @@
> > # ]
> > #}
> > #
> > -# Note: This example has been shortened as the real response is too
> > -# long.
> > +# This example has been shortened as the real response is too long.
>
> Squash into PATCH 09.
>
> > #
> > ##
> > { 'command': 'query-pci', 'returns': ['PciInfo'] }
>
> Otherwise looks good to me except for the somewhat ugly rendering in
> HTML mentioned in the commit message.
>
ACK
> [...]
>
>
Re: [PATCH 03/13] docs/qapidoc: delint a tiny portion of the module
On Wed, Jun 19, 2024, 2:28 AM Markus Armbruster wrote: > John Snow writes: > > > In a forthcoming series that adds a new QMP documentation generator, it > > will be helpful to have a linting baseline. However, there's no need to > > shuffle around the deck chairs too much, because most of this code will > > be removed once that new qapidoc generator (the "transmogrifier") is in > > place. > > > > To ease my pain: just turn off the black auto-formatter for most, but > > not all, of qapidoc.py. This will help ensure that *new* code follows a > > coding standard without bothering too much with cleaning up the existing > > code. > > > > Code that I intend to keep is still subject to the delinting beam. > > > > Signed-off-by: John Snow > > --- > > docs/sphinx/qapidoc.py | 66 +- > > 1 file changed, 40 insertions(+), 26 deletions(-) > > > > diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py > > index f270b494f01..e675966defa 100644 > > --- a/docs/sphinx/qapidoc.py > > +++ b/docs/sphinx/qapidoc.py > > @@ -28,33 +28,42 @@ > > import re > > > > from docutils import nodes > > +from docutils.parsers.rst import Directive, directives > > from docutils.statemachine import ViewList > > -from docutils.parsers.rst import directives, Directive > > -from sphinx.errors import ExtensionError > > -from sphinx.util.nodes import nested_parse_with_titles > > -import sphinx > > -from qapi.gen import QAPISchemaVisitor > > from qapi.error import QAPIError, QAPISemError > > +from qapi.gen import QAPISchemaVisitor > > from qapi.schema import QAPISchema > > > > +import sphinx > > +from sphinx.errors import ExtensionError > > +from sphinx.util.nodes import nested_parse_with_titles > > + > > > > # Sphinx up to 1.6 uses AutodocReporter; 1.7 and later > > # use switch_source_input. Check borrowed from kerneldoc.py. > > -Use_SSI = sphinx.__version__[:3] >= '1.7' > > -if Use_SSI: > > +USE_SSI = sphinx.__version__[:3] >= "1.7" > > +if USE_SSI: > > from sphinx.util.docutils import switch_source_input > > else: > > -from sphinx.ext.autodoc import AutodocReporter > > +from sphinx.ext.autodoc import ( # pylint: > disable=no-name-in-module > > +AutodocReporter, > > +) > > > > > > -__version__ = '1.0' > > +__version__ = "1.0" > > > > > > +# Disable black auto-formatter until re-enabled: > > +# fmt: off > > + > > # Function borrowed from pydash, which is under the MIT license > > def intersperse(iterable, separator): > > """Yield the members of *iterable* interspersed with *separator*.""" > > iterable = iter(iterable) > > -yield next(iterable) > > +try: > > +yield next(iterable) > > +except StopIteration: > > +return > > This gets rid of pylint's > > docs/sphinx/qapidoc.py:82:10: R1708: Do not raise StopIteration in > generator, use return statement instead (stop-iteration-return) > > I considered the same change some time ago, and decided against it to > avoid deviating from pydash. StopIteration would be a programming error > here. > > Please *delete* the function instead: commit fd62bff901b removed its > last use. I'd do it in a separate commit, but that's up to you. > Oh! I didn't realize it wasn't being used. That's certainly easier :) > > for item in iterable: > > yield separator > > yield item > > @@ -451,6 +460,10 @@ def get_document_nodes(self): > > return self._top_node.children > > > > > > +# Turn the black formatter on for the rest of the file. > > +# fmt: on > > + > > + > > class QAPISchemaGenDepVisitor(QAPISchemaVisitor): > > """A QAPI schema visitor which adds Sphinx dependencies each module > > > > @@ -458,34 +471,34 @@ class QAPISchemaGenDepVisitor(QAPISchemaVisitor): > > that the generated documentation output depends on the input > > schema file associated with each module in the QAPI input. > > """ > > + > > def __init__(self, env, qapidir): > > self._env = env > > self._qapidir = qapidir > > > > def visit_module(self, name): > > if name != "./builtin": > > -qapifile = self._qapidir + '/' + name &
[PATCH 13/13] qapi: convert "Example" sections to rST
Eliminate the "Example" sections in QAPI doc blocks, converting them
into QMP example code blocks. This is generally done in this patch by
converting "Example:" or "Examples:" lines into ".. code-block:: QMP"
lines.
The old "Example:" or "Examples:" syntax is now caught as an error; but
with the previous commit, "Example::" is still permitted as explicit rST
syntax. ('Example' is not special in this case; any sentence that ends
with "::" will start an indented code block in rST.)
The ".. code-block:: QMP" form explicitly applies the QMP lexer and will
loosely validate an example as valid QMP/JSON. The "::" form does not
apply any lexer in particular and will not emit any errors.
It is possible to choose the QMP lexer with the "::" form by using the
Sphinx directive ".. highlight:: QMP" in the document above where the
example appears; but this changes the lexer for *all* subsequent "::"
style code-blocks in the document thereafter.
This patch does not change the default lexer for the legacy qapidoc
generator documents; future patches for the new qapidoc generator *may*
change this default.
This patch has several benefits:
1. Example sections can now be written more arbitrarily, mixing
explanatory paragraphs and code blocks however desired.
2. "Example sections" can now use fully arbitrary rST.
3. All code blocks are now lexed and validated as QMP; increasing
usability of the docs and ensuring validity of example snippets.
(To some extent - This patch only gaurantees it lexes correctly, not
that it's valid under the JSON or QMP grammars. It will catch most
small mistakes, however.)
4. Each code-block can be captioned independently without bypassing the
QMP lexer/validator.
(i.e. code blocks are now for *code* only, so we don't have to
sacrifice annotations/captions for having lexicographically correct
examples.)
For any sections with more than one example, examples are split up into
multiple code-block regions. If annotations are present, those
annotations are converted into code-block captions instead, e.g.
```
Examples:
1. Lorem Ipsum
-> { "foo": "bar" }
```
Is rewritten as:
```
.. code-block:: QMP
:caption: Example: Lorem Ipsum
-> { "foo": "bar" }
```
This process was only semi-automated:
1. Replace "Examples?:" sections with sed:
sed -i 's|# Example:|# .. code-block:: QMP|' *.json
sed -i 's|# Examples:|# .. code-block:: QMP|' *.json
2. Identify sections that no longer parse successfully by attempting the
doc build, convert annotations into captions manually.
(Tedious, oh well.)
3. Add captions where still needed:
sed -zi 's|# .. code-block:: QMP\n#\n|# .. code-block:: QMP\n#:caption:
Example\n#\n|g' *.json
Not fully ideal, but hopefully not something that has to be done very
often. (Or ever again.)
Signed-off-by: John Snow
Acked-by: Stefan Hajnoczi [for block*.json]
---
docs/devel/qapi-code-gen.rst| 15 ++--
qapi/acpi.json | 6 +-
qapi/block-core.json| 120 --
qapi/block.json | 60 +++--
qapi/char.json | 36 ++--
qapi/control.json | 12 ++-
qapi/dump.json | 12 ++-
qapi/machine-target.json| 3 +-
qapi/machine.json | 76 +++--
qapi/migration.json | 145 +++-
qapi/misc-target.json | 33 +---
qapi/misc.json | 48 +++
qapi/net.json | 30 +--
qapi/pci.json | 6 +-
qapi/qapi-schema.json | 6 +-
qapi/qdev.json | 15 +++-
qapi/qom.json | 20 +++--
qapi/replay.json| 12 ++-
qapi/rocker.json| 12 ++-
qapi/run-state.json | 45 ++
qapi/tpm.json | 9 +-
qapi/trace.json | 6 +-
qapi/transaction.json | 3 +-
qapi/ui.json| 62 +-
qapi/vfio.json | 3 +-
qapi/virtio.json| 38 +
qapi/yank.json | 6 +-
scripts/qapi/parser.py | 11 ++-
tests/qapi-schema/doc-good.json | 13 +--
tests/qapi-schema/doc-good.out | 18 ++--
tests/qapi-schema/doc-good.txt | 17 +---
31 files changed, 579 insertions(+), 319 deletions(-)
diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index ae97b335cbf..5bf184d16ce 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -995,8 +995,8 @@ line "Features:", like this::
# @feature: Description text
A tagged section begins with a paragraph that starts with on
[PATCH 12/13] qapi/parser: don't parse rST markup as section headers
The double-colon synax is rST formatting that precedes a literal code block. We do not want to capture these as QAPI-specific sections. Coerce blocks that start with e.g. "Example::" to be parsed as untagged paragraphs instead of special tagged sections. Signed-off-by: John Snow --- scripts/qapi/parser.py | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index 53b06a94508..971fdf61a09 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -545,10 +545,15 @@ def get_doc(self) -> 'QAPIDoc': line = self.get_doc_indented(doc) no_more_args = True elif match := re.match( -r'(Returns|Errors|Since|Notes?|Examples?|TODO): *', -line): +r'(Returns|Errors|Since|Notes?|Examples?|TODO)' +r'(?!::): *', +line, +): # tagged section +# Note: "sections" with two colons are left alone as +# rST markup and not interpreted as a section heading. + # TODO: Remove this error sometime in 2025 or so # after we've fully transitioned to the new qapidoc # generator. -- 2.44.0
[PATCH 04/13] qapi/parser: preserve indentation in QAPIDoc sections
Change get_doc_indented() to preserve indentation on all subsequent text
lines, and create a compatibility dedent() function for qapidoc.py to
remove that indentation. This is being done for the benefit of a new
qapidoc generator which requires that indentation in argument and
features sections are preserved.
Prior to this patch, a section like this:
```
@name: lorem ipsum
dolor sit amet
consectetur adipiscing elit
```
would have its body text be parsed as:
(first and final newline only for presentation)
```
lorem ipsum
dolor sit amet
consectetur adipiscing elit
```
We want to preserve the indentation for even the first body line so that
the entire block can be parsed directly as rST. This patch would now
parse that segment as:
```
lorem ipsum
dolor sit amet
consectetur adipiscing elit
```
This is helpful for formatting arguments and features as field lists in
rST, where the new generator will format this information as:
```
:arg type name: lorem ipsum
dolor sit amet
consectetur apidiscing elit
```
...and can be formed by the simple concatenation of the field list
construct and the body text. The indents help preserve the continuation
of a block-level element, and further allow the use of additional rST
block-level constructs such as code blocks, lists, and other such
markup. Avoiding reflowing the text conditionally also helps preserve
source line context for better rST error reporting from sphinx through
generated source, too.
This understandably breaks the existing qapidoc.py; so a new function is
added there to dedent the text for compatibility. Once the new generator
is merged, this function will not be needed any longer and can be
dropped.
I verified this patch changes absolutely nothing by comparing the
md5sums of the QMP ref html pages both before and after the change, so
it's certified inert. QAPI test output has been updated to reflect the
new strategy of preserving indents for rST.
before:
69cde3d6f18b0f324badbb447d4381ce manual_before/interop/qemu-ga-ref.html
446e9381833def2adc779f1b90f2215f manual_before/interop/qemu-qmp-ref.html
df0ad6c26cb4c28b85d663fe44609c12
manual_before/interop/qemu-storage-daemon-qmp-ref.html
after:
69cde3d6f18b0f324badbb447d4381ce manual/interop/qemu-ga-ref.html
446e9381833def2adc779f1b90f2215f manual/interop/qemu-qmp-ref.html
df0ad6c26cb4c28b85d663fe44609c12
manual/interop/qemu-storage-daemon-qmp-ref.html
Signed-off-by: John Snow
---
docs/sphinx/qapidoc.py | 29 -
scripts/qapi/parser.py | 5 +++--
tests/qapi-schema/doc-good.out | 32
3 files changed, 43 insertions(+), 23 deletions(-)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index e675966defa..f2f2005dd5f 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -26,6 +26,7 @@
import os
import re
+import textwrap
from docutils import nodes
from docutils.parsers.rst import Directive, directives
@@ -53,6 +54,21 @@
__version__ = "1.0"
+def dedent(text: str) -> str:
+# Temporary: In service of the new QAPI Sphinx domain, the QAPI doc
+# parser now preserves indents in args/members/features text.
+# QAPIDoc does not handle this well, so undo that change here.
+
+lines = text.splitlines(True)
+if re.match(r"\s+", lines[0]):
+# First line is indented; description started on the line after
+# the name. dedent the whole block.
+return textwrap.dedent(text)
+
+# Descr started on same line. Dedent line 2+.
+return lines[0] + textwrap.dedent("".join(lines[1:]))
+
+
# Disable black auto-formatter until re-enabled:
# fmt: off
@@ -176,7 +192,7 @@ def _nodes_for_members(self, doc, what, base=None,
branches=None):
term = self._nodes_for_one_member(section.member)
# TODO drop fallbacks when undocumented members are outlawed
if section.text:
-defn = section.text
+defn = dedent(section.text)
else:
defn = [nodes.Text('Not documented')]
@@ -214,7 +230,7 @@ def _nodes_for_enum_values(self, doc):
termtext.extend(self._nodes_for_ifcond(section.member.ifcond))
# TODO drop fallbacks when undocumented members are outlawed
if section.text:
-defn = section.text
+defn = dedent(section.text)
else:
defn = [nodes.Text('Not documented')]
@@ -249,7 +265,7 @@ def _nodes_for_features(self, doc):
dlnode = nodes.definition_list()
for section in doc.features.values():
dlnode += self._make_dlitem(
-[nodes.literal('', section.member.name)], section.text)
+[nodes.literal('', section.member.name)], dedent(section.text))
seen_item = True
if not seen_item:
@@ -
[PATCH 10/13] qapi: update prose in note blocks
Where I've noticed, rephrase the note to read more fluently. Signed-off-by: John Snow --- qapi/block-core.json | 4 ++-- qga/qapi-schema.json | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index cacedfb771c..9ef23ec02ae 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -6048,9 +6048,9 @@ # # @name: the name of the internal snapshot to be created # -# .. note:: In transaction, if @name is empty, or any snapshot matching +# .. note:: In a transaction, if @name is empty or any snapshot matching #@name exists, the operation will fail. Only some image formats -#support it, for example, qcow2, and rbd. +#support it; for example, qcow2, and rbd. # # Since: 1.7 ## diff --git a/qga/qapi-schema.json b/qga/qapi-schema.json index 57598331c5c..1273d85bb5f 100644 --- a/qga/qapi-schema.json +++ b/qga/qapi-schema.json @@ -480,7 +480,7 @@ # # Returns: Number of file systems thawed by this call # -# .. note:: If return value does not match the previous call to +# .. note:: If the return value does not match the previous call to #guest-fsfreeze-freeze, this likely means some freezable filesystems #were unfrozen before this call, and that the filesystem state may #have changed before issuing this command. -- 2.44.0
