Re: [PATCH v2 6/9] qapi: convert "Example" sections without titles
On Wed, Jul 17, 2024, 3:44 AM Markus Armbruster wrote:
> John Snow writes:
>
> > Use the no-option form of ".. qmp-example::" to convert any Examples
> > that do not have any form of caption or explanation whatsoever. Note
> > that in a few cases, example sections are split into two or more
> > separate example blocks. This is only done stylistically to create a
> > delineation between two or more logically independent examples.
> >
> > See commit-3: "docs/qapidoc: create qmp-example directive", for a
> > detailed explanation of this custom directive syntax.
> >
> > See commit+3: "qapi: remove "Example" doc section" for a detailed
> > explanation of why.
> >
> > Note: an empty "TODO" line was added to announce-self to keep the
> > example from floating up into the body; this will be addressed more
> > rigorously in the new qapidoc generator.
> >
> > Signed-off-by: John Snow
>
> [...]
>
> > diff --git a/qapi/run-state.json b/qapi/run-state.json
> > index 4d40c88876c..ab7116680b3 100644
> > --- a/qapi/run-state.json
> > +++ b/qapi/run-state.json
>
> [...]
>
> > @@ -469,7 +469,7 @@
> > #
> > # Since: 9.1
> > #
> > -# Example:
> > +# ..qmp-example::
>
> Lacks a space, output is messed up. Can fix in my tree.
>
Whoops. Wish rST was a bit pickier sometimes...
> > #
> > # <- { "event": "GUEST_PVSHUTDOWN",
> > # "timestamp": { "seconds": 1648245259, "microseconds": 893771
> } }
>
> [...]
>
> R-by stands.
>
>
Re: [PATCH v2 0/9] qapi: convert example sections to qmp-example rST directives
On Wed, Jul 17, 2024, 6:47 AM Markus Armbruster wrote: > John Snow writes: > > > This patchset focuses on converting example sections to rST directives > > using a new `.. qmp-example::` directive. > > Queued, thanks! > Yay! >
[PATCH v2 5/9] docs/sphinx: add CSS styling for qmp-example directive
From: Harmonie Snow
Add CSS styling for qmp-example directives to increase readability and
consistently style all example blocks.
Signed-off-by: Harmonie Snow
Signed-off-by: John Snow
Acked-by: Markus Armbruster
---
docs/sphinx-static/theme_overrides.css | 49 ++
1 file changed, 49 insertions(+)
diff --git a/docs/sphinx-static/theme_overrides.css
b/docs/sphinx-static/theme_overrides.css
index c70ef951286..965ecac54fd 100644
--- a/docs/sphinx-static/theme_overrides.css
+++ b/docs/sphinx-static/theme_overrides.css
@@ -87,6 +87,55 @@ div[class^="highlight"] pre {
padding-bottom: 1px;
}
+/* qmp-example directive styling */
+
+.rst-content .admonition-example {
+/* do not apply the standard admonition background */
+background-color: transparent;
+border: solid #ffd2ed 1px;
+}
+
+.rst-content .admonition-example > .admonition-title:before {
+content: "▷";
+}
+
+.rst-content .admonition-example > .admonition-title {
+background-color: #5980a6;
+}
+
+.rst-content .admonition-example > div[class^="highlight"] {
+/* make code boxes take up the full width of the admonition w/o margin */
+margin-left: -12px;
+margin-right: -12px;
+
+border-top: 1px solid #ffd2ed;
+border-bottom: 1px solid #ffd2ed;
+border-left: 0px;
+border-right: 0px;
+}
+
+.rst-content .admonition-example > div[class^="highlight"]:nth-child(2) {
+/* If a code box is the second element in an example admonition,
+ * it is the first child after the title. let it sit flush against
+ * the title. */
+margin-top: -12px;
+border-top: 0px;
+}
+
+.rst-content .admonition-example > div[class^="highlight"]:last-child {
+/* If a code box is the final element in an example admonition, don't
+ * render margin below it; let it sit flush with the end of the
+ * admonition box */
+margin-bottom: -12px;
+border-bottom: 0px;
+}
+
+.rst-content .admonition-example .highlight {
+background-color: #fffafd;
+}
+
+/* end qmp-example styling */
+
@media screen {
/* content column
--
2.45.0
[PATCH v2 2/9] docs/qapidoc: factor out do_parse()
Factor out the compatibility parser helper into a base class, so it can be shared by other directives. Signed-off-by: John Snow Reviewed-by: Markus Armbruster --- docs/sphinx/qapidoc.py | 32 +++- 1 file changed, 19 insertions(+), 13 deletions(-) diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py index 62b39833ca0..b3be82998a8 100644 --- a/docs/sphinx/qapidoc.py +++ b/docs/sphinx/qapidoc.py @@ -481,7 +481,25 @@ def visit_module(self, name): super().visit_module(name) -class QAPIDocDirective(Directive): +class NestedDirective(Directive): +def run(self): +raise NotImplementedError + +def do_parse(self, rstlist, node): +""" +Parse rST source lines and add them to the specified node + +Take the list of rST source lines rstlist, parse them as +rST, and add the resulting docutils nodes as children of node. +The nodes are parsed in a way that allows them to include +subheadings (titles) without confusing the rendering of +anything else. +""" +with switch_source_input(self.state, rstlist): +nested_parse_with_titles(self.state, rstlist, node) + + +class QAPIDocDirective(NestedDirective): """Extract documentation from the specified QAPI .json file""" required_argument = 1 @@ -519,18 +537,6 @@ def run(self): # so they are displayed nicely to the user raise ExtensionError(str(err)) from err -def do_parse(self, rstlist, node): -"""Parse rST source lines and add them to the specified node - -Take the list of rST source lines rstlist, parse them as -rST, and add the resulting docutils nodes as children of node. -The nodes are parsed in a way that allows them to include -subheadings (titles) without confusing the rendering of -anything else. -""" -with switch_source_input(self.state, rstlist): -nested_parse_with_titles(self.state, rstlist, node) - def setup(app): """Register qapi-doc directive with Sphinx""" -- 2.45.0
[PATCH v2 0/9] qapi: convert example sections to qmp-example rST directives
This patchset focuses on converting example sections to rST directives using a new `.. qmp-example::` directive. V2: - Rebased on origin/master; converted one more example - Fixed (most?) minor nits from last review - Didn't address lack of newline in text mode or enhanced lexer (yet) Changes since this was split out from the prior series: - Harmonie updated the CSS for the example block section. I think it's really tidy now! Thanks Harmonie! - Dependence on SphinxDirective was removed, but it will likely re-appear in the next series anyway. - qapi-code-gen.rst was updated with a section on how to write examples. - Various minor tweaks to comments, commit messages, docs, etc. Harmonie Snow (1): docs/sphinx: add CSS styling for qmp-example directive John Snow (8): [DO-NOT-MERGE]: Add some ad-hoc linting helpers. docs/qapidoc: factor out do_parse() docs/qapidoc: create qmp-example directive docs/qapidoc: add QMP highlighting to annotated qmp-example blocks qapi: convert "Example" sections without titles qapi: convert "Example" sections with titles qapi: convert "Example" sections with longer prose qapi: remove "Example" doc section docs/devel/qapi-code-gen.rst | 58 +-- docs/sphinx-static/theme_overrides.css | 49 ++ docs/sphinx/qapidoc.py | 130 ++--- qapi/acpi.json | 4 +- qapi/block-core.json | 88 + qapi/block.json| 57 ++- qapi/char.json | 24 +++-- qapi/control.json | 8 +- qapi/dump.json | 8 +- qapi/machine-target.json | 2 +- qapi/machine.json | 68 +++-- qapi/migration.json| 90 + qapi/misc-target.json | 22 ++--- qapi/misc.json | 32 +++--- qapi/net.json | 22 +++-- qapi/pci.json | 2 +- qapi/qdev.json | 10 +- qapi/qom.json | 16 +-- qapi/replay.json | 8 +- qapi/rocker.json | 8 +- qapi/run-state.json| 32 +++--- qapi/tpm.json | 6 +- qapi/trace.json| 4 +- qapi/transaction.json | 2 +- qapi/ui.json | 45 - qapi/vfio.json | 2 +- qapi/virtio.json | 45 ++--- qapi/yank.json | 4 +- scripts/qapi-lint.sh | 61 scripts/qapi/Makefile | 5 + scripts/qapi/parser.py | 10 +- tests/qapi-schema/doc-good.json| 19 ++-- tests/qapi-schema/doc-good.out | 26 +++-- tests/qapi-schema/doc-good.txt | 23 ++--- 34 files changed, 662 insertions(+), 328 deletions(-) create mode 100755 scripts/qapi-lint.sh create mode 100644 scripts/qapi/Makefile -- 2.45.0
[PATCH v2 7/9] qapi: convert "Example" sections with titles
When an Example section has a brief explanation, convert it to a
qmp-example:: section using the :title: option.
Rule of thumb: If the title can fit on a single line and requires no rST
markup, it's a good candidate for using the :title: option of
qmp-example.
In this patch, trailing punctuation is removed from the title section
for consistent headline aesthetics. In just one case, specifics of the
example are removed to make the title read better.
See commit-4: "docs/qapidoc: create qmp-example directive", for a
detailed explanation of this custom directive syntax.
See commit+2: "qapi: remove "Example" doc section" for a detailed
explanation of why.
Signed-off-by: John Snow
Reviewed-by: Markus Armbruster
---
qapi/block-core.json | 24
qapi/block.json | 13 ++---
qapi/migration.json | 25 ++---
qapi/qom.json| 8
qapi/ui.json | 11 ++-
qapi/virtio.json | 19 ++-
6 files changed, 52 insertions(+), 48 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 84a020aa9e3..f400b334c84 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -5881,9 +5881,8 @@
#
# Since: 2.7
#
-# Examples:
-#
-# 1. Add a new node to a quorum
+# .. qmp-example::
+#:title: Add a new node to a quorum
#
# -> { "execute": "blockdev-add",
# "arguments": {
@@ -5897,7 +5896,8 @@
# "node": "new_node" } }
# <- { "return": {} }
#
-# 2. Delete a quorum's node
+# .. qmp-example::
+#:title: Delete a quorum's node
#
# -> { "execute": "x-blockdev-change",
# "arguments": { "parent": "disk1",
@@ -5933,16 +5933,16 @@
#
# Since: 2.12
#
-# Examples:
-#
-# 1. Move a node into an IOThread
+# .. qmp-example::
+#:title: Move a node into an IOThread
#
# -> { "execute": "x-blockdev-set-iothread",
# "arguments": { "node-name": "disk1",
# "iothread": "iothread0" } }
# <- { "return": {} }
#
-# 2. Move a node into the main loop
+# .. qmp-example::
+#:title: Move a node into the main loop
#
# -> { "execute": "x-blockdev-set-iothread",
# "arguments": { "node-name": "disk1",
@@ -6018,16 +6018,16 @@
#
# Since: 2.0
#
-# Examples:
-#
-# 1. Read operation
+# .. qmp-example::
+#:title: Read operation
#
# <- { "event": "QUORUM_REPORT_BAD",
# "data": { "node-name": "node0", "sector-num": 345435,
"sectors-count": 5,
#"type": "read" },
# "timestamp": { "seconds": 1344522075, "microseconds": 745528 } }
#
-# 2. Flush operation
+# .. qmp-example::
+#:title: Flush operation
#
# <- { "event": "QUORUM_REPORT_BAD",
# "data": { "node-name": "node0", "sector-num": 0, "sectors-count":
2097120,
diff --git a/qapi/block.json b/qapi/block.json
index c8e52bc2d29..5ddd061e964 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -342,9 +342,8 @@
#
# Since: 2.5
#
-# Examples:
-#
-# 1. Change a removable medium
+# .. qmp-example::
+#:title: Change a removable medium
#
# -> { "execute": "blockdev-change-medium",
# "arguments": { "id": "ide0-1-0",
@@ -352,7 +351,8 @@
# "format": "raw" } }
# <- { "return": {} }
#
-# 2. Load a read-only medium into a writable drive
+# .. qmp-example::
+#:title: Load a read-only medium into a writable drive
#
# -> { "execute": "blockdev-change-medium",
# "arguments": { "id": "floppyA",
@@ -577,9 +577,8 @@
# "boundaries-write": [1000, 5000] } }
# <- { "return": {} }
#
-# Example:
-#
-# Remove all latency histograms:
+# .. qmp-example::
+#:title: Remove all latency histograms
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0" } }
diff --git a/qapi/migration.json b/qapi/migration.json
index 3c657202389..74968a14173 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -287,14 +287,14 @@
#
# Since: 0.14
#
-# Examples:
-#
-# 1. Before the first migration
+# .. qmp-example::
+#:title: Before the first migration
#
# -> { "execute": "query-migrate&qu
[PATCH v2 9/9] qapi: remove "Example" doc section
Fully eliminate the "Example" sections in QAPI doc blocks now that they
have all been converted to arbitrary rST syntax using the
".. qmp-example::" directive. Update tests to match.
Migrating to the new syntax
---
The old "Example:" or "Examples:" section syntax is now caught as an
error, but "Example::" is stil permitted as explicit rST syntax for an
un-lexed, generic preformatted text block.
('Example' is not special in this case, any sentence that ends with "::"
will start an indented code block in rST.)
Arbitrary rST for Examples is now possible, but it's strongly
recommended that documentation authors use the ".. qmp-example::"
directive for consistent visual formatting in rendered HTML docs. The
":title:" directive option may be used to add extra information into the
title bar for the example. The ":annotated:" option can be used to write
arbitrary rST instead, with nested "::" blocks applying QMP formatting
where desired.
Other choices available are ".. code-block:: QMP" which will not create
an "Example:" box, or the short-form "::" code-block syntax which will
not apply QMP highlighting when used outside of the qmp-example
directive.
Why?
This patch has several benefits:
1. Example sections can now be written more arbitrarily, mixing
explanatory paragraphs and code blocks however desired.
2. Example sections can now use fully arbitrary rST.
3. All code blocks are now lexed and validated as QMP; increasing
usability of the docs and ensuring validity of example snippets.
(To some extent - This patch only gaurantees it lexes correctly, not
that it's valid under the JSON or QMP grammars. It will catch most
small mistakes, however.)
4. Each qmp-example can be titled or annotated independently without
bypassing the QMP lexer/validator.
(i.e. code blocks are now for *code* only, so we don't have to
sacrifice exposition for having lexically valid examples.)
NOTE: As with the "Notes" conversion (d461c279737), this patch (and the
three preceding) may change the rendering order for Examples in
the current generator. The forthcoming qapidoc rewrite will fix
this by always generating documentation in source order.
Signed-off-by: John Snow
---
docs/devel/qapi-code-gen.rst| 58 -
scripts/qapi/parser.py | 10 +-
tests/qapi-schema/doc-good.json | 19 +++
tests/qapi-schema/doc-good.out | 26 ++-
tests/qapi-schema/doc-good.txt | 23 ++---
5 files changed, 98 insertions(+), 38 deletions(-)
diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index ae97b335cbf..583207a8ec2 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -899,7 +899,7 @@ Documentation markup
Documentation comments can use most rST markup. In particular,
-a ``::`` literal block can be used for examples::
+a ``::`` literal block can be used for pre-formatted text::
# ::
#
@@ -995,8 +995,8 @@ line "Features:", like this::
# @feature: Description text
A tagged section begins with a paragraph that starts with one of the
-following words: "Since:", "Example:"/"Examples:", "Returns:",
-"Errors:", "TODO:". It ends with the start of a new section.
+following words: "Since:", "Returns:", "Errors:", "TODO:". It ends with
+the start of a new section.
The second and subsequent lines of tagged sections must be indented
like this::
@@ -1020,13 +1020,53 @@ detailing a relevant error condition. For example::
A "Since: x.y.z" tagged section lists the release that introduced the
definition.
-An "Example" or "Examples" section is rendered entirely
-as literal fixed-width text. "TODO" sections are not rendered at all
-(they are for developers, not users of QMP). In other sections, the
-text is formatted, and rST markup can be used.
+"TODO" sections are not rendered (they are for developers, not users of
+QMP). In other sections, the text is formatted, and rST markup can be
+used.
+
+QMP Examples can be added by using the ``.. qmp-example::``
+directive. In its simplest form, this can be used to contain a single
+QMP code block which accepts standard JSON syntax with additional server
+directionality indicators (``->`` and ``<-``), and elisions (``...``).
+
+Optionally, a plaintext title may be provided by using the ``:title:``
+directive option. If the title is omitted, the example title will
+default to "Example:".
+
+A simple QMP example::
+
+ # .. qmp-example::
+ #:title: Using query-block
+ #
+ #-> { "execute": "query-block" }
+ #&l
[PATCH v2 8/9] qapi: convert "Example" sections with longer prose
These examples require longer explanations or have explanations that
require markup to look reasonable when rendered and so use the longer
form of the ".. qmp-example::" directive.
By using the :annotated: option, the content in the example block is
assumed *not* to be a code block literal and is instead parsed as normal
rST - with the exception that any code literal blocks after `::` will
assumed to be a QMP code literal block.
Note: There's one title-less conversion in this patch that comes along
for the ride because it's part of a larger "Examples" block that was
better to convert all at once.
See commit-5: "docs/qapidoc: create qmp-example directive", for a
detailed explanation of this custom directive syntax.
See commit+1: "qapi: remove "Example" doc section" for a detailed
explanation of why.
Signed-off-by: John Snow
Reviewed-by: Markus Armbruster
---
qapi/block.json | 30 +++---
qapi/machine.json | 30 --
qapi/migration.json | 7 +--
qapi/virtio.json| 24 ++--
4 files changed, 62 insertions(+), 29 deletions(-)
diff --git a/qapi/block.json b/qapi/block.json
index 5ddd061e964..ce9490a3678 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -545,31 +545,37 @@
#
# Since: 4.0
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# Set new histograms for all io types with intervals
-# [0, 10), [10, 50), [50, 100), [100, +inf):
+#Set new histograms for all io types with intervals
+#[0, 10), [10, 50), [50, 100), [100, +inf)::
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0",
# "boundaries": [10, 50, 100] } }
# <- { "return": {} }
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# Set new histogram only for write, other histograms will remain
-# not changed (or not created):
+#Set new histogram only for write, other histograms will remain
+#not changed (or not created)::
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0",
# "boundaries-write": [10, 50, 100] } }
# <- { "return": {} }
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# Set new histograms with the following intervals:
-# read, flush: [0, 10), [10, 50), [50, 100), [100, +inf)
-# write: [0, 1000), [1000, 5000), [5000, +inf)
+#Set new histograms with the following intervals:
+#
+#- read, flush: [0, 10), [10, 50), [50, 100), [100, +inf)
+#- write: [0, 1000), [1000, 5000), [5000, +inf)
+#
+#::
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0",
@@ -578,7 +584,9 @@
# <- { "return": {} }
#
# .. qmp-example::
-#:title: Remove all latency histograms
+#:annotated:
+#
+#Remove all latency histograms::
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0" } }
diff --git a/qapi/machine.json b/qapi/machine.json
index 83f60b319c7..0a5ffe652b7 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -1047,10 +1047,11 @@
#
# Since: 2.7
#
-# Examples:
+# .. qmp-example::
+#:annotated:
#
-# For pseries machine type started with -smp 2,cores=2,maxcpus=4
-# -cpu POWER8:
+#For pseries machine type started with
+#``-smp 2,cores=2,maxcpus=4 -cpu POWER8``::
#
# -> { "execute": "query-hotpluggable-cpus" }
# <- {"return": [
@@ -1060,7 +1061,10 @@
#"vcpus-count": 1, "qom-path": "/machine/unattached/device[0]"}
#]}
#
-# For pc machine type started with -smp 1,maxcpus=2:
+# .. qmp-example::
+#:annotated:
+#
+#For pc machine type started with ``-smp 1,maxcpus=2``::
#
# -> { "execute": "query-hotpluggable-cpus" }
# <- {"return": [
@@ -1075,8 +1079,11 @@
# }
#]}
#
-# For s390x-virtio-ccw machine type started with -smp 1,maxcpus=2
-# -cpu qemu (Since: 2.11):
+# .. qmp-example::
+#:annotated:
+#
+#For s390x-virtio-ccw machine type started with
+#``-smp 1,maxcpus=2 -cpu qemu`` (Since: 2.11)::
#
# -> { "execute": "query-hotpluggable-cpus" }
# <- {"return": [
@@ -1130,12 +1137,15 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# -> { "execute": "balloon", "arguments": { "value": 536870912 } }
-# <- { "return": {} }
+#::
#
-#
[PATCH v2 6/9] qapi: convert "Example" sections without titles
Use the no-option form of ".. qmp-example::" to convert any Examples
that do not have any form of caption or explanation whatsoever. Note
that in a few cases, example sections are split into two or more
separate example blocks. This is only done stylistically to create a
delineation between two or more logically independent examples.
See commit-3: "docs/qapidoc: create qmp-example directive", for a
detailed explanation of this custom directive syntax.
See commit+3: "qapi: remove "Example" doc section" for a detailed
explanation of why.
Note: an empty "TODO" line was added to announce-self to keep the
example from floating up into the body; this will be addressed more
rigorously in the new qapidoc generator.
Signed-off-by: John Snow
---
qapi/acpi.json | 4 +--
qapi/block-core.json | 64 +---
qapi/block.json | 18 ++-
qapi/char.json | 24 +--
qapi/control.json| 8 ++---
qapi/dump.json | 8 ++---
qapi/machine-target.json | 2 +-
qapi/machine.json| 38
qapi/migration.json | 58 ++--
qapi/misc-target.json| 22 +++---
qapi/misc.json | 32 ++--
qapi/net.json| 22 --
qapi/pci.json| 2 +-
qapi/qdev.json | 10 ---
qapi/qom.json| 8 ++---
qapi/replay.json | 8 ++---
qapi/rocker.json | 8 ++---
qapi/run-state.json | 32 ++--
qapi/tpm.json| 6 ++--
qapi/trace.json | 4 +--
qapi/transaction.json| 2 +-
qapi/ui.json | 34 ++---
qapi/vfio.json | 2 +-
qapi/virtio.json | 2 +-
qapi/yank.json | 4 +--
25 files changed, 219 insertions(+), 203 deletions(-)
diff --git a/qapi/acpi.json b/qapi/acpi.json
index aa4dbe57943..045dab6228b 100644
--- a/qapi/acpi.json
+++ b/qapi/acpi.json
@@ -111,7 +111,7 @@
#
# Since: 2.1
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-acpi-ospm-status" }
# <- { "return": [ { "device": "d1", "slot": "0", "slot-type": "DIMM",
"source": 1, "status": 0},
@@ -131,7 +131,7 @@
#
# Since: 2.1
#
-# Example:
+# .. qmp-example::
#
# <- { "event": "ACPI_DEVICE_OST",
# "data": { "info": { "device": "d1", "slot": "0",
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 096bdbe0aad..84a020aa9e3 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -764,7 +764,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-block" }
# <- {
@@ -1168,7 +1168,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-blockstats" }
# <- {
@@ -1461,7 +1461,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "block_resize",
# "arguments": { "device": "scratch", "size": 1073741824 } }
@@ -1680,7 +1680,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "blockdev-snapshot-sync",
# "arguments": { "device": "ide-hd0",
@@ -1711,7 +1711,7 @@
#
# Since: 2.5
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "blockdev-add",
# "arguments": { "driver": "qcow2",
@@ -1857,7 +1857,7 @@
#
# Since: 1.3
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "block-commit",
# "arguments": { "device": "virtio0",
@@ -1895,7 +1895,7 @@
#
# Since: 1.6
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "drive-backup",
# "arguments": { "device": "drive0",
@@ -1921,7 +1921,7 @@
#
# Since: 2.3
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "blockdev-backup",
# "arguments": { "device": "src-id",
@@ -1945,7 +1945,7 @@
#
# Since: 2.0
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-named-block-nodes" }
# <- { "return": [ { "ro":false,
@@ -2126,7 +2126,7 @@
#
# Since: 1.3
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "drive-mirror",
# "arguments": { "device": "ide-hd0",
@@ -2303,7 +2303,7 @@
#
# Since: 2.4
#
-# Example:
+# .. qmp-exampl
[PATCH v2 1/9] [DO-NOT-MERGE]: Add some ad-hoc linting helpers.
These aren't ready for upstream inclusion, because they do not properly manage version dependencies, execution environment and so on. These are just the tools I use in my Own Special Environment :tm: for testing and debugging. They've been tested only on Fedora 38 for right now, which means: Python 3.11.4-1.fc38 pylint 2.17.4-2.fc38 mypy 1.4.0-1.fc38 isort 5.12.0-1.fc38 flake8 5.0.3-2.fc38 "Soon" :tm: I'll move the qapi generator code under the python/ directory to take advantage of the more robust linting infrastructure there, but that's going to happen after this series. Sorry about that! Signed-off-by: John Snow --- scripts/qapi-lint.sh | 61 +++ scripts/qapi/Makefile | 5 2 files changed, 66 insertions(+) create mode 100755 scripts/qapi-lint.sh create mode 100644 scripts/qapi/Makefile diff --git a/scripts/qapi-lint.sh b/scripts/qapi-lint.sh new file mode 100755 index 000..732716f2444 --- /dev/null +++ b/scripts/qapi-lint.sh @@ -0,0 +1,61 @@ +#!/usr/bin/env bash +set -e + +if [[ -f qapi/.flake8 ]]; then +echo "flake8 --config=qapi/.flake8 qapi/" +flake8 --config=qapi/.flake8 qapi/ +fi +if [[ -f qapi/pylintrc ]]; then +echo "pylint --rcfile=qapi/pylintrc qapi/" +pylint --rcfile=qapi/pylintrc qapi/ +fi +if [[ -f qapi/mypy.ini ]]; then +echo "mypy --config-file=qapi/mypy.ini qapi/" +mypy --config-file=qapi/mypy.ini qapi/ +fi + +if [[ -f qapi/.isort.cfg ]]; then +pushd qapi +echo "isort -c ." +isort -c . +popd +fi + +if [[ -f ../docs/sphinx/qapi-domain.py ]]; then +pushd ../docs/sphinx + +echo "mypy --strict qapi-domain.py" +mypy --strict qapi-domain.py +echo "flake8 qapi-domain.py --max-line-length=99" +flake8 qapi-domain.py --max-line-length=99 +echo "isort qapi-domain.py" +isort qapi-domain.py +echo "black --check qapi-domain.py" +black --check qapi-domain.py + +popd +fi + +if [[ -f ../docs/sphinx/qapidoc.py ]]; then +pushd ../docs/sphinx + +echo "pylint --rc-file ../../scripts/qapi/pylintrc qapidoc.py" +PYTHONPATH=../scripts/ pylint \ + --rc-file ../../scripts/qapi/pylintrc \ + qapidoc.py +echo "flake8 qapidoc.py --max-line-length=80" +flake8 qapidoc.py --max-line-length=80 +echo "isort qapidoc.py" +isort qapidoc.py +echo "black --line-length 80 --check qapidoc.py" +black --line-length 80 --check qapidoc.py + +popd +fi + +pushd ../build +make -j13 +make check-qapi-schema +make docs +make sphinxdocs +popd diff --git a/scripts/qapi/Makefile b/scripts/qapi/Makefile new file mode 100644 index 000..314e8a5505e --- /dev/null +++ b/scripts/qapi/Makefile @@ -0,0 +1,5 @@ +check: + isort -c . + flake8 . + cd .. && pylint --rcfile=qapi/pylintrc qapi + cd .. && mypy -p qapi --config-file=qapi/mypy.ini -- 2.45.0
[PATCH v2 3/9] docs/qapidoc: create qmp-example directive
This is a directive that creates a syntactic sugar for creating
"Example" boxes very similar to the ones already used in the bitmaps.rst
document, please see e.g.
https://www.qemu.org/docs/master/interop/bitmaps.html#creation-block-dirty-bitmap-add
In its simplest form, when a custom title is not needed or wanted, and
the example body is *solely* a QMP example:
```
.. qmp-example::
{body}
```
is syntactic sugar for:
```
.. admonition:: Example:
.. code-block:: QMP
{body}
```
When a custom, plaintext title that describes the example is desired,
this form:
```
.. qmp-example::
:title: Defrobnification
{body}
```
Is syntactic sugar for:
```
.. admonition:: Example: Defrobnification
.. code-block:: QMP
{body}
```
Lastly, when Examples are multi-step processes that require non-QMP
exposition, have lengthy titles, or otherwise involve prose with rST
markup (lists, cross-references, etc), the most complex form:
```
.. qmp-example::
:annotated:
This example shows how to use `foo-command`::
{body}
For more information, please see `frobnozz`.
```
Is desugared to:
```
.. admonition:: Example:
This example shows how to use `foo-command`::
{body}
For more information, please see `frobnozz`.
```
Note that :annotated: and :title: options can be combined together, if
desired.
The primary benefit here being documentation source consistently using
the same directive for all forms of examples to ensure consistent visual
styling, and ensuring all relevant prose is visually grouped alongside
the code literal block.
Note that as of this commit, the code-block rST syntax "::" does not
apply QMP highlighting; you would need to use ".. code-block:: QMP". The
very next commit changes this behavior to assume all "::" code blocks
within this directive are QMP blocks.
Signed-off-by: John Snow
Acked-by: Markus Armbruster
---
docs/sphinx/qapidoc.py | 55 ++
1 file changed, 55 insertions(+)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index b3be82998a8..11defcfa3f6 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -27,6 +27,7 @@
import os
import re
import textwrap
+from typing import List
from docutils import nodes
from docutils.parsers.rst import Directive, directives
@@ -35,6 +36,7 @@
from qapi.gen import QAPISchemaVisitor
from qapi.schema import QAPISchema
+from sphinx.directives.code import CodeBlock
from sphinx.errors import ExtensionError
from sphinx.util.docutils import switch_source_input
from sphinx.util.nodes import nested_parse_with_titles
@@ -538,10 +540,63 @@ def run(self):
raise ExtensionError(str(err)) from err
+class QMPExample(CodeBlock, NestedDirective):
+"""
+Custom admonition for QMP code examples.
+
+When the :annotated: option is present, the body of this directive
+is parsed as normal rST instead. Code blocks must be explicitly
+written by the user, but this allows for intermingling explanatory
+paragraphs with arbitrary rST syntax and code blocks for more
+involved examples.
+
+When :annotated: is absent, the directive body is treated as a
+simple standalone QMP code block literal.
+"""
+
+required_argument = 0
+optional_arguments = 0
+has_content = True
+option_spec = {
+"annotated": directives.flag,
+"title": directives.unchanged,
+}
+
+def admonition_wrap(self, *content) -> List[nodes.Node]:
+title = "Example:"
+if "title" in self.options:
+title = f"{title} {self.options['title']}"
+
+admon = nodes.admonition(
+"",
+nodes.title("", title),
+*content,
+classes=["admonition", "admonition-example"],
+)
+return [admon]
+
+def run_annotated(self) -> List[nodes.Node]:
+content_node: nodes.Element = nodes.section()
+self.do_parse(self.content, content_node)
+return content_node.children
+
+def run(self) -> List[nodes.Node]:
+annotated = "annotated" in self.options
+
+if annotated:
+content_nodes = self.run_annotated()
+else:
+self.arguments = ["QMP"]
+content_nodes = super().run()
+
+return self.admonition_wrap(*content_nodes)
+
+
def setup(app):
"""Register qapi-doc directive with Sphinx"""
app.add_config_value("qapidoc_srctree", None, "env")
app.add_directive("qapi-doc", QAPIDocDirective)
+app.add_directive("qmp-example", QMPExample)
return {
"version": __version__,
--
2.45.0
[PATCH v2 4/9] docs/qapidoc: add QMP highlighting to annotated qmp-example blocks
For any code literal blocks inside of a qmp-example directive, apply and
enforce the QMP lexer/highlighter to those blocks.
This way, you won't need to write:
```
.. qmp-example::
:annotated:
Blah blah
.. code-block:: QMP
-> { "lorem": "ipsum" }
```
But instead, simply:
```
.. qmp-example::
:annotated:
Blah blah::
-> { "lorem": "ipsum" }
```
Once the directive block is exited, whatever the previous default
highlight language was will be restored; localizing the forced QMP
lexing to exclusively this directive.
Note, if the default language is *already* QMP, this directive will not
generate and restore redundant highlight configuration nodes. We may
well decide that the default language ought to be QMP for any QAPI
reference pages, but this way the directive behaves consistently no
matter where it is used.
Signed-off-by: John Snow
Acked-by: Markus Armbruster
---
docs/sphinx/qapidoc.py | 53 ++
1 file changed, 49 insertions(+), 4 deletions(-)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index 11defcfa3f6..738b2450fb1 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -26,6 +26,7 @@
import os
import re
+import sys
import textwrap
from typing import List
@@ -36,6 +37,7 @@
from qapi.gen import QAPISchemaVisitor
from qapi.schema import QAPISchema
+from sphinx import addnodes
from sphinx.directives.code import CodeBlock
from sphinx.errors import ExtensionError
from sphinx.util.docutils import switch_source_input
@@ -545,10 +547,10 @@ class QMPExample(CodeBlock, NestedDirective):
Custom admonition for QMP code examples.
When the :annotated: option is present, the body of this directive
-is parsed as normal rST instead. Code blocks must be explicitly
-written by the user, but this allows for intermingling explanatory
-paragraphs with arbitrary rST syntax and code blocks for more
-involved examples.
+is parsed as normal rST, but with any '::' code blocks set to use
+the QMP lexer. Code blocks must be explicitly written by the user,
+but this allows for intermingling explanatory paragraphs with
+arbitrary rST syntax and code blocks for more involved examples.
When :annotated: is absent, the directive body is treated as a
simple standalone QMP code block literal.
@@ -562,6 +564,33 @@ class QMPExample(CodeBlock, NestedDirective):
"title": directives.unchanged,
}
+def _highlightlang(self) -> addnodes.highlightlang:
+"""Return the current highlightlang setting for the document"""
+node = None
+doc = self.state.document
+
+if hasattr(doc, "findall"):
+# docutils >= 0.18.1
+for node in doc.findall(addnodes.highlightlang):
+pass
+else:
+for elem in doc.traverse():
+if isinstance(elem, addnodes.highlightlang):
+node = elem
+
+if node:
+return node
+
+# No explicit directive found, use defaults
+node = addnodes.highlightlang(
+lang=self.env.config.highlight_language,
+force=False,
+# Yes, Sphinx uses this value to effectively disable line
+# numbers and not 0 or None or -1 or something. ¯\_(ツ)_/¯
+linenothreshold=sys.maxsize,
+)
+return node
+
def admonition_wrap(self, *content) -> List[nodes.Node]:
title = "Example:"
if "title" in self.options:
@@ -576,8 +605,24 @@ def admonition_wrap(self, *content) -> List[nodes.Node]:
return [admon]
def run_annotated(self) -> List[nodes.Node]:
+lang_node = self._highlightlang()
+
content_node: nodes.Element = nodes.section()
+
+# Configure QMP highlighting for "::" blocks, if needed
+if lang_node["lang"] != "QMP":
+content_node += addnodes.highlightlang(
+lang="QMP",
+force=False, # "True" ignores lexing errors
+linenothreshold=lang_node["linenothreshold"],
+)
+
self.do_parse(self.content, content_node)
+
+# Restore prior language highlighting, if needed
+if lang_node["lang"] != "QMP":
+content_node += addnodes.highlightlang(**lang_node.attributes)
+
return content_node.children
def run(self) -> List[nodes.Node]:
--
2.45.0
Re: [RFC PATCH 0/8] Convert avocado tests to normal Python unittests
On Thu, Jul 11, 2024, 7:55 AM Thomas Huth wrote: > The Avocado v88 that we use in QEMU is already on a life support > system: It is not supported by upstream anymore, and with the latest > versions of Python, it won't work anymore since it depends on the > "imp" module that has been removed in Python 3.12. > > There have been several attempts to update the test suite in QEMU > to a newer version of Avocado, but so far no attempt has successfully > been merged yet. > > Additionally, the whole "make check" test suite in QEMU is using the > meson test runner nowadays, so running the python-based tests via the > Avocodo test runner looks and feels quite like an oddball, requiring > the users to deal with the knowledge of multiple test runners in > parallel. > > So instead of trying to update the python-based test suite in QEMU > to a newer version of Avocado, we should maybe try to better integrate > it with the meson test runner instead. Indeed most tests work quite > nicely without the Avocado framework already, as you can see with > this patch series - it does not convert all tests, just a subset since > it is just an RFC so far, but as you can see, many tests only need > small modifications to work without Avocado. > > If you want to try it: Apply the patches, make sure that you have the > "pytest" program installed, then recompile and then run: > > make check-pytest > > Things that need further attention though: > > - All tests that use the LinuxTest / LinuxDistro classes (e.g. based > on cloud-init images) really depend on the Avocado framework, > thus we'd need a solution for those if we want to continue with > this approach > > - Same for all tests that require the LinuxSSHMixIn class - we'd > need to provide a solution for ssh-based tests, too. > > - We lose the way of running tests via the avocado tags this way... > single targets can still be tested by running "make check-pytest-arm" > for example, but running selected tests by other tags does not > work anymore. > > - I haven't looked into logging yet ... this still needs some work > so that you could e.g. inspect the console output of the guests > somewhere > This has spilled the most developer blood of any other problem with the Python-based tests. Be very careful here. I still have a prototype for replacing QMPMachine with an asyncio variant that should have more robust logging features, but I put it on the back-burner. Avocado tests are the primary user of the QMP Machine interface I hate the very most, a multi-threaded buffer-reader that works only by the grace of god. If you do go down this path, I may want to take the opportunity to abolish that interface once and for all. I think simplifying the console buffering will help ease debuggability. (Note, this isn't an avocado exclusive problem so much as it is the emergent evolution of both qmp machine and avocado developing their own solutions to console logging problems, resulting in two layers that are trying to do similar things.) > - I did not work on documentation updates yet (will do that if we > agree to continue with this patch series) > > What's your thoughts? Is it worth to continue with this approach? > Or shall I rather forget about it and wait for the Avocado version > update? > I'm personally ambivalent on avocado; I use it for the python self-tests as dogfooding but I can likely switch back over to plain pytest if that's the direction we head. I don't think I use any crazy features except some asyncio helpers i advocated for. I'm not sure what pytest's asyncio support looks like, but I have to imagine as the premier testing framework that it has *something* for me to use. My only ask is that we keep the tests running in the custom venv environment we set up at build time. We have some funky post-hoc initialization of avocado that allows us to use internet packages post-config for testing purposes. If we move to pytest, it's possible we can eliminate that funkiness, which would be a win. I'm also not so sure about recreating all of the framework that pulls vm images on demand, that sounds like it'd be a lot of work, but maybe I'm wrong about that. Tacit ACK from me on this project in general, provided we are still using the configure venv. > Thomas > > > Ani Sinha (1): > tests/pytest: add pytest to the meson build system > > Thomas Huth (7): > tests/pytest: Add base classes for the upcoming pytest-based tests > tests/pytest: Convert some simple avocado tests into pytests > tests/pytest: Convert info_usernet and version test with small > adjustments > tests_pytest: Implement fetch_asset() method for downloading assets > tests/pytest: Convert some tests that download files via fetch_asset() > tests/pytest: Add a function for extracting files from an archive > tests/pytest: Convert avocado test that needed avocado.utils.archive > > tests/Makefile.include| 4 +- > tests/meson.build
Re: [Stable-9.0.2 26/27] sphinx/qapidoc: Fix to generate doc for explicit, unboxed arguments
On Mon, Jul 15, 2024, 5:34 AM Michael Tokarev wrote:
> From: Markus Armbruster
>
> When a command's arguments are specified as an explicit type T,
> generated documentation points to the members of T.
>
> Example:
>
> ##
> # @announce-self:
> #
> # Trigger generation of broadcast RARP frames to update network
> [...]
> ##
> { 'command': 'announce-self', 'boxed': true,
> 'data' : 'AnnounceParameters'}
>
> generates
>
> "announce-self" (Command)
> -
>
> Trigger generation of broadcast RARP frames to update network
> [...]
>
> Arguments
> ~
>
> The members of "AnnounceParameters"
>
> Except when the command takes its arguments unboxed , i.e. it doesn't
> have 'boxed': true, we generate *nothing*. A few commands have a
> reference in their doc comment to compensate, but most don't.
>
> Example:
>
> ##
> # @blockdev-snapshot-sync:
> #
> # Takes a synchronous snapshot of a block device.
> #
> # For the arguments, see the documentation of BlockdevSnapshotSync.
> [...]
> ##
> { 'command': 'blockdev-snapshot-sync',
> 'data': 'BlockdevSnapshotSync',
> 'allow-preconfig': true }
>
> generates
>
> "blockdev-snapshot-sync" (Command)
> ~~
>
> Takes a synchronous snapshot of a block device.
>
> For the arguments, see the documentation of BlockdevSnapshotSync.
> [...]
>
> Same for event data.
>
> Fix qapidoc.py to generate the reference regardless of boxing. Delete
> now redundant references in the doc comments.
>
> Fixes: 4078ee5469e5 (docs/sphinx: Add new qapi-doc Sphinx extension)
> Cc: qemu-sta...@nongnu.org
> Signed-off-by: Markus Armbruster
> Message-ID: <20240628112756.794237-1-arm...@redhat.com>
> Reviewed-by: John Snow
> (cherry picked from commit e389929d19a543ea5b34d02553b355f9f1c03162)
> Signed-off-by: Michael Tokarev
>
Just do me a favor and test build the html docs in the stable branch and
verify that they appear to render correctly with a quick visual glance --
we've been poking around a lot in that area.
I don't see any obvious point of conflict in this patch, but I have a
nagging doubt.
As long as they build and they look alright, I'm confident it's fine.
> diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> index 8d428c64b0..8d3518e887 100644
> --- a/docs/sphinx/qapidoc.py
> +++ b/docs/sphinx/qapidoc.py
> @@ -219,15 +219,15 @@ def _nodes_for_enum_values(self, doc):
> section += dlnode
> return [section]
>
> -def _nodes_for_arguments(self, doc, boxed_arg_type):
> +def _nodes_for_arguments(self, doc, arg_type):
> """Return list of doctree nodes for the arguments section"""
> -if boxed_arg_type:
> +if arg_type and not arg_type.is_implicit():
> assert not doc.args
> section = self._make_section('Arguments')
> dlnode = nodes.definition_list()
> dlnode += self._make_dlitem(
> [nodes.Text('The members of '),
> - nodes.literal('', boxed_arg_type.name)],
> + nodes.literal('', arg_type.name)],
> None)
> section += dlnode
> return [section]
> @@ -331,8 +331,7 @@ def visit_command(self, name, info, ifcond, features,
> arg_type,
>allow_preconfig, coroutine):
> doc = self._cur_doc
> self._add_doc('Command',
> - self._nodes_for_arguments(doc,
> -arg_type if boxed else
> None)
> + self._nodes_for_arguments(doc, arg_type)
>+ self._nodes_for_features(doc)
>+ self._nodes_for_sections(doc)
>+ self._nodes_for_if_section(ifcond))
> @@ -340,8 +339,7 @@ def visit_command(self, name, info, ifcond, features,
> arg_type,
> def visit_event(self, name, info, ifcond, features, arg_type, boxed):
> doc = self._cur_doc
> self._add_doc('Event',
> - self._nodes_for_arguments(doc,
> -arg_type if boxed else
> None)
> + self._nodes_for_arguments(doc, arg_type)
>+ self._nodes_for_features(doc)
>+ self._nodes_for_sections(doc)
>+ self._nodes_for_if_section(ifcond))
> diff --git a/qapi/block-core.json b/qapi/block-core.json
> index 746
Re: [PATCH 5/5] qapi/ui: Drop note on naming of SpiceQueryMouseMode
On Thu, Jul 11, 2024, 7:22 AM Markus Armbruster wrote:
> Doc comments are reference documentation for users of QMP.
> SpiceQueryMouseMode's doc comment contains a note explaining why it's
> not named SpiceMouseMode: spice/enums.h has it already. Irrelevant
> for users of QMP; delete the note.
>
Hmmm, yeah, alright. I feel like this one's sorta borderline but I don't
feel strongly enough about it to argue to keep it :)
> Signed-off-by: Markus Armbruster
>
Reviewed-by: John Snow
---
> qapi/ui.json | 2 --
> 1 file changed, 2 deletions(-)
>
> diff --git a/qapi/ui.json b/qapi/ui.json
> index 5bcccbfc93..df089869a5 100644
> --- a/qapi/ui.json
> +++ b/qapi/ui.json
> @@ -273,8 +273,6 @@
> # @unknown: No information is available about mouse mode used by the
> # spice server.
> #
> -# .. note:: spice/enums.h has a SpiceMouseMode already, hence the name.
> -#
> # Since: 1.1
> ##
> { 'enum': 'SpiceQueryMouseMode',
> --
> 2.45.0
>
>
Re: [PATCH 4/5] qapi/sockets: Move deprecation note out of SocketAddress doc comment
On Thu, Jul 11, 2024, 7:22 AM Markus Armbruster wrote:
> Doc comments are reference documentation for users of QMP.
> SocketAddress's doc comment contains a deprecation note advising
> developers to use SocketAddress for new code. Irrelevant for users of
> QMP. Move the note out of the doc comment.
>
Sure.
> Signed-off-by: Markus Armbruster
>
Reviewed-by: John Snow
---
> qapi/sockets.json | 7 +++
> 1 file changed, 3 insertions(+), 4 deletions(-)
>
> diff --git a/qapi/sockets.json b/qapi/sockets.json
> index 4d78d2ccb7..e76fdb9925 100644
> --- a/qapi/sockets.json
> +++ b/qapi/sockets.json
> @@ -179,10 +179,6 @@
> #
> # @type: Transport type
> #
> -# .. note:: This type is deprecated in favor of SocketAddress. The
> -#difference between SocketAddressLegacy and SocketAddress is that
> -#the latter has fewer ``{}`` on the wire.
> -#
> # Since: 1.3
> ##
> { 'union': 'SocketAddressLegacy',
> @@ -193,6 +189,9 @@
> 'unix': 'UnixSocketAddressWrapper',
> 'vsock': 'VsockSocketAddressWrapper',
> 'fd': 'FdSocketAddressWrapper' } }
> +# Note: This type is deprecated in favor of SocketAddress. The
> +# difference between SocketAddressLegacy and SocketAddress is that the
> +# latter has fewer ``{}`` on the wire.
>
> ##
> # @SocketAddressType:
> --
> 2.45.0
>
>
Re: [PATCH 3/5] qapi/machine: Clarify query-uuid value when none has been specified
On Thu, Jul 11, 2024, 7:22 AM Markus Armbruster wrote:
> When no UUID has been specified, query-uuid returns
>
> {"UUID": "----"}
>
> The doc comment calls this "a null UUID", which I find less than
> clear. Change it to "an all-zero UUID".
>
Technically it's a "nil UUID";
https://datatracker.ietf.org/doc/html/rfc9562#name-nil-uuid
If you wanted to be pedantic, you could say "the nil UUID (all zeroes) is
returned"
but your rephrasing is clear even w/o using the standard name, so I'm fine
either way.
> Signed-off-by: Markus Armbruster
>
Reviewed-by: John Snow
---
> qapi/machine.json | 5 ++---
> 1 file changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/qapi/machine.json b/qapi/machine.json
> index 50ff102d56..f40427f21a 100644
> --- a/qapi/machine.json
> +++ b/qapi/machine.json
> @@ -305,9 +305,8 @@
> #
> # Since: 0.14
> #
> -# .. note:: If no UUID was specified for the guest, a null UUID is
> -#returned.
> -#
> +# .. note:: If no UUID was specified for the guest, an all-zero UUID
> +#is returned.
> ##
> { 'struct': 'UuidInfo', 'data': {'UUID': 'str'} }
>
> --
> 2.45.0
>
>
Re: [PATCH 2/5] qapi/machine: Clean up documentation around CpuInstanceProperties
On Thu, Jul 11, 2024, 7:22 AM Markus Armbruster wrote:
> CpuInstanceProperties' doc comment describes its members as properties
> to be passed to device_add when hot-plugging a CPU.
>
> This was in fact the initial use of this type, with
> query-hotpluggable-cpus: letting management applications find out what
> properties need to be passed with device_add to hot-plug a CPU.
>
> We've since added other uses: set-numa-node (commit 419fcdec3c1 and
> f3be67812c2), and query-cpus-fast (commit ce74ee3dea6). These are not
> about device-add.
>
> query-hotpluggable-cpus uses CpuInstanceProperties within
> HotpluggableCPU. Lift the documentation related to device-add from
> CpuInstanceProperties to HotpluggableCPU.
>
> Signed-off-by: Markus Armbruster
>
Reviewed-by: John Snow
---
> qapi/machine.json | 13 ++---
> 1 file changed, 6 insertions(+), 7 deletions(-)
>
> diff --git a/qapi/machine.json b/qapi/machine.json
> index f15ad1b43e..50ff102d56 100644
> --- a/qapi/machine.json
> +++ b/qapi/machine.json
> @@ -960,9 +960,7 @@
> ##
> # @CpuInstanceProperties:
> #
> -# List of properties to be used for hotplugging a CPU instance, it
> -# should be passed by management with device_add command when a CPU is
> -# being hotplugged.
> +# Properties identifying a CPU.
> #
> # Which members are optional and which mandatory depends on the
> # architecture and board.
> @@ -996,9 +994,6 @@
> #
> # @thread-id: thread number within the core the CPU belongs to
> #
> -# .. note:: Management should be prepared to pass through additional
> -#properties with device_add.
> -#
> # Since: 2.7
> ##
> { 'struct': 'CpuInstanceProperties',
> @@ -1020,7 +1015,8 @@
> #
> # @type: CPU object type for usage with device_add command
> #
> -# @props: list of properties to be used for hotplugging CPU
> +# @props: list of properties to pass for hotplugging a CPU with
> +# device_add
> #
> # @vcpus-count: number of logical VCPU threads @HotpluggableCPU
> # provides
> @@ -1028,6 +1024,9 @@
> # @qom-path: link to existing CPU object if CPU is present or omitted
> # if CPU is not present.
> #
> +# .. note:: Management should be prepared to pass through additional
> +#properties with device_add.
> +#
> # Since: 2.7
> ##
> { 'struct': 'HotpluggableCPU',
> --
> 2.45.0
>
>
Re: [PATCH 1/5] qapi/pci: Clean up documentation around PciDeviceClass
On Thu, Jul 11, 2024, 7:24 AM Markus Armbruster wrote:
> PciDeviceInfo's doc comment has a note on PciDeviceClass member @desc.
> Since the note applies always, not just within PciDeviceInfo, merge it
> into PciDeviceClass's description of member @desc.
>
> Signed-off-by: Markus Armbruster
>
Reviewed-by: John Snow
---
> qapi/pci.json | 6 ++
> 1 file changed, 2 insertions(+), 4 deletions(-)
>
> diff --git a/qapi/pci.json b/qapi/pci.json
> index 8287d15dd0..97179920fb 100644
> --- a/qapi/pci.json
> +++ b/qapi/pci.json
> @@ -93,7 +93,8 @@
> #
> # Information about the Class of a PCI device
> #
> -# @desc: a string description of the device's class
> +# @desc: a string description of the device's class (not stable, and
> +# should only be treated as informational)
> #
> # @class: the class code of the device
> #
> @@ -146,9 +147,6 @@
> #
> # @regions: a list of the PCI I/O regions associated with the device
> #
> -# .. note:: The contents of @class_info.desc are not stable and should
> -#only be treated as informational.
> -#
> # Since: 0.14
> ##
> { 'struct': 'PciDeviceInfo',
> --
> 2.45.0
>
>
>
[PULL 0/6] Python patches
The following changes since commit 4469bee2c529832d762af4a2f89468c926f02fe4: Merge tag 'nvme-next-pull-request' of https://gitlab.com/birkelund/qemu into staging (2024-07-11 14:32:51 -0700) are available in the Git repository at: https://gitlab.com/jsnow/qemu.git tags/python-pull-request for you to fetch changes up to dd23f9ec519db9c424223cff8767715de5532718: docs: remove Sphinx 1.x compatibility code (2024-07-12 16:46:21 -0400) Python: 3.13 compat & sphinx minver bump ---- John Snow (6): python: linter changes for pylint 3.x python: Do not use pylint 3.2.4 with python 3.8 iotests: Change imports for Python 3.13 python: enable testing for 3.13 Python: bump minimum sphinx version to 3.4.3 docs: remove Sphinx 1.x compatibility code docs/conf.py | 7 ++--- docs/sphinx/hxtool.py | 21 +++--- docs/sphinx/kerneldoc.py | 38 -- docs/sphinx/kernellog.py | 28 --- docs/sphinx/qapidoc.py | 33 ++ python/qemu/machine/machine.py | 1 + python/qemu/utils/qemu_ga_client.py| 2 +- python/setup.cfg | 4 ++- pythondeps.toml| 2 +- tests/docker/dockerfiles/python.docker | 1 + tests/qemu-iotests/testenv.py | 7 - tests/qemu-iotests/testrunner.py | 9 -- 12 files changed, 41 insertions(+), 112 deletions(-) delete mode 100644 docs/sphinx/kernellog.py -- 2.45.0
[PULL 3/6] iotests: Change imports for Python 3.13
Python 3.13 isn't out yet, but it's in beta and Fedora is ramping up to make it the default system interpreter for Fedora 41. They moved our cheese for where ContextManager lives; add a conditional to locate it while we support both pre-3.9 and 3.13+. Signed-off-by: John Snow Message-id: 20240626232230.408004-4-js...@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/testenv.py| 7 ++- tests/qemu-iotests/testrunner.py | 9 ++--- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py index 588f30a4f14..96d69e56963 100644 --- a/tests/qemu-iotests/testenv.py +++ b/tests/qemu-iotests/testenv.py @@ -25,7 +25,12 @@ import random import subprocess import glob -from typing import List, Dict, Any, Optional, ContextManager +from typing import List, Dict, Any, Optional + +if sys.version_info >= (3, 9): +from contextlib import AbstractContextManager as ContextManager +else: +from typing import ContextManager DEF_GDB_OPTIONS = 'localhost:12345' diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py index 7b322272e92..2e236c8fa39 100644 --- a/tests/qemu-iotests/testrunner.py +++ b/tests/qemu-iotests/testrunner.py @@ -27,11 +27,14 @@ import shutil import sys from multiprocessing import Pool -from typing import List, Optional, Any, Sequence, Dict, \ -ContextManager - +from typing import List, Optional, Any, Sequence, Dict from testenv import TestEnv +if sys.version_info >= (3, 9): +from contextlib import AbstractContextManager as ContextManager +else: +from typing import ContextManager + def silent_unlink(path: Path) -> None: try: -- 2.45.0
[PULL 6/6] docs: remove Sphinx 1.x compatibility code
In general, the Use_SSI workaround is no longer needed, and neither is
the pre-1.6 logging shim for kerneldoc.
Signed-off-by: John Snow
Acked-by: Markus Armbruster
Reviewed-by: Thomas Huth
Message-id: 20240703175235.239004-3-js...@redhat.com
[rebased on top of origin/master. --js]
Signed-off-by: John Snow
---
docs/sphinx/hxtool.py| 21 -
docs/sphinx/kerneldoc.py | 38 --
docs/sphinx/kernellog.py | 28
docs/sphinx/qapidoc.py | 33 +++--
4 files changed, 19 insertions(+), 101 deletions(-)
delete mode 100644 docs/sphinx/kernellog.py
diff --git a/docs/sphinx/hxtool.py b/docs/sphinx/hxtool.py
index 3729084a36c..a84723be19e 100644
--- a/docs/sphinx/hxtool.py
+++ b/docs/sphinx/hxtool.py
@@ -24,16 +24,10 @@
from docutils.statemachine import ViewList
from docutils.parsers.rst import directives, Directive
from sphinx.errors import ExtensionError
+from sphinx.util.docutils import switch_source_input
from sphinx.util.nodes import nested_parse_with_titles
import sphinx
-# Sphinx up to 1.6 uses AutodocReporter; 1.7 and later
-# use switch_source_input. Check borrowed from kerneldoc.py.
-Use_SSI = sphinx.__version__[:3] >= '1.7'
-if Use_SSI:
-from sphinx.util.docutils import switch_source_input
-else:
-from sphinx.ext.autodoc import AutodocReporter
__version__ = '1.0'
@@ -185,16 +179,9 @@ def run(self):
# of title_styles and section_level that kerneldoc.py does,
# because nested_parse_with_titles() does that for us.
def do_parse(self, result, node):
-if Use_SSI:
-with switch_source_input(self.state, result):
-nested_parse_with_titles(self.state, result, node)
-else:
-save = self.state.memo.reporter
-self.state.memo.reporter = AutodocReporter(result,
self.state.memo.reporter)
-try:
-nested_parse_with_titles(self.state, result, node)
-finally:
-self.state.memo.reporter = save
+with switch_source_input(self.state, result):
+nested_parse_with_titles(self.state, result, node)
+
def setup(app):
""" Register hxtool-doc directive with Sphinx"""
diff --git a/docs/sphinx/kerneldoc.py b/docs/sphinx/kerneldoc.py
index 72c403a7379..3aa972f2e89 100644
--- a/docs/sphinx/kerneldoc.py
+++ b/docs/sphinx/kerneldoc.py
@@ -38,20 +38,14 @@
from docutils.statemachine import ViewList
from docutils.parsers.rst import directives, Directive
-#
-# AutodocReporter is only good up to Sphinx 1.7
-#
import sphinx
+from sphinx.util import logging
+from sphinx.util.docutils import switch_source_input
-Use_SSI = sphinx.__version__[:3] >= '1.7'
-if Use_SSI:
-from sphinx.util.docutils import switch_source_input
-else:
-from sphinx.ext.autodoc import AutodocReporter
-
-import kernellog
__version__ = '1.0'
+logger = logging.getLogger('kerneldoc')
+
class KernelDocDirective(Directive):
"""Extract kernel-doc comments from the specified file"""
@@ -111,8 +105,7 @@ def run(self):
cmd += [filename]
try:
-kernellog.verbose(env.app,
- 'calling kernel-doc \'%s\'' % (" ".join(cmd)))
+logger.verbose('calling kernel-doc \'%s\'' % (" ".join(cmd)))
p = subprocess.Popen(cmd, stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
out, err = p.communicate()
@@ -122,8 +115,10 @@ def run(self):
if p.returncode != 0:
sys.stderr.write(err)
-kernellog.warn(env.app,
- 'kernel-doc \'%s\' failed with return code %d'
% (" ".join(cmd), p.returncode))
+logger.warning(
+'kernel-doc \'%s\' failed with return code %d' %
+(" ".join(cmd), p.returncode)
+)
return [nodes.error(None, nodes.paragraph(text = "kernel-doc
missing"))]
elif env.config.kerneldoc_verbosity > 0:
sys.stderr.write(err)
@@ -149,22 +144,13 @@ def run(self):
return node.children
except Exception as e: # pylint: disable=W0703
-kernellog.warn(env.app, 'kernel-doc \'%s\' processing failed with:
%s' %
+logger.warning('kernel-doc \'%s\' processing failed with: %s' %
(" ".join(cmd), str(e)))
return [nodes.error(None, nodes.paragraph(text = "kernel-doc
missing"))]
def do_parse(self, result, node):
-if Use_SSI:
-with switch_source_input(self.state, result):
-self.state.nested_parse(result, 0, node, match_titles=1)
-else:
-save = self.state.memo.title_styles,
self.state.memo.section_level, self.state.memo.re
[PULL 4/6] python: enable testing for 3.13
Python 3.13 is in beta and Fedora 41 is preparing to make it the default system interpreter; enable testing for it. (In the event problems develop prior to release, it should only impact the check-python-tox job, which is not run by default and is allowed to fail.) Signed-off-by: John Snow Reviewed-by: Alex Bennée Tested-by: Alex Bennée Message-id: 20240626232230.408004-5-js...@redhat.com Signed-off-by: John Snow --- python/setup.cfg | 3 ++- tests/docker/dockerfiles/python.docker | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/python/setup.cfg b/python/setup.cfg index 8ebd345d7ed..3b4e2cc5501 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -19,6 +19,7 @@ classifiers = Programming Language :: Python :: 3.10 Programming Language :: Python :: 3.11 Programming Language :: Python :: 3.12 +Programming Language :: Python :: 3.13 Typing :: Typed [options] @@ -184,7 +185,7 @@ multi_line_output=3 # of python available on your system to run this test. [tox:tox] -envlist = py38, py39, py310, py311, py312 +envlist = py38, py39, py310, py311, py312, py313 skip_missing_interpreters = true [testenv] diff --git a/tests/docker/dockerfiles/python.docker b/tests/docker/dockerfiles/python.docker index a3c1321190c..8f0af9ef25f 100644 --- a/tests/docker/dockerfiles/python.docker +++ b/tests/docker/dockerfiles/python.docker @@ -14,6 +14,7 @@ ENV PACKAGES \ python3.10 \ python3.11 \ python3.12 \ +python3.13 \ python3.8 \ python3.9 -- 2.45.0
[PULL 5/6] Python: bump minimum sphinx version to 3.4.3
With RHEL 8 support retired (It's been two years since RHEL9 released),
our very oldest build platform version of Sphinx is now 3.4.3; and
keeping backwards compatibility for versions as old as v1.6 when using
domain extensions is a lot of work we don't need to do.
This patch is motivated by my work creating a new QAPI domain, which
unlike the dbus documentation, cannot be allowed to regress by creating
a "dummy" doc when operating under older sphinx versions. Easier is to
raise our minimum version as far as we can push it forwards, reducing my
burden in creating cross-compatibility hacks and patches.
A sampling of sphinx versions from various distributions, courtesy
https://repology.org/project/python:sphinx/versions
Alpine 3.16: v4.3.0 (QEMU support ended 2024-05-23)
Alpine 3.17: v5.3.0
Alpine 3.18: v6.1.3
Alpine 3.19: v6.2.1
Ubuntu 20.04 LTS: EOL
Ubuntu 22.04 LTS: v4.3.2
Ubuntu 22.10: EOL
Ubuntu 23.04: EOL
Ubuntu 23.10: v5.3.0
Ubuntu 24.04 LTS: v7.2.6
Debian 11: v3.4.3 (QEMU support ends 2024-07-xx)
Debian 12: v5.3.0
Fedora 38: EOL
Fedora 39: v6.2.1
Fedora 40: v7.2.6
CentOS Stream 8: v1.7.6 (QEMU support ended 2024-05-17)
CentOS Stream 9: v3.4.3
OpenSUSE Leap 15.4: EOL
OpenSUSE Leap 15.5: 2.3.1, 4.2.0 and 7.2.6
RHEL9 / CentOS Stream 9 becomes the new defining factor in staying at
Sphinx 3.4.3 due to downstream offline build requirements that force us
to use platform Sphinx instead of newer packages from PyPI.
Signed-off-by: John Snow
Acked-by: Paolo Bonzini
Acked-by: Markus Armbruster
Reviewed-by: Thomas Huth
Message-id: 20240703175235.239004-2-js...@redhat.com
Signed-off-by: John Snow
---
docs/conf.py| 7 +++
pythondeps.toml | 2 +-
2 files changed, 4 insertions(+), 5 deletions(-)
diff --git a/docs/conf.py b/docs/conf.py
index aae0304ac6e..876f6768815 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -53,10 +53,9 @@
# If your documentation needs a minimal Sphinx version, state it here.
#
-# Sphinx 1.5 and earlier can't build our docs because they are too
-# picky about the syntax of the argument to the option:: directive
-# (see Sphinx bugs #646, #3366).
-needs_sphinx = '1.6'
+# 3.4.3 is the oldest version of Sphinx that ships on a platform we
+# pledge build support for.
+needs_sphinx = '3.4.3'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
diff --git a/pythondeps.toml b/pythondeps.toml
index 6aba0c9daaa..f6e590fdd86 100644
--- a/pythondeps.toml
+++ b/pythondeps.toml
@@ -23,7 +23,7 @@ meson = { accepted = ">=1.1.0", installed = "1.2.3", canary =
"meson" }
[docs]
# Please keep the installed versions in sync with docs/requirements.txt
-sphinx = { accepted = ">=1.6", installed = "5.3.0", canary = "sphinx-build" }
+sphinx = { accepted = ">=3.4.3", installed = "5.3.0", canary = "sphinx-build" }
sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.1.1" }
[avocado]
--
2.45.0
[PULL 1/6] python: linter changes for pylint 3.x
New bleeding edge versions, new nits to iron out. This addresses the 'check-python-tox' optional GitLab test, while 'check-python-minreqs' saw no regressions, since it's frozen on an older version of pylint. Fixes: qemu/machine/machine.py:345:52: E0606: Possibly using variable 'sock' before assignment (possibly-used-before-assignment) qemu/utils/qemu_ga_client.py:168:4: R1711: Useless return at end of function or method (useless-return) Signed-off-by: John Snow Reviewed-by: Alex Bennée Message-id: 20240626232230.408004-2-js...@redhat.com Signed-off-by: John Snow --- python/qemu/machine/machine.py | 1 + python/qemu/utils/qemu_ga_client.py | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index f648f6af451..ebb58d5b68c 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -335,6 +335,7 @@ def binary(self) -> str: def _pre_launch(self) -> None: if self._qmp_set: +sock = None if self._monitor_address is None: self._sock_pair = socket.socketpair() os.set_inheritable(self._sock_pair[0].fileno(), True) diff --git a/python/qemu/utils/qemu_ga_client.py b/python/qemu/utils/qemu_ga_client.py index 9a665e6e990..cf0fcf9a8bb 100644 --- a/python/qemu/utils/qemu_ga_client.py +++ b/python/qemu/utils/qemu_ga_client.py @@ -174,7 +174,7 @@ def suspend(self, mode: str) -> None: # On error exception will raise except asyncio.TimeoutError: # On success command will timed out -return +pass def shutdown(self, mode: str = 'powerdown') -> None: if mode not in ['powerdown', 'halt', 'reboot']: -- 2.45.0
[PULL 2/6] python: Do not use pylint 3.2.4 with python 3.8
There is a bug in this version, see: https://github.com/pylint-dev/pylint/issues/9751 Signed-off-by: John Snow Reviewed-by: Alex Bennée Message-id: 20240626232230.408004-3-js...@redhat.com Signed-off-by: John Snow --- python/setup.cfg | 1 + 1 file changed, 1 insertion(+) diff --git a/python/setup.cfg b/python/setup.cfg index 48668609d3e..8ebd345d7ed 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -41,6 +41,7 @@ devel = isort >= 5.1.2 mypy >= 1.4.0 pylint >= 2.17.3 +pylint != 3.2.4; python_version<"3.9" tox >= 3.18.0 urwid >= 2.1.2 urwid-readline >= 0.13 -- 2.45.0
Re: [PATCH 8/8] qapi: remove "Example" doc section
On Wed, Jul 10, 2024 at 1:45 AM Markus Armbruster wrote:
> John Snow writes:
>
> > On Tue, Jul 9, 2024 at 6:52 AM Markus Armbruster
> wrote:
> >
> >> John Snow writes:
> >>
> >> > Fully eliminate the "Example" sections in QAPI doc blocks now that
> they
> >> > have all been converted to arbitrary rST syntax using the
> >> > ".. qmp-example::" directive. Update tests to match.
> >> >
> >> > Migrating to the new syntax
> >> > ---
> >> >
> >> > The old "Example:" or "Examples:" section syntax is now caught as an
> >> > error, but "Example::" is stil permitted as explicit rST syntax for an
> >> > un-lexed, generic preformatted text block.
> >> >
> >> > ('Example' is not special in this case, any sentence that ends with
> "::"
> >> > will start an indented code block in rST.)
> >> >
> >> > Arbitrary rST for Examples is now possible, but it's strongly
> >> > recommended that documentation authors use the ".. qmp-example::"
> >> > directive for consistent visual formatting in rendered HTML docs. The
> >> > ":title:" directive option may be used to add extra information into
> the
> >> > title bar for the example. The ":annotated:" option can be used to
> write
> >> > arbitrary rST instead, with nested "::" blocks applying QMP formatting
> >> > where desired.
> >> >
> >> > Other choices available are ".. code-block:: QMP" which will not
> create
> >> > an "Example:" box, or the short-form "::" code-block syntax which will
> >> > not apply QMP highlighting when used outside of the qmp-example
> >> > directive.
> >> >
> >> > Why?
> >> >
> >> >
> >> > This patch has several benefits:
> >> >
> >> > 1. Example sections can now be written more arbitrarily, mixing
> >> >explanatory paragraphs and code blocks however desired.
> >> >
> >> > 2. Example sections can now use fully arbitrary rST.
> >> >
> >> > 3. All code blocks are now lexed and validated as QMP; increasing
> >> >usability of the docs and ensuring validity of example snippets.
> >> >
> >> >(To some extent - This patch only gaurantees it lexes correctly,
> not
> >> >that it's valid under the JSON or QMP grammars. It will catch most
> >> >small mistakes, however.)
> >> >
> >> > 4. Each qmp-example can be titled or annotated independently without
> >> >bypassing the QMP lexer/validator.
> >> >
> >> >(i.e. code blocks are now for *code* only, so we don't have to
> >> >sacrifice exposition for having lexically valid examples.)
> >> >
> >> > NOTE: As with the "Notes" conversion patch,
> >>
> >> Commit d461c279737 (qapi: convert "Note" sections to plain rST).
> >>
> >
> > Didn't have a stable commit ID at the time, will work it in if/when the
> > notes patches hit main.
>
> They have.
>
> >> > this patch (and those
> >> > preceding) may change the rendering order for Examples in the
> >>
> >> The three preceding ones, to be precise.
> >>
> >> > current generator. The forthcoming qapidoc rewrite will fix this
> >> > by always generating documentation in source order.
> >>
> >> Conversions from "Example" section to plain reST may change order. This
> >> patch converts a test, and the preceding three convert the real uses.
> >>
> >> Does any of the patches actually change order?
> >
> > I do not actually know ...! It has the *potential* in the same exact way
> > that the notes patch did, but I don't actually know if it *did*. My hunch
> > is "no" because there's only one intermediate section we identified with
> > the notes series, but I didn't exhaustively prove it. That's why I used
> the
> > "may" weasel wording.
>
> Alright, I checked.
>
> In documentation of command announce-self, the example moves from after
> the arguments to before. Unwanted change.
>
> I can keep it in place if I insert a TODO before the example like this:
>
> diff --git a/qapi/net
Re: [PATCH 8/8] qapi: remove "Example" doc section
On Tue, Jul 9, 2024 at 6:52 AM Markus Armbruster wrote:
> John Snow writes:
>
> > Fully eliminate the "Example" sections in QAPI doc blocks now that they
> > have all been converted to arbitrary rST syntax using the
> > ".. qmp-example::" directive. Update tests to match.
> >
> > Migrating to the new syntax
> > ---
> >
> > The old "Example:" or "Examples:" section syntax is now caught as an
> > error, but "Example::" is stil permitted as explicit rST syntax for an
> > un-lexed, generic preformatted text block.
> >
> > ('Example' is not special in this case, any sentence that ends with "::"
> > will start an indented code block in rST.)
> >
> > Arbitrary rST for Examples is now possible, but it's strongly
> > recommended that documentation authors use the ".. qmp-example::"
> > directive for consistent visual formatting in rendered HTML docs. The
> > ":title:" directive option may be used to add extra information into the
> > title bar for the example. The ":annotated:" option can be used to write
> > arbitrary rST instead, with nested "::" blocks applying QMP formatting
> > where desired.
> >
> > Other choices available are ".. code-block:: QMP" which will not create
> > an "Example:" box, or the short-form "::" code-block syntax which will
> > not apply QMP highlighting when used outside of the qmp-example
> > directive.
> >
> > Why?
> >
> >
> > This patch has several benefits:
> >
> > 1. Example sections can now be written more arbitrarily, mixing
> >explanatory paragraphs and code blocks however desired.
> >
> > 2. Example sections can now use fully arbitrary rST.
> >
> > 3. All code blocks are now lexed and validated as QMP; increasing
> >usability of the docs and ensuring validity of example snippets.
> >
> >(To some extent - This patch only gaurantees it lexes correctly, not
> >that it's valid under the JSON or QMP grammars. It will catch most
> >small mistakes, however.)
> >
> > 4. Each qmp-example can be titled or annotated independently without
> >bypassing the QMP lexer/validator.
> >
> >(i.e. code blocks are now for *code* only, so we don't have to
> >sacrifice exposition for having lexically valid examples.)
> >
> > NOTE: As with the "Notes" conversion patch,
>
> Commit d461c279737 (qapi: convert "Note" sections to plain rST).
>
Didn't have a stable commit ID at the time, will work it in if/when the
notes patches hit main.
>
> > this patch (and those
> > preceding) may change the rendering order for Examples in the
>
> The three preceding ones, to be precise.
>
> > current generator. The forthcoming qapidoc rewrite will fix this
> > by always generating documentation in source order.
>
> Conversions from "Example" section to plain reST may change order. This
> patch converts a test, and the preceding three convert the real uses.
>
> Does any of the patches actually change order?
>
I do not actually know ...! It has the *potential* in the same exact way
that the notes patch did, but I don't actually know if it *did*. My hunch
is "no" because there's only one intermediate section we identified with
the notes series, but I didn't exhaustively prove it. That's why I used the
"may" weasel wording.
>
> > Signed-off-by: John Snow
> > ---
> > docs/devel/qapi-code-gen.rst| 58 -
> > scripts/qapi/parser.py | 10 +-
> > tests/qapi-schema/doc-good.json | 19 +++
> > tests/qapi-schema/doc-good.out | 26 ++-
> > tests/qapi-schema/doc-good.txt | 23 ++---
> > 5 files changed, 98 insertions(+), 38 deletions(-)
> >
> > diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
> > index ae97b335cbf..2e10a3cbd69 100644
> > --- a/docs/devel/qapi-code-gen.rst
> > +++ b/docs/devel/qapi-code-gen.rst
> > @@ -899,7 +899,7 @@ Documentation markup
> >
> >
> > Documentation comments can use most rST markup. In particular,
> > -a ``::`` literal block can be used for examples::
> > +a ``::`` literal block can be used for pre-formatted text::
> >
> > # ::
> > #
> > @@ -995,8 +995,8 @@ line "Features:", like this::
> ># @feature: Description text
> >
&g
Re: [PATCH 7/8] qapi: convert "Example" sections with longer prose
On Tue, Jul 9, 2024 at 7:35 AM Markus Armbruster wrote:
> John Snow writes:
>
> > These examples require longer explanations or have explanations that
> > require markup to look reasonable when rendered and so use the longer
> > form of the ".. qmp-example::" directive.
> >
> > By using the :annotated: option, the content in the example block is
> > assumed *not* to be a code block literal and is instead parsed as normal
> > rST - with the exception that any code literal blocks after `::` will
> > assumed to be a QMP code literal block.
> >
> > Note: There's one title-less conversion in this patch that comes along
> > for the ride because it's part of a larger "Examples" block that was
> > better to convert all at once.
> >
> > See commit-5: "docs/qapidoc: create qmp-example directive", for a
> > detailed explanation of this custom directive syntax.
> >
> > See commit+1: "qapi: remove "Example" doc section" for a detailed
> > explanation of why.
> >
> > Signed-off-by: John Snow
> > ---
> > qapi/block.json | 26 --
> > qapi/machine.json | 30 --
> > qapi/migration.json | 7 +--
> > qapi/virtio.json| 24 ++--
> > 4 files changed, 59 insertions(+), 28 deletions(-)
> >
> > diff --git a/qapi/block.json b/qapi/block.json
> > index 5ddd061e964..d95e9fd8140 100644
> > --- a/qapi/block.json
> > +++ b/qapi/block.json
> > @@ -545,31 +545,37 @@
> > #
> > # Since: 4.0
> > #
> > -# Example:
> > +# .. qmp-example::
> > +#:annotated:
> > #
> > -# Set new histograms for all io types with intervals
> > -# [0, 10), [10, 50), [50, 100), [100, +inf):
> > +#Set new histograms for all io types with intervals
> > +#[0, 10), [10, 50), [50, 100), [100, +inf)::
> > #
> > # -> { "execute": "block-latency-histogram-set",
> > # "arguments": { "id": "drive0",
> > # "boundaries": [10, 50, 100] } }
> > # <- { "return": {} }
> > #
> > -# Example:
> > +# .. qmp-example::
> > +#:annotated:
> > #
> > -# Set new histogram only for write, other histograms will remain
> > -# not changed (or not created):
> > +#Set new histogram only for write, other histograms will remain
> > +#not changed (or not created)::
> > #
> > # -> { "execute": "block-latency-histogram-set",
> > # "arguments": { "id": "drive0",
> > # "boundaries-write": [10, 50, 100] } }
> > # <- { "return": {} }
> > #
> > -# Example:
> > +# .. qmp-example::
> > +#:annotated:
> > #
> > -# Set new histograms with the following intervals:
> > -# read, flush: [0, 10), [10, 50), [50, 100), [100, +inf)
> > -# write: [0, 1000), [1000, 5000), [5000, +inf)
> > +#Set new histograms with the following intervals:
> > +#
> > +#- read, flush: [0, 10), [10, 50), [50, 100), [100, +inf)
> > +#- write: [0, 1000), [1000, 5000), [5000, +inf)
> > +#
> > +#::
> > #
> > # -> { "execute": "block-latency-histogram-set",
> > # "arguments": { "id": "drive0",
># "boundaries": [10, 50, 100],
># "boundaries-write": [1000, 5000] } }
># <- { "return": {} }
>#
># .. qmp-example::
>#:title: Remove all latency histograms
>#
># -> { "execute": "block-latency-histogram-set",
># "arguments": { "id": "drive0" } }
># <- { "return": {} }
>##
>
> I think using :annotated: for this one as well will look better.
>
Sure, why not. I tried to minimize more complex rewrites as a rule, but
it's no problem to change the styling to taste.
>
> [...]
>
> Reviewed-by: Markus Armbruster
>
>
Re: [PATCH 4/8] docs/sphinx: add CSS styling for qmp-example directive
On Tue, Jul 9, 2024 at 6:34 AM Markus Armbruster wrote: > John Snow writes: > > > From: Harmonie Snow > > > > Add CSS styling for qmp-example directives to increase readability and > > consistently style all example blocks. > > > > Signed-off-by: Harmonie Snow > > Signed-off-by: John Snow > > Same sadness as for the previous patch. > Should we do anything about that? In the long run, I don't expect anyone will actually need to care about what this directive looked like in some intermediate state before we ever used it. If you want to evaluate the directive in the in-between states, I recommend modifying a document and seeing what it does; but I didn't really intend for anyone to really see it that way. (Isn't it a bit overboard to write unit tests for intermediate tree states...?) > > Acked-by: Markus Armbruster > >
Re: [PATCH 3/8] docs/qapidoc: add QMP highlighting to annotated qmp-example blocks
On Tue, Jul 9, 2024 at 6:33 AM Markus Armbruster wrote:
> John Snow writes:
>
> > For any code literal blocks inside of a qmp-example directive, apply and
> > enforce the QMP lexer/highlighter to those blocks.
> >
> > This way, you won't need to write:
> >
> > ```
> > .. qmp-example::
> >:annotated:
> >
> >Blah blah
> >
> >.. code-block:: QMP
> >
> > -> { "lorem": "ipsum" }
> > ```
> >
> > But instead, simply:
> >
> > ```
> > .. qmp-example::
> >:annotated:
> >
> >Blah blah::
> >
> > -> { "lorem": "ipsum" }
> > ```
> >
> > Once the directive block is exited, whatever the previous default
> > highlight language was will be restored; localizing the forced QMP
> > lexing to exclusively this directive.
> >
> > Note, if the default language is *already* QMP, this directive will not
> > generate and restore redundant highlight configuration nodes. We may
> > well decide that the default language ought to be QMP for any QAPI
> > reference pages, but this way the directive behaves consistently no
> > matter where it is used.
> >
> > Signed-off-by: John Snow
>
> Sadly, the previous patch didn't add tests, so this patch's effect
> isn't as easy to observe as it could be.
>
Sorry O:-)
In truth, I never intended it to *not* have this feature, but thought it
was helpful to split out just the code responsible for this feature into
its own patch so its soul could be independently judged to see if it was
lighter than a feather.
(i.e., does the convenience justify the SLOC?)
>
> Acked-by: Markus Armbruster
>
Re: [PATCH] hw/nvme: Add support for setting the MQES for the NVMe emulation
On Wed, 2024-05-01 at 14:35 +0200, Klaus Jensen wrote:
> On May 1 12:27, Berg, John wrote:
> > On Thu, 2024-04-04 at 15:01 +0200, Klaus Jensen wrote:
> > > On Apr 4 13:04, John Berg wrote:
> > > > From: John Berg
> > > >
> > > > The MQES field in the CAP register describes the Maximum Queue
> > > > Entries
> > > > Supported for the IO queues of an NVMe controller. Adding a +1
> > > > to
> > > > the
> > > > value in this field results in the total queue size. A full
> > > > queue
> > > > is
> > > > when a queue of size N contains N - 1 entries, and the minimum
> > > > queue
> > > > size is 2. Thus the lowest MQES value is 1.
> > > >
> > > > This patch adds the new mqes property to the NVMe emulation
> > > > which
> > > > allows
> > > > a user to specify the maximum queue size by setting this
> > > > property.
> > > > This
> > > > is useful as it enables testing of NVMe controller where the
> > > > MQES
> > > > is
> > > > relatively small. The smallest NVMe queue size supported in
> > > > NVMe is
> > > > 2
> > > > submission and completion entries, which means that the
> > > > smallest
> > > > legal
> > > > mqes value is 1.
> > > >
> > > > The following example shows how the mqes can be set for a the
> > > > NVMe
> > > > emulation:
> > > >
> > > > -drive id=nvme0,if=none,file=nvme.img,format=raw
> > > > -device nvme,drive=nvme0,serial=foo,mqes=1
> > > >
> > > > If the mqes property is not provided then the default mqes will
> > > > still be
> > > > 0x7ff (the queue size is 2048 entries).
> > > >
> > > > Signed-off-by: John Berg
> > > > ---
> > > > hw/nvme/ctrl.c | 9 -
> > > > hw/nvme/nvme.h | 1 +
> > > > 2 files changed, 9 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c
> > > > index 127c3d2383..86cda9bc73 100644
> > > > --- a/hw/nvme/ctrl.c
> > > > +++ b/hw/nvme/ctrl.c
> > > > @@ -7805,6 +7805,12 @@ static bool nvme_check_params(NvmeCtrl
> > > > *n,
> > > > Error **errp)
> > > > return false;
> > > > }
> > > >
> > > > + if (params->mqes < 1)
> > > > + {
> > >
> > > Please keep the `{` on the same line as the `if`. I think
> > > checkpatch.pl
> > > should catch this.
> > >
> > > No need to send a v2, I'll fix it up when I apply it to nvme-next
> > > :)
> > >
> > > Thanks!
> >
> >
> > Hello
> >
> > Sorry for chasing. I was just wondering when this patch will be
> > applied. I can send a second revision if that helps.
> >
>
> No need for the sorry. My apologies. It feel off my radar, so thanks
> for
> the bump.
>
> I've queued this on nvme-next, will send a pull for master ASAP.
Hello, just checking in again. Will this be merged into master soon?
Re: [PATCH 1/8] docs/qapidoc: factor out do_parse()
On Sat, Jul 6, 2024, 10:47 AM Markus Armbruster wrote: > John Snow writes: > > > Factor out the compatibility parser helper into a base class, so it can > > be shared by other directives. > > > > Signed-off-by: John Snow > > R-by stands. > Assuming true even if I rebase on top of the 3.x patches and do_parse() becomes quite a bit more trivial? --js >
Re: [PATCH 5/8] qapi: convert "Example" sections without titles
On Sat, Jul 6, 2024, 10:42 AM Markus Armbruster wrote:
> John Snow writes:
>
> > Use the no-option form of ".. qmp-example::" to convert any Examples
> > that do not have any form of caption or explanation whatsoever. Note
> > that in a few cases, example sections are split into two or more
> > separate example blocks. This is only done stylistically to create a
> > delineation between two or more logically independent examples.
> >
> > See commit-3: "docs/qapidoc: create qmp-example directive", for a
> > detailed explanation of this custom directive syntax.
> >
> > See commit+3: "qapi: remove "Example" doc section" for a detailed
> > explanation of why.
> >
> > Signed-off-by: John Snow
>
> [...]
>
> > diff --git a/qapi/run-state.json b/qapi/run-state.json
> > index 252d7d6afa7..718a3c958e9 100644
> > --- a/qapi/run-state.json
> > +++ b/qapi/run-state.json
>
> [...]
>
> > @@ -453,7 +453,7 @@
> > #
> > # Since: 5.0
> > #
> > -# Example:
> > +# .. qmp-example::
> > #
> > # <- { "event": "GUEST_CRASHLOADED",
> > # "data": { "action": "run" },
>
> Trivial semantic conflict, we need
>
Caught on rebase late Fri, already fixed locally and will be in v2 (which I
rebased on top of my sphinx 3.x patches, which change the do_parse() stuff
too.)
> @@ -469,7 +469,7 @@
>#
># Since: 9.1
>#
> -# Example:
> +# .. qmp-example::
>#
># <- { "event": "GUEST_PVSHUTDOWN",
># "timestamp": { "seconds": 1648245259, "microseconds": 893771
> } }
>
>
> > @@ -597,7 +597,7 @@
> > #
> > # Since: 5.2
> > #
> > -# Example:
> > +# .. qmp-example::
> > #
> > # <- { "event": "MEMORY_FAILURE",
> > # "data": { "recipient": "hypervisor",
>
> [...]
>
>
Re: [PATCH 0/4] Python: Add 3.13 support, play linter whackamole
On Wed, Jun 26, 2024, 7:22 PM John Snow wrote: > Fix some regressions in check-python-tox that have crept in since Pylint > 3.x, and add Python 3.13 support to the pipeline. > > GitLab pipeline (before I fixed the missing DCO, but let's be honest, it > can't possibly be worth re-running so many tests for just that): > https://gitlab.com/jsnow/qemu/-/pipelines/1349737188 > > John Snow (4): > python: linter changes for pylint 3.x > python: Do not use pylint 3.2.4 with python 3.8 > iotests: Change imports for Python 3.13 > python: enable testing for 3.13 > > python/qemu/machine/machine.py | 1 + > python/qemu/utils/qemu_ga_client.py| 2 +- > python/setup.cfg | 4 +++- > tests/docker/dockerfiles/python.docker | 1 + > tests/qemu-iotests/testenv.py | 7 ++- > tests/qemu-iotests/testrunner.py | 9 ++--- > 6 files changed, 18 insertions(+), 6 deletions(-) > > -- > 2.45.0 > Staging under my Python branch, since I need to get tests green to roll forward with some more substantial changes. --js >
Re: [PATCH v2 0/2] docs/python: bump minimum Sphinx version
On Wed, Jul 3, 2024, 1:52 PM John Snow wrote: > With recent deprecations, we can advance our minimum sphinx version > safely. This is heavily motivated by new qapidoc work which is much > easier to maintain cross-version compatibility for - see difficulties in > our dbus documentation which only works on sphinx >= 4. > > We can only guarantee >= 3.4.3 now, but that's still vastly easier than > maintaining compatibility all the way back to 1.x. > > https://gitlab.com/jsnow/qemu/-/pipelines/1359549621 > > (failures appear to be unrelated to the series.) > > V2: refactor all code deletions into patch 2 > > John Snow (2): > Python: bump minimum sphinx version to 3.4.3 > docs: remove Sphinx 1.x compatibility code > > docs/conf.py | 7 +++ > docs/sphinx/hxtool.py| 21 - > docs/sphinx/kerneldoc.py | 38 -- > docs/sphinx/kernellog.py | 28 > docs/sphinx/qapidoc.py | 29 +++-- > pythondeps.toml | 2 +- > 6 files changed, 23 insertions(+), 102 deletions(-) > delete mode 100644 docs/sphinx/kernellog.py > > -- > 2.45.0 > Going to assume It's okay for me to stage this one under my Python tree. Markus: This will create a minor conflict with the notes patches (the delint patch), but the resolution is straightforward. --js
[PATCH 7/8] qapi: convert "Example" sections with longer prose
These examples require longer explanations or have explanations that
require markup to look reasonable when rendered and so use the longer
form of the ".. qmp-example::" directive.
By using the :annotated: option, the content in the example block is
assumed *not* to be a code block literal and is instead parsed as normal
rST - with the exception that any code literal blocks after `::` will
assumed to be a QMP code literal block.
Note: There's one title-less conversion in this patch that comes along
for the ride because it's part of a larger "Examples" block that was
better to convert all at once.
See commit-5: "docs/qapidoc: create qmp-example directive", for a
detailed explanation of this custom directive syntax.
See commit+1: "qapi: remove "Example" doc section" for a detailed
explanation of why.
Signed-off-by: John Snow
---
qapi/block.json | 26 --
qapi/machine.json | 30 --
qapi/migration.json | 7 +--
qapi/virtio.json| 24 ++--
4 files changed, 59 insertions(+), 28 deletions(-)
diff --git a/qapi/block.json b/qapi/block.json
index 5ddd061e964..d95e9fd8140 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -545,31 +545,37 @@
#
# Since: 4.0
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# Set new histograms for all io types with intervals
-# [0, 10), [10, 50), [50, 100), [100, +inf):
+#Set new histograms for all io types with intervals
+#[0, 10), [10, 50), [50, 100), [100, +inf)::
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0",
# "boundaries": [10, 50, 100] } }
# <- { "return": {} }
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# Set new histogram only for write, other histograms will remain
-# not changed (or not created):
+#Set new histogram only for write, other histograms will remain
+#not changed (or not created)::
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0",
# "boundaries-write": [10, 50, 100] } }
# <- { "return": {} }
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# Set new histograms with the following intervals:
-# read, flush: [0, 10), [10, 50), [50, 100), [100, +inf)
-# write: [0, 1000), [1000, 5000), [5000, +inf)
+#Set new histograms with the following intervals:
+#
+#- read, flush: [0, 10), [10, 50), [50, 100), [100, +inf)
+#- write: [0, 1000), [1000, 5000), [5000, +inf)
+#
+#::
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0",
diff --git a/qapi/machine.json b/qapi/machine.json
index 83f60b319c7..0a5ffe652b7 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -1047,10 +1047,11 @@
#
# Since: 2.7
#
-# Examples:
+# .. qmp-example::
+#:annotated:
#
-# For pseries machine type started with -smp 2,cores=2,maxcpus=4
-# -cpu POWER8:
+#For pseries machine type started with
+#``-smp 2,cores=2,maxcpus=4 -cpu POWER8``::
#
# -> { "execute": "query-hotpluggable-cpus" }
# <- {"return": [
@@ -1060,7 +1061,10 @@
#"vcpus-count": 1, "qom-path": "/machine/unattached/device[0]"}
#]}
#
-# For pc machine type started with -smp 1,maxcpus=2:
+# .. qmp-example::
+#:annotated:
+#
+#For pc machine type started with ``-smp 1,maxcpus=2``::
#
# -> { "execute": "query-hotpluggable-cpus" }
# <- {"return": [
@@ -1075,8 +1079,11 @@
# }
#]}
#
-# For s390x-virtio-ccw machine type started with -smp 1,maxcpus=2
-# -cpu qemu (Since: 2.11):
+# .. qmp-example::
+#:annotated:
+#
+#For s390x-virtio-ccw machine type started with
+#``-smp 1,maxcpus=2 -cpu qemu`` (Since: 2.11)::
#
# -> { "execute": "query-hotpluggable-cpus" }
# <- {"return": [
@@ -1130,12 +1137,15 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# -> { "execute": "balloon", "arguments": { "value": 536870912 } }
-# <- { "return": {} }
+#::
#
-# With a 2.5GiB guest this command inflated the ballon to 3GiB.
+# -> { "execute": "balloon", "arguments": { "value": 536870912 } }
+# <- { "return": {} }
+#
+#With a 2.5GiB guest this command inflated the ballon to 3GiB.
##
{ 'command': 'balloon', 'data': {'value': 'int'} }
diff --git a/qapi/migration.js
[PATCH 3/8] docs/qapidoc: add QMP highlighting to annotated qmp-example blocks
For any code literal blocks inside of a qmp-example directive, apply and
enforce the QMP lexer/highlighter to those blocks.
This way, you won't need to write:
```
.. qmp-example::
:annotated:
Blah blah
.. code-block:: QMP
-> { "lorem": "ipsum" }
```
But instead, simply:
```
.. qmp-example::
:annotated:
Blah blah::
-> { "lorem": "ipsum" }
```
Once the directive block is exited, whatever the previous default
highlight language was will be restored; localizing the forced QMP
lexing to exclusively this directive.
Note, if the default language is *already* QMP, this directive will not
generate and restore redundant highlight configuration nodes. We may
well decide that the default language ought to be QMP for any QAPI
reference pages, but this way the directive behaves consistently no
matter where it is used.
Signed-off-by: John Snow
---
docs/sphinx/qapidoc.py | 53 ++
1 file changed, 49 insertions(+), 4 deletions(-)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index b0f3917dc5b..fb2b23698a0 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -26,6 +26,7 @@
import os
import re
+import sys
import textwrap
from typing import List
@@ -37,6 +38,7 @@
from qapi.schema import QAPISchema
import sphinx
+from sphinx import addnodes
from sphinx.directives.code import CodeBlock
from sphinx.errors import ExtensionError
from sphinx.util.nodes import nested_parse_with_titles
@@ -574,10 +576,10 @@ class QMPExample(CodeBlock, NestedDirective):
Custom admonition for QMP code examples.
When the :annotated: option is present, the body of this directive
-is parsed as normal rST instead. Code blocks must be explicitly
-written by the user, but this allows for intermingling explanatory
-paragraphs with arbitrary rST syntax and code blocks for more
-involved examples.
+is parsed as normal rST, but with any '::' code blocks set to use
+the QMP lexer. Code blocks must be explicitly written by the user,
+but this allows for intermingling explanatory paragraphs with
+arbitrary rST syntax and code blocks for more involved examples.
When :annotated: is absent, the directive body is treated as a
simple standalone QMP code block literal.
@@ -591,6 +593,33 @@ class QMPExample(CodeBlock, NestedDirective):
"title": directives.unchanged,
}
+def _highlightlang(self) -> addnodes.highlightlang:
+"""Return the current highlightlang setting for the document"""
+node = None
+doc = self.state.document
+
+if hasattr(doc, "findall"):
+# docutils >= 0.18.1
+for node in doc.findall(addnodes.highlightlang):
+pass
+else:
+for elem in doc.traverse():
+if isinstance(elem, addnodes.highlightlang):
+node = elem
+
+if node:
+return node
+
+# No explicit directive found, use defaults
+node = addnodes.highlightlang(
+lang=self.env.config.highlight_language,
+force=False,
+# Yes, Sphinx uses this value to effectively disable line
+# numbers and not 0 or None or -1 or something. ¯\_(ツ)_/¯
+linenothreshold=sys.maxsize,
+)
+return node
+
def admonition_wrap(self, *content) -> List[nodes.Node]:
title = "Example:"
if "title" in self.options:
@@ -605,8 +634,24 @@ def admonition_wrap(self, *content) -> List[nodes.Node]:
return [admon]
def run_annotated(self) -> List[nodes.Node]:
+lang_node = self._highlightlang()
+
content_node: nodes.Element = nodes.section()
+
+# Configure QMP highlighting for "::" blocks, if needed
+if lang_node["lang"] != "QMP":
+content_node += addnodes.highlightlang(
+lang="QMP",
+force=False, # "True" ignores lexing errors
+linenothreshold=lang_node["linenothreshold"],
+)
+
self.do_parse(self.content, content_node)
+
+# Restore prior language highlighting, if needed
+if lang_node["lang"] != "QMP":
+content_node += addnodes.highlightlang(**lang_node.attributes)
+
return content_node.children
def run(self) -> List[nodes.Node]:
--
2.45.0
[PATCH 4/8] docs/sphinx: add CSS styling for qmp-example directive
From: Harmonie Snow
Add CSS styling for qmp-example directives to increase readability and
consistently style all example blocks.
Signed-off-by: Harmonie Snow
Signed-off-by: John Snow
---
docs/sphinx-static/theme_overrides.css | 49 ++
1 file changed, 49 insertions(+)
diff --git a/docs/sphinx-static/theme_overrides.css
b/docs/sphinx-static/theme_overrides.css
index c70ef951286..965ecac54fd 100644
--- a/docs/sphinx-static/theme_overrides.css
+++ b/docs/sphinx-static/theme_overrides.css
@@ -87,6 +87,55 @@ div[class^="highlight"] pre {
padding-bottom: 1px;
}
+/* qmp-example directive styling */
+
+.rst-content .admonition-example {
+/* do not apply the standard admonition background */
+background-color: transparent;
+border: solid #ffd2ed 1px;
+}
+
+.rst-content .admonition-example > .admonition-title:before {
+content: "▷";
+}
+
+.rst-content .admonition-example > .admonition-title {
+background-color: #5980a6;
+}
+
+.rst-content .admonition-example > div[class^="highlight"] {
+/* make code boxes take up the full width of the admonition w/o margin */
+margin-left: -12px;
+margin-right: -12px;
+
+border-top: 1px solid #ffd2ed;
+border-bottom: 1px solid #ffd2ed;
+border-left: 0px;
+border-right: 0px;
+}
+
+.rst-content .admonition-example > div[class^="highlight"]:nth-child(2) {
+/* If a code box is the second element in an example admonition,
+ * it is the first child after the title. let it sit flush against
+ * the title. */
+margin-top: -12px;
+border-top: 0px;
+}
+
+.rst-content .admonition-example > div[class^="highlight"]:last-child {
+/* If a code box is the final element in an example admonition, don't
+ * render margin below it; let it sit flush with the end of the
+ * admonition box */
+margin-bottom: -12px;
+border-bottom: 0px;
+}
+
+.rst-content .admonition-example .highlight {
+background-color: #fffafd;
+}
+
+/* end qmp-example styling */
+
@media screen {
/* content column
--
2.45.0
[PATCH 5/8] qapi: convert "Example" sections without titles
Use the no-option form of ".. qmp-example::" to convert any Examples
that do not have any form of caption or explanation whatsoever. Note
that in a few cases, example sections are split into two or more
separate example blocks. This is only done stylistically to create a
delineation between two or more logically independent examples.
See commit-3: "docs/qapidoc: create qmp-example directive", for a
detailed explanation of this custom directive syntax.
See commit+3: "qapi: remove "Example" doc section" for a detailed
explanation of why.
Signed-off-by: John Snow
---
qapi/acpi.json | 4 +--
qapi/block-core.json | 64 +---
qapi/block.json | 18 ++-
qapi/char.json | 24 +--
qapi/control.json| 8 ++---
qapi/dump.json | 8 ++---
qapi/machine-target.json | 2 +-
qapi/machine.json| 38
qapi/migration.json | 58 ++--
qapi/misc-target.json| 22 +++---
qapi/misc.json | 32 ++--
qapi/net.json| 20 +++--
qapi/pci.json| 2 +-
qapi/qdev.json | 10 ---
qapi/qom.json| 8 ++---
qapi/replay.json | 8 ++---
qapi/rocker.json | 8 ++---
qapi/run-state.json | 30 +--
qapi/tpm.json| 6 ++--
qapi/trace.json | 4 +--
qapi/transaction.json| 2 +-
qapi/ui.json | 34 ++---
qapi/vfio.json | 2 +-
qapi/virtio.json | 2 +-
qapi/yank.json | 4 +--
25 files changed, 216 insertions(+), 202 deletions(-)
diff --git a/qapi/acpi.json b/qapi/acpi.json
index aa4dbe57943..045dab6228b 100644
--- a/qapi/acpi.json
+++ b/qapi/acpi.json
@@ -111,7 +111,7 @@
#
# Since: 2.1
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-acpi-ospm-status" }
# <- { "return": [ { "device": "d1", "slot": "0", "slot-type": "DIMM",
"source": 1, "status": 0},
@@ -131,7 +131,7 @@
#
# Since: 2.1
#
-# Example:
+# .. qmp-example::
#
# <- { "event": "ACPI_DEVICE_OST",
# "data": { "info": { "device": "d1", "slot": "0",
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 9ef23ec02ae..4e0f0395146 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -764,7 +764,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-block" }
# <- {
@@ -1168,7 +1168,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-blockstats" }
# <- {
@@ -1461,7 +1461,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "block_resize",
# "arguments": { "device": "scratch", "size": 1073741824 } }
@@ -1682,7 +1682,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "blockdev-snapshot-sync",
# "arguments": { "device": "ide-hd0",
@@ -1715,7 +1715,7 @@
#
# Since: 2.5
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "blockdev-add",
# "arguments": { "driver": "qcow2",
@@ -1861,7 +1861,7 @@
#
# Since: 1.3
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "block-commit",
# "arguments": { "device": "virtio0",
@@ -1899,7 +1899,7 @@
#
# Since: 1.6
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "drive-backup",
# "arguments": { "device": "drive0",
@@ -1925,7 +1925,7 @@
#
# Since: 2.3
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "blockdev-backup",
# "arguments": { "device": "src-id",
@@ -1949,7 +1949,7 @@
#
# Since: 2.0
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-named-block-nodes" }
# <- { "return": [ { "ro":false,
@@ -2130,7 +2130,7 @@
#
# Since: 1.3
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "drive-mirror",
# "arguments": { "device": "ide-hd0",
@@ -2307,7 +2307,7 @@
#
# Since: 2.4
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "block-dirty-bitmap-add",
# "arguments": { "node": "drive0", "name": "bitma
[PATCH 8/8] qapi: remove "Example" doc section
Fully eliminate the "Example" sections in QAPI doc blocks now that they
have all been converted to arbitrary rST syntax using the
".. qmp-example::" directive. Update tests to match.
Migrating to the new syntax
---
The old "Example:" or "Examples:" section syntax is now caught as an
error, but "Example::" is stil permitted as explicit rST syntax for an
un-lexed, generic preformatted text block.
('Example' is not special in this case, any sentence that ends with "::"
will start an indented code block in rST.)
Arbitrary rST for Examples is now possible, but it's strongly
recommended that documentation authors use the ".. qmp-example::"
directive for consistent visual formatting in rendered HTML docs. The
":title:" directive option may be used to add extra information into the
title bar for the example. The ":annotated:" option can be used to write
arbitrary rST instead, with nested "::" blocks applying QMP formatting
where desired.
Other choices available are ".. code-block:: QMP" which will not create
an "Example:" box, or the short-form "::" code-block syntax which will
not apply QMP highlighting when used outside of the qmp-example
directive.
Why?
This patch has several benefits:
1. Example sections can now be written more arbitrarily, mixing
explanatory paragraphs and code blocks however desired.
2. Example sections can now use fully arbitrary rST.
3. All code blocks are now lexed and validated as QMP; increasing
usability of the docs and ensuring validity of example snippets.
(To some extent - This patch only gaurantees it lexes correctly, not
that it's valid under the JSON or QMP grammars. It will catch most
small mistakes, however.)
4. Each qmp-example can be titled or annotated independently without
bypassing the QMP lexer/validator.
(i.e. code blocks are now for *code* only, so we don't have to
sacrifice exposition for having lexically valid examples.)
NOTE: As with the "Notes" conversion patch, this patch (and those
preceding) may change the rendering order for Examples in the
current generator. The forthcoming qapidoc rewrite will fix this
by always generating documentation in source order.
Signed-off-by: John Snow
---
docs/devel/qapi-code-gen.rst| 58 -
scripts/qapi/parser.py | 10 +-
tests/qapi-schema/doc-good.json | 19 +++
tests/qapi-schema/doc-good.out | 26 ++-
tests/qapi-schema/doc-good.txt | 23 ++---
5 files changed, 98 insertions(+), 38 deletions(-)
diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index ae97b335cbf..2e10a3cbd69 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -899,7 +899,7 @@ Documentation markup
Documentation comments can use most rST markup. In particular,
-a ``::`` literal block can be used for examples::
+a ``::`` literal block can be used for pre-formatted text::
# ::
#
@@ -995,8 +995,8 @@ line "Features:", like this::
# @feature: Description text
A tagged section begins with a paragraph that starts with one of the
-following words: "Since:", "Example:"/"Examples:", "Returns:",
-"Errors:", "TODO:". It ends with the start of a new section.
+following words: "Since:", "Returns:", "Errors:", "TODO:". It ends with
+the start of a new section.
The second and subsequent lines of tagged sections must be indented
like this::
@@ -1020,13 +1020,53 @@ detailing a relevant error condition. For example::
A "Since: x.y.z" tagged section lists the release that introduced the
definition.
-An "Example" or "Examples" section is rendered entirely
-as literal fixed-width text. "TODO" sections are not rendered at all
-(they are for developers, not users of QMP). In other sections, the
-text is formatted, and rST markup can be used.
+"TODO" sections are not rendered at all (they are for developers, not
+users of QMP). In other sections, the text is formatted, and rST markup
+can be used.
+
+QMP Examples can be added by using the ``.. qmp-example::``
+directive. In its simplest form, this can be used to contain a single
+QMP code block which accepts standard JSON syntax with additional server
+directionality indicators (``->`` and ``<-``), and elisions (``...``).
+
+Optionally, a plaintext title may be provided by using the ``:title:``
+directive option. If the title is omitted, the example title will
+default to "Example:".
+
+A simple QMP example::
+
+ # .. qmp-example::
+ #:title: Using query-block
+ #
+ #-> { "execute": "query-block" }
+ #<- { ... }
+
+
[PATCH 0/8] qapi: convert example sections to qmp-example rST directives
GitLab: https://gitlab.com/jsnow/qemu/-/pipelines/1359714660 This patchset focuses on converting example sections to rST directives using a new `.. qmp-example::` directive. It is based on what I *assume* will be Markus' next pull request that covers note conversion. Pull these patches from GitLab directly if that's too annoying: https://gitlab.com/jsnow/qemu/-/commits/sphinx-domain-prereqs-examples It is also annoyingly the case that both Markus' next pull request and this series conflicts with a separate series I sent out, "docs/python: bump minimum Sphinx version" - so it's extremely likely I'll need to rebase and respin this series depending on what goes in and in what order. Ah well... Changes since this was split out from the prior series: - Harmonie updated the CSS for the example block section. I think it's really tidy now! Thanks Harmonie! - Dependence on SphinxDirective was removed, but it will likely re-appear in the next series anyway. - qapi-code-gen.rst was updated with a section on how to write examples. - Various minor tweaks to comments, commit messages, docs, etc. Harmonie Snow (1): docs/sphinx: add CSS styling for qmp-example directive John Snow (7): docs/qapidoc: factor out do_parse() docs/qapidoc: create qmp-example directive docs/qapidoc: add QMP highlighting to annotated qmp-example blocks qapi: convert "Example" sections without titles qapi: convert "Example" sections with titles qapi: convert "Example" sections with longer prose qapi: remove "Example" doc section docs/devel/qapi-code-gen.rst | 58 +++-- docs/sphinx-static/theme_overrides.css | 49 docs/sphinx/qapidoc.py | 156 + qapi/acpi.json | 4 +- qapi/block-core.json | 88 +++--- qapi/block.json| 57 + qapi/char.json | 24 ++-- qapi/control.json | 8 +- qapi/dump.json | 8 +- qapi/machine-target.json | 2 +- qapi/machine.json | 68 ++- qapi/migration.json| 90 +++--- qapi/misc-target.json | 22 ++-- qapi/misc.json | 32 ++--- qapi/net.json | 20 ++-- qapi/pci.json | 2 +- qapi/qdev.json | 10 +- qapi/qom.json | 16 +-- qapi/replay.json | 8 +- qapi/rocker.json | 8 +- qapi/run-state.json| 30 ++--- qapi/tpm.json | 6 +- qapi/trace.json| 4 +- qapi/transaction.json | 2 +- qapi/ui.json | 45 +++ qapi/vfio.json | 2 +- qapi/virtio.json | 45 --- qapi/yank.json | 4 +- scripts/qapi/parser.py | 10 +- tests/qapi-schema/doc-good.json| 19 +-- tests/qapi-schema/doc-good.out | 26 +++-- tests/qapi-schema/doc-good.txt | 23 ++-- 32 files changed, 605 insertions(+), 341 deletions(-) -- 2.45.0
[PATCH 6/8] qapi: convert "Example" sections with titles
When an Example section has a brief explanation, convert it to a
qmp-example:: section using the :title: option.
Rule of thumb: If the title can fit on a single line and requires no rST
markup, it's a good candidate for using the :title: option of
qmp-example.
In this patch, trailing punctuation is removed from the title section
for consistent headline aesthetics. In just one case, specifics of the
example are removed to make the title read better.
See commit-4: "docs/qapidoc: create qmp-example directive", for a
detailed explanation of this custom directive syntax.
See commit+2: "qapi: remove "Example" doc section" for a detailed
explanation of why.
Signed-off-by: John Snow
---
qapi/block-core.json | 24
qapi/block.json | 13 ++---
qapi/migration.json | 25 ++---
qapi/qom.json| 8
qapi/ui.json | 11 ++-
qapi/virtio.json | 19 ++-
6 files changed, 52 insertions(+), 48 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 4e0f0395146..a371e3464d2 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -5885,9 +5885,8 @@
#
# Since: 2.7
#
-# Examples:
-#
-# 1. Add a new node to a quorum
+# .. qmp-example::
+#:title: Add a new node to a quorum
#
# -> { "execute": "blockdev-add",
# "arguments": {
@@ -5901,7 +5900,8 @@
# "node": "new_node" } }
# <- { "return": {} }
#
-# 2. Delete a quorum's node
+# .. qmp-example::
+#:title: Delete a quorum's node
#
# -> { "execute": "x-blockdev-change",
# "arguments": { "parent": "disk1",
@@ -5937,16 +5937,16 @@
#
# Since: 2.12
#
-# Examples:
-#
-# 1. Move a node into an IOThread
+# .. qmp-example::
+#:title: Move a node into an IOThread
#
# -> { "execute": "x-blockdev-set-iothread",
# "arguments": { "node-name": "disk1",
# "iothread": "iothread0" } }
# <- { "return": {} }
#
-# 2. Move a node into the main loop
+# .. qmp-example::
+#:title: Move a node into the main loop
#
# -> { "execute": "x-blockdev-set-iothread",
# "arguments": { "node-name": "disk1",
@@ -6022,16 +6022,16 @@
#
# Since: 2.0
#
-# Examples:
-#
-# 1. Read operation
+# .. qmp-example::
+#:title: Read operation
#
# <- { "event": "QUORUM_REPORT_BAD",
# "data": { "node-name": "node0", "sector-num": 345435,
"sectors-count": 5,
#"type": "read" },
# "timestamp": { "seconds": 1344522075, "microseconds": 745528 } }
#
-# 2. Flush operation
+# .. qmp-example::
+#:title: Flush operation
#
# <- { "event": "QUORUM_REPORT_BAD",
# "data": { "node-name": "node0", "sector-num": 0, "sectors-count":
2097120,
diff --git a/qapi/block.json b/qapi/block.json
index c8e52bc2d29..5ddd061e964 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -342,9 +342,8 @@
#
# Since: 2.5
#
-# Examples:
-#
-# 1. Change a removable medium
+# .. qmp-example::
+#:title: Change a removable medium
#
# -> { "execute": "blockdev-change-medium",
# "arguments": { "id": "ide0-1-0",
@@ -352,7 +351,8 @@
# "format": "raw" } }
# <- { "return": {} }
#
-# 2. Load a read-only medium into a writable drive
+# .. qmp-example::
+#:title: Load a read-only medium into a writable drive
#
# -> { "execute": "blockdev-change-medium",
# "arguments": { "id": "floppyA",
@@ -577,9 +577,8 @@
# "boundaries-write": [1000, 5000] } }
# <- { "return": {} }
#
-# Example:
-#
-# Remove all latency histograms:
+# .. qmp-example::
+#:title: Remove all latency histograms
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0" } }
diff --git a/qapi/migration.json b/qapi/migration.json
index a4391ea7e6f..37ce8afa380 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -287,14 +287,14 @@
#
# Since: 0.14
#
-# Examples:
-#
-# 1. Before the first migration
+# .. qmp-example::
+#:title: Before the first migration
#
# -> { "execute": "query-migrate" }
# <- { "return":
[PATCH 1/8] docs/qapidoc: factor out do_parse()
Factor out the compatibility parser helper into a base class, so it can be shared by other directives. Signed-off-by: John Snow --- docs/sphinx/qapidoc.py | 64 +++--- 1 file changed, 35 insertions(+), 29 deletions(-) diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py index efcd84656fa..43dd99e21e6 100644 --- a/docs/sphinx/qapidoc.py +++ b/docs/sphinx/qapidoc.py @@ -494,7 +494,41 @@ def visit_module(self, name): super().visit_module(name) -class QAPIDocDirective(Directive): +class NestedDirective(Directive): +def run(self): +raise NotImplementedError + +def do_parse(self, rstlist, node): +""" +Parse rST source lines and add them to the specified node + +Take the list of rST source lines rstlist, parse them as +rST, and add the resulting docutils nodes as children of node. +The nodes are parsed in a way that allows them to include +subheadings (titles) without confusing the rendering of +anything else. +""" +# This is from kerneldoc.py -- it works around an API change in +# Sphinx between 1.6 and 1.7. Unlike kerneldoc.py, we use +# sphinx.util.nodes.nested_parse_with_titles() rather than the +# plain self.state.nested_parse(), and so we can drop the saving +# of title_styles and section_level that kerneldoc.py does, +# because nested_parse_with_titles() does that for us. +if USE_SSI: +with switch_source_input(self.state, rstlist): +nested_parse_with_titles(self.state, rstlist, node) +else: +save = self.state.memo.reporter +self.state.memo.reporter = AutodocReporter( +rstlist, self.state.memo.reporter +) +try: +nested_parse_with_titles(self.state, rstlist, node) +finally: +self.state.memo.reporter = save + + +class QAPIDocDirective(NestedDirective): """Extract documentation from the specified QAPI .json file""" required_argument = 1 @@ -532,34 +566,6 @@ def run(self): # so they are displayed nicely to the user raise ExtensionError(str(err)) from err -def do_parse(self, rstlist, node): -"""Parse rST source lines and add them to the specified node - -Take the list of rST source lines rstlist, parse them as -rST, and add the resulting docutils nodes as children of node. -The nodes are parsed in a way that allows them to include -subheadings (titles) without confusing the rendering of -anything else. -""" -# This is from kerneldoc.py -- it works around an API change in -# Sphinx between 1.6 and 1.7. Unlike kerneldoc.py, we use -# sphinx.util.nodes.nested_parse_with_titles() rather than the -# plain self.state.nested_parse(), and so we can drop the saving -# of title_styles and section_level that kerneldoc.py does, -# because nested_parse_with_titles() does that for us. -if USE_SSI: -with switch_source_input(self.state, rstlist): -nested_parse_with_titles(self.state, rstlist, node) -else: -save = self.state.memo.reporter -self.state.memo.reporter = AutodocReporter( -rstlist, self.state.memo.reporter -) -try: -nested_parse_with_titles(self.state, rstlist, node) -finally: -self.state.memo.reporter = save - def setup(app): """Register qapi-doc directive with Sphinx""" -- 2.45.0
[PATCH 2/8] docs/qapidoc: create qmp-example directive
This is a directive that creates a syntactic sugar for creating
"Example" boxes very similar to the ones already used in the bitmaps.rst
document, please see e.g.
https://www.qemu.org/docs/master/interop/bitmaps.html#creation-block-dirty-bitmap-add
In its simplest form, when a custom title is not needed or wanted, and
the example body is *solely* a QMP example:
```
.. qmp-example::
{body}
```
is syntactic sugar for:
```
.. admonition:: Example:
.. code-block:: QMP
{body}
```
When a custom, plaintext title that describes the example is desired,
this form:
```
.. qmp-example::
:title: Defrobnification
{body}
```
Is syntactic sugar for:
```
.. admonition:: Example: Defrobnification
.. code-block:: QMP
{body}
```
Lastly, when Examples are multi-step processes that require non-QMP
exposition, have lengthy titles, or otherwise involve prose with rST
markup (lists, cross-references, etc), the most complex form:
```
.. qmp-example::
:annotated:
This example shows how to use `foo-command`::
{body}
For more information, please see `frobnozz`.
```
Is desugared to:
```
.. admonition:: Example:
This example shows how to use `foo-command`::
{body}
For more information, please see `frobnozz`.
```
Note that :annotated: and :title: options can be combined together, if
desired.
The primary benefit here being documentation source consistently using
the same directive for all forms of examples to ensure consistent visual
styling, and ensuring all relevant prose is visually grouped alongside
the code literal block.
Note that as of this commit, the code-block rST syntax "::" does not
apply QMP highlighting; you would need to use ".. code-block:: QMP". The
very next commit changes this behavior to assume all "::" code blocks
within this directive are QMP blocks.
Signed-off-by: John Snow
---
docs/sphinx/qapidoc.py | 55 ++
1 file changed, 55 insertions(+)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index 43dd99e21e6..b0f3917dc5b 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -27,6 +27,7 @@
import os
import re
import textwrap
+from typing import List
from docutils import nodes
from docutils.parsers.rst import Directive, directives
@@ -36,6 +37,7 @@
from qapi.schema import QAPISchema
import sphinx
+from sphinx.directives.code import CodeBlock
from sphinx.errors import ExtensionError
from sphinx.util.nodes import nested_parse_with_titles
@@ -567,10 +569,63 @@ def run(self):
raise ExtensionError(str(err)) from err
+class QMPExample(CodeBlock, NestedDirective):
+"""
+Custom admonition for QMP code examples.
+
+When the :annotated: option is present, the body of this directive
+is parsed as normal rST instead. Code blocks must be explicitly
+written by the user, but this allows for intermingling explanatory
+paragraphs with arbitrary rST syntax and code blocks for more
+involved examples.
+
+When :annotated: is absent, the directive body is treated as a
+simple standalone QMP code block literal.
+"""
+
+required_argument = 0
+optional_arguments = 0
+has_content = True
+option_spec = {
+"annotated": directives.flag,
+"title": directives.unchanged,
+}
+
+def admonition_wrap(self, *content) -> List[nodes.Node]:
+title = "Example:"
+if "title" in self.options:
+title = f"{title} {self.options['title']}"
+
+admon = nodes.admonition(
+"",
+nodes.title("", title),
+*content,
+classes=["admonition", "admonition-example"],
+)
+return [admon]
+
+def run_annotated(self) -> List[nodes.Node]:
+content_node: nodes.Element = nodes.section()
+self.do_parse(self.content, content_node)
+return content_node.children
+
+def run(self) -> List[nodes.Node]:
+annotated = "annotated" in self.options
+
+if annotated:
+content_nodes = self.run_annotated()
+else:
+self.arguments = ["QMP"]
+content_nodes = super().run()
+
+return self.admonition_wrap(*content_nodes)
+
+
def setup(app):
"""Register qapi-doc directive with Sphinx"""
app.add_config_value("qapidoc_srctree", None, "env")
app.add_directive("qapi-doc", QAPIDocDirective)
+app.add_directive("qmp-example", QMPExample)
return {
"version": __version__,
--
2.45.0
Re: [PATCH 3/4] iotests: Change imports for Python 3.13
On Tue, Jul 2, 2024, 1:51 PM Nir Soffer wrote: > > On 2 Jul 2024, at 17:44, John Snow wrote: > > > > On Tue, Jul 2, 2024 at 7:52 AM Nir Soffer wrote: > >> On Thu, Jun 27, 2024 at 2:23 AM John Snow wrote: >> > >> > Python 3.13 isn't out yet, but it's in beta and Fedora is ramping up to >> > make it the default system interpreter for Fedora 41. >> > >> > They moved our cheese for where ContextManager lives; add a conditional >> > to locate it while we support both pre-3.9 and 3.13+. >> > >> > Signed-off-by: John Snow >> > --- >> > tests/qemu-iotests/testenv.py| 7 ++- >> > tests/qemu-iotests/testrunner.py | 9 ++--- >> > 2 files changed, 12 insertions(+), 4 deletions(-) >> > >> > diff --git a/tests/qemu-iotests/testenv.py >> b/tests/qemu-iotests/testenv.py >> > index 588f30a4f14..96d69e56963 100644 >> > --- a/tests/qemu-iotests/testenv.py >> > +++ b/tests/qemu-iotests/testenv.py >> > @@ -25,7 +25,12 @@ >> > import random >> > import subprocess >> > import glob >> > -from typing import List, Dict, Any, Optional, ContextManager >> > +from typing import List, Dict, Any, Optional >> > + >> > +if sys.version_info >= (3, 9): >> > +from contextlib import AbstractContextManager as ContextManager >> > +else: >> > +from typing import ContextManager >> >> It can be cleaner to add a compat module hiding the details so the >> entire project >> can have a single instance of this. Other code will just use: >> >> from compat import ContextManager >> > > If there were more than two uses, I'd consider it. As it stands, a > compat.py module with just one import conditional in it doesn't seem worth > the hassle. Are there more cases of compatibility goop inside iotests that > need to be factored out to make it worth it? > > > I don’t about other. For me even one instance is ugly enough :-) > I was going to add it to qemu/utils, but then I remembered the testenv/testrunner script here needs to operate without external dependencies because part of the function of these modules is to *locate* those dependencies. Ehh. I'm going to say that repeating the import scaffolding in just two places is fine enough for now and really not worth adding a compat.py for *just* this. Let's just get the tests green. --js
[PATCH v2 0/2] docs/python: bump minimum Sphinx version
With recent deprecations, we can advance our minimum sphinx version safely. This is heavily motivated by new qapidoc work which is much easier to maintain cross-version compatibility for - see difficulties in our dbus documentation which only works on sphinx >= 4. We can only guarantee >= 3.4.3 now, but that's still vastly easier than maintaining compatibility all the way back to 1.x. https://gitlab.com/jsnow/qemu/-/pipelines/1359549621 (failures appear to be unrelated to the series.) V2: refactor all code deletions into patch 2 John Snow (2): Python: bump minimum sphinx version to 3.4.3 docs: remove Sphinx 1.x compatibility code docs/conf.py | 7 +++ docs/sphinx/hxtool.py| 21 - docs/sphinx/kerneldoc.py | 38 -- docs/sphinx/kernellog.py | 28 docs/sphinx/qapidoc.py | 29 +++-- pythondeps.toml | 2 +- 6 files changed, 23 insertions(+), 102 deletions(-) delete mode 100644 docs/sphinx/kernellog.py -- 2.45.0
[PATCH v2 1/2] Python: bump minimum sphinx version to 3.4.3
With RHEL 8 support retired (It's been two years since RHEL9 released),
our very oldest build platform version of Sphinx is now 3.4.3; and
keeping backwards compatibility for versions as old as v1.6 when using
domain extensions is a lot of work we don't need to do.
This patch is motivated by my work creating a new QAPI domain, which
unlike the dbus documentation, cannot be allowed to regress by creating
a "dummy" doc when operating under older sphinx versions. Easier is to
raise our minimum version as far as we can push it forwards, reducing my
burden in creating cross-compatibility hacks and patches.
A sampling of sphinx versions from various distributions, courtesy
https://repology.org/project/python:sphinx/versions
Alpine 3.16: v4.3.0 (QEMU support ended 2024-05-23)
Alpine 3.17: v5.3.0
Alpine 3.18: v6.1.3
Alpine 3.19: v6.2.1
Ubuntu 20.04 LTS: EOL
Ubuntu 22.04 LTS: v4.3.2
Ubuntu 22.10: EOL
Ubuntu 23.04: EOL
Ubuntu 23.10: v5.3.0
Ubuntu 24.04 LTS: v7.2.6
Debian 11: v3.4.3 (QEMU support ends 2024-07-xx)
Debian 12: v5.3.0
Fedora 38: EOL
Fedora 39: v6.2.1
Fedora 40: v7.2.6
CentOS Stream 8: v1.7.6 (QEMU support ended 2024-05-17)
CentOS Stream 9: v3.4.3
OpenSUSE Leap 15.4: EOL
OpenSUSE Leap 15.5: 2.3.1, 4.2.0 and 7.2.6
RHEL9 / CentOS Stream 9 becomes the new defining factor in staying at
Sphinx 3.4.3 due to downstream offline build requirements that force us
to use platform Sphinx instead of newer packages from PyPI.
Signed-off-by: John Snow
Acked-by: Paolo Bonzini
Acked-by: Markus Armbruster
---
docs/conf.py| 7 +++
pythondeps.toml | 2 +-
2 files changed, 4 insertions(+), 5 deletions(-)
diff --git a/docs/conf.py b/docs/conf.py
index aae0304ac6e..876f6768815 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -53,10 +53,9 @@
# If your documentation needs a minimal Sphinx version, state it here.
#
-# Sphinx 1.5 and earlier can't build our docs because they are too
-# picky about the syntax of the argument to the option:: directive
-# (see Sphinx bugs #646, #3366).
-needs_sphinx = '1.6'
+# 3.4.3 is the oldest version of Sphinx that ships on a platform we
+# pledge build support for.
+needs_sphinx = '3.4.3'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
diff --git a/pythondeps.toml b/pythondeps.toml
index 9c16602d303..bc656376caa 100644
--- a/pythondeps.toml
+++ b/pythondeps.toml
@@ -23,7 +23,7 @@ meson = { accepted = ">=0.63.0", installed = "1.2.3", canary
= "meson" }
[docs]
# Please keep the installed versions in sync with docs/requirements.txt
-sphinx = { accepted = ">=1.6", installed = "5.3.0", canary = "sphinx-build" }
+sphinx = { accepted = ">=3.4.3", installed = "5.3.0", canary = "sphinx-build" }
sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.1.1" }
[avocado]
--
2.45.0
[PATCH v2 2/2] docs: remove Sphinx 1.x compatibility code
In general, the Use_SSI workaround is no longer needed, and neither is
the pre-1.6 logging shim for kerneldoc.
Signed-off-by: John Snow
Acked-by: Markus Armbruster
---
docs/sphinx/hxtool.py| 21 -
docs/sphinx/kerneldoc.py | 38 --
docs/sphinx/kernellog.py | 28
docs/sphinx/qapidoc.py | 29 +++--
4 files changed, 19 insertions(+), 97 deletions(-)
delete mode 100644 docs/sphinx/kernellog.py
diff --git a/docs/sphinx/hxtool.py b/docs/sphinx/hxtool.py
index 3729084a36c..a84723be19e 100644
--- a/docs/sphinx/hxtool.py
+++ b/docs/sphinx/hxtool.py
@@ -24,16 +24,10 @@
from docutils.statemachine import ViewList
from docutils.parsers.rst import directives, Directive
from sphinx.errors import ExtensionError
+from sphinx.util.docutils import switch_source_input
from sphinx.util.nodes import nested_parse_with_titles
import sphinx
-# Sphinx up to 1.6 uses AutodocReporter; 1.7 and later
-# use switch_source_input. Check borrowed from kerneldoc.py.
-Use_SSI = sphinx.__version__[:3] >= '1.7'
-if Use_SSI:
-from sphinx.util.docutils import switch_source_input
-else:
-from sphinx.ext.autodoc import AutodocReporter
__version__ = '1.0'
@@ -185,16 +179,9 @@ def run(self):
# of title_styles and section_level that kerneldoc.py does,
# because nested_parse_with_titles() does that for us.
def do_parse(self, result, node):
-if Use_SSI:
-with switch_source_input(self.state, result):
-nested_parse_with_titles(self.state, result, node)
-else:
-save = self.state.memo.reporter
-self.state.memo.reporter = AutodocReporter(result,
self.state.memo.reporter)
-try:
-nested_parse_with_titles(self.state, result, node)
-finally:
-self.state.memo.reporter = save
+with switch_source_input(self.state, result):
+nested_parse_with_titles(self.state, result, node)
+
def setup(app):
""" Register hxtool-doc directive with Sphinx"""
diff --git a/docs/sphinx/kerneldoc.py b/docs/sphinx/kerneldoc.py
index 72c403a7379..3aa972f2e89 100644
--- a/docs/sphinx/kerneldoc.py
+++ b/docs/sphinx/kerneldoc.py
@@ -38,20 +38,14 @@
from docutils.statemachine import ViewList
from docutils.parsers.rst import directives, Directive
-#
-# AutodocReporter is only good up to Sphinx 1.7
-#
import sphinx
+from sphinx.util import logging
+from sphinx.util.docutils import switch_source_input
-Use_SSI = sphinx.__version__[:3] >= '1.7'
-if Use_SSI:
-from sphinx.util.docutils import switch_source_input
-else:
-from sphinx.ext.autodoc import AutodocReporter
-
-import kernellog
__version__ = '1.0'
+logger = logging.getLogger('kerneldoc')
+
class KernelDocDirective(Directive):
"""Extract kernel-doc comments from the specified file"""
@@ -111,8 +105,7 @@ def run(self):
cmd += [filename]
try:
-kernellog.verbose(env.app,
- 'calling kernel-doc \'%s\'' % (" ".join(cmd)))
+logger.verbose('calling kernel-doc \'%s\'' % (" ".join(cmd)))
p = subprocess.Popen(cmd, stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
out, err = p.communicate()
@@ -122,8 +115,10 @@ def run(self):
if p.returncode != 0:
sys.stderr.write(err)
-kernellog.warn(env.app,
- 'kernel-doc \'%s\' failed with return code %d'
% (" ".join(cmd), p.returncode))
+logger.warning(
+'kernel-doc \'%s\' failed with return code %d' %
+(" ".join(cmd), p.returncode)
+)
return [nodes.error(None, nodes.paragraph(text = "kernel-doc
missing"))]
elif env.config.kerneldoc_verbosity > 0:
sys.stderr.write(err)
@@ -149,22 +144,13 @@ def run(self):
return node.children
except Exception as e: # pylint: disable=W0703
-kernellog.warn(env.app, 'kernel-doc \'%s\' processing failed with:
%s' %
+logger.warning('kernel-doc \'%s\' processing failed with: %s' %
(" ".join(cmd), str(e)))
return [nodes.error(None, nodes.paragraph(text = "kernel-doc
missing"))]
def do_parse(self, result, node):
-if Use_SSI:
-with switch_source_input(self.state, result):
-self.state.nested_parse(result, 0, node, match_titles=1)
-else:
-save = self.state.memo.title_styles,
self.state.memo.section_level, self.state.memo.reporter
-self.state.memo.reporter = AutodocReporter(result,
self.state.memo.reporter)
-self.state.memo.title_styles, self.state.
Re: [PATCH] qapi/qom: Document feature unstable of @x-vfio-user-server
On Wed, Jul 3, 2024 at 5:53 AM Markus Armbruster wrote: > Commit 8f9a9259d32c added ObjectType member @x-vfio-user-server with > feature unstable, but neglected to explain why it is unstable. Do > that now. > > Fixes: 8f9a9259d32c (vfio-user: define vfio-user-server object) > Cc: Elena Ufimtseva > Cc: John G Johnson > Cc: Jagannathan Raman > Cc: qemu-sta...@nongnu.org Reported-by: John Snow (... This is a joke, to be extremely clear) > > Signed-off-by: Markus Armbruster > Reviewed-by: John Snow > --- > qapi/qom.json | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/qapi/qom.json b/qapi/qom.json > index 8bd299265e..3c0f8c633d 100644 > --- a/qapi/qom.json > +++ b/qapi/qom.json > @@ -1012,7 +1012,8 @@ > # > # Features: > # > -# @unstable: Member @x-remote-object is experimental. > +# @unstable: Members @x-remote-object and @x-vfio-user-server are > +# experimental. > # > # Since: 6.0 > ## > -- > 2.45.0 > >
Re: [PATCH 1/2] Python: bump minimum sphinx version to 3.4.3
On Wed, Jul 3, 2024 at 8:12 AM Paolo Bonzini wrote: > On Wed, Jul 3, 2024 at 2:06 PM John Snow wrote: > > On Wed, Jul 3, 2024, 4:00 AM Paolo Bonzini wrote: > >> On 7/2/24 21:59, John Snow wrote: > >> > With RHEL 8 support retired (It's been two years today since RHEL 9 > >> > came out), our very oldest build platform version of Sphinx is now > >> > 3.4.3; and keeping backwards compatibility for versions as old as v1.6 > >> > when using domain extensions is a lot of work we don't need to do. > >> > >> Technically that's unrelated: thanks to your venv work, :) builds on > >> RHEL 8 / CentOS Stream 8 do not pick the platform Sphinx, because it > >> runs under Python 3.6. Therefore the version included in RHEL 8 does > >> not matter for picking the minimum supported Sphinx version. > > > > I think I can't mandate 4.x because of RHEL 9 builds though, and offline > requirements. > > Offline requirements are not a problem; on RHEL 8 you just have to > install with pip in order to build docs offline. But yes, RHEL 9 is > still using platform Python and therefore 3.4.3 remains the limit even > after we stop supporting bullseye. > To be clear I mean offline, isolated RPM builds under RHEL9 where I don't think we can utilize PyPI at all; and vendoring Sphinx is I think not a practical option due to the number of dependencies and non-pure Python deps. It's not a problem for developer workflow, just downstream packaging. Luckily OpenSUSE offers newer Sphinx, but RHEL doesn't yet. Maybe that can be rectified eventually - possibly after 3.8 is EOL and there is increased demand for newer Python packages to be made available in RHEL... but not yet, today. > > Paolo > > Thanks for the ack O:-)
Re: [PATCH 1/2] Python: bump minimum sphinx version to 3.4.3
On Wed, Jul 3, 2024, 4:00 AM Paolo Bonzini wrote:
> On 7/2/24 21:59, John Snow wrote:
> > With RHEL 8 support retired (It's been two years today since RHEL 9
> > came out), our very oldest build platform version of Sphinx is now
> > 3.4.3; and keeping backwards compatibility for versions as old as v1.6
> > when using domain extensions is a lot of work we don't need to do.
>
> Technically that's unrelated: thanks to your venv work, :) builds on
> RHEL 8 / CentOS Stream 8 do not pick the platform Sphinx, because it
> runs under Python 3.6. Therefore the version included in RHEL 8 does
> not matter for picking the minimum supported Sphinx version.
>
...!
I think I can't mandate 4.x because of RHEL 9 builds though, and offline
requirements.
> > Debian 11: v3.4.3 (QEMU support ends 2024-07-xx)
>
> Nice. :)
>
> > diff --git a/pythondeps.toml b/pythondeps.toml
> > index 9c16602d303..bc656376caa 100644
> > --- a/pythondeps.toml
> > +++ b/pythondeps.toml
> > @@ -23,7 +23,7 @@ meson = { accepted = ">=0.63.0", installed = "1.2.3",
> canary = "meson" }
> >
> > [docs]
> > # Please keep the installed versions in sync with docs/requirements.txt
> > -sphinx = { accepted = ">=1.6", installed = "5.3.0", canary =
> "sphinx-build" }
> > +sphinx = { accepted = ">=3.4.3", installed = "5.3.0", canary =
> "sphinx-build" }
> > sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.1.1" }
> >
> > [avocado]
>
> Acked-by: Paolo Bonzini
>
> Paolo
>
>
Re: [PATCH 0/2] docs/python: bump minimum Sphinx version
On Tue, Jul 2, 2024 at 3:59 PM John Snow wrote: > With recent deprecations, we can advance our minimum sphinx version > safely. This is heavily motivated by new qapidoc work which is much > easier to maintain cross-version compatibility for - see difficulties in > our dbus documentation which only works on sphinx >= 4. > > We can only guarantee >= 3.4.3 now, but that's still vastly easier than > maintaining compatibility all the way back to 1.x. > > GitLab: https://gitlab.com/jsnow/qemu/-/pipelines/1357902509 > > (failures appear to be unrelated to the series.) > > John Snow (2): > Python: bump minimum sphinx version to 3.4.3 > docs: remove Sphinx 1.x compatibility code > > docs/conf.py | 7 +++ > docs/sphinx/hxtool.py| 21 - > docs/sphinx/kerneldoc.py | 38 -- > docs/sphinx/kernellog.py | 28 > docs/sphinx/qapidoc.py | 29 +++-- > pythondeps.toml | 2 +- > 6 files changed, 23 insertions(+), 102 deletions(-) > delete mode 100644 docs/sphinx/kernellog.py > > -- > 2.45.0 > Bleurgh. I meant to shuffle some of the changes in patch 1 into patch 2, I'll fix that on re-spin. If you want to review it anyway, just presume I'll fix that next go-around. --js
[PATCH 1/2] Python: bump minimum sphinx version to 3.4.3
With RHEL 8 support retired (It's been two years today since RHEL 9
came out), our very oldest build platform version of Sphinx is now
3.4.3; and keeping backwards compatibility for versions as old as v1.6
when using domain extensions is a lot of work we don't need to do.
This patch is motivated by my work creating a new QAPI domain, which
unlike the dbus documentation, cannot be allowed to regress by creating
a "dummy" doc. Easier is to improve our minimum version as far as we can
push it forwards, reducing my burden in creating cross-compatibility
hacks and patches.
A sampling of sphinx versions from various distributions, courtesy
https://repology.org/project/python:sphinx/versions
Alpine 3.16: v4.3.0 (QEMU support ended 2024-05-23)
Alpine 3.17: v5.3.0
Alpine 3.18: v6.1.3
Alpine 3.19: v6.2.1
Ubuntu 20.04 LTS: EOL
Ubuntu 22.04 LTS: v4.3.2
Ubuntu 22.10: EOL
Ubuntu 23.04: EOL
Ubuntu 23.10: v5.3.0
Ubuntu 24.04 LTS: v7.2.6
Debian 11: v3.4.3 (QEMU support ends 2024-07-xx)
Debian 12: v5.3.0
Fedora 38: EOL
Fedora 39: v6.2.1
Fedora 40: v7.2.6
CentOS Stream 8: v1.7.6 (QEMU support ended 2024-05-17)
CentOS Stream 9: v3.4.3
OpenSUSE Leap 15.4: EOL
OpenSUSE Leap 15.5: 2.3.1, 4.2.0 and 7.2.6
Signed-off-by: John Snow
---
docs/conf.py | 7 +++
docs/sphinx/qapidoc.py | 29 +++--
pythondeps.toml| 2 +-
3 files changed, 7 insertions(+), 31 deletions(-)
diff --git a/docs/conf.py b/docs/conf.py
index aae0304ac6e..876f6768815 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -53,10 +53,9 @@
# If your documentation needs a minimal Sphinx version, state it here.
#
-# Sphinx 1.5 and earlier can't build our docs because they are too
-# picky about the syntax of the argument to the option:: directive
-# (see Sphinx bugs #646, #3366).
-needs_sphinx = '1.6'
+# 3.4.3 is the oldest version of Sphinx that ships on a platform we
+# pledge build support for.
+needs_sphinx = '3.4.3'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index f270b494f01..9a0cfcbce75 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -31,6 +31,7 @@
from docutils.statemachine import ViewList
from docutils.parsers.rst import directives, Directive
from sphinx.errors import ExtensionError
+from sphinx.util.docutils import switch_source_input
from sphinx.util.nodes import nested_parse_with_titles
import sphinx
from qapi.gen import QAPISchemaVisitor
@@ -38,15 +39,6 @@
from qapi.schema import QAPISchema
-# Sphinx up to 1.6 uses AutodocReporter; 1.7 and later
-# use switch_source_input. Check borrowed from kerneldoc.py.
-Use_SSI = sphinx.__version__[:3] >= '1.7'
-if Use_SSI:
-from sphinx.util.docutils import switch_source_input
-else:
-from sphinx.ext.autodoc import AutodocReporter
-
-
__version__ = '1.0'
@@ -517,23 +509,8 @@ def do_parse(self, rstlist, node):
subheadings (titles) without confusing the rendering of
anything else.
"""
-# This is from kerneldoc.py -- it works around an API change in
-# Sphinx between 1.6 and 1.7. Unlike kerneldoc.py, we use
-# sphinx.util.nodes.nested_parse_with_titles() rather than the
-# plain self.state.nested_parse(), and so we can drop the saving
-# of title_styles and section_level that kerneldoc.py does,
-# because nested_parse_with_titles() does that for us.
-if Use_SSI:
-with switch_source_input(self.state, rstlist):
-nested_parse_with_titles(self.state, rstlist, node)
-else:
-save = self.state.memo.reporter
-self.state.memo.reporter = AutodocReporter(
-rstlist, self.state.memo.reporter)
-try:
-nested_parse_with_titles(self.state, rstlist, node)
-finally:
-self.state.memo.reporter = save
+with switch_source_input(self.state, rstlist):
+nested_parse_with_titles(self.state, rstlist, node)
def setup(app):
diff --git a/pythondeps.toml b/pythondeps.toml
index 9c16602d303..bc656376caa 100644
--- a/pythondeps.toml
+++ b/pythondeps.toml
@@ -23,7 +23,7 @@ meson = { accepted = ">=0.63.0", installed = "1.2.3", canary
= "meson" }
[docs]
# Please keep the installed versions in sync with docs/requirements.txt
-sphinx = { accepted = ">=1.6", installed = "5.3.0", canary = "sphinx-build" }
+sphinx = { accepted = ">=3.4.3", installed = "5.3.0", canary = "sphinx-build" }
sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.1.1" }
[avocado]
--
2.45.0
[PATCH 0/2] docs/python: bump minimum Sphinx version
With recent deprecations, we can advance our minimum sphinx version safely. This is heavily motivated by new qapidoc work which is much easier to maintain cross-version compatibility for - see difficulties in our dbus documentation which only works on sphinx >= 4. We can only guarantee >= 3.4.3 now, but that's still vastly easier than maintaining compatibility all the way back to 1.x. GitLab: https://gitlab.com/jsnow/qemu/-/pipelines/1357902509 (failures appear to be unrelated to the series.) John Snow (2): Python: bump minimum sphinx version to 3.4.3 docs: remove Sphinx 1.x compatibility code docs/conf.py | 7 +++ docs/sphinx/hxtool.py| 21 - docs/sphinx/kerneldoc.py | 38 -- docs/sphinx/kernellog.py | 28 docs/sphinx/qapidoc.py | 29 +++-- pythondeps.toml | 2 +- 6 files changed, 23 insertions(+), 102 deletions(-) delete mode 100644 docs/sphinx/kernellog.py -- 2.45.0
[PATCH 2/2] docs: remove Sphinx 1.x compatibility code
In general, the Use_SSI workaround is no longer needed, and neither is
the pre-1.6 logging shim for kerneldoc.
Signed-off-by: John Snow
---
docs/sphinx/hxtool.py| 21 -
docs/sphinx/kerneldoc.py | 38 --
docs/sphinx/kernellog.py | 28
3 files changed, 16 insertions(+), 71 deletions(-)
delete mode 100644 docs/sphinx/kernellog.py
diff --git a/docs/sphinx/hxtool.py b/docs/sphinx/hxtool.py
index 3729084a36c..a84723be19e 100644
--- a/docs/sphinx/hxtool.py
+++ b/docs/sphinx/hxtool.py
@@ -24,16 +24,10 @@
from docutils.statemachine import ViewList
from docutils.parsers.rst import directives, Directive
from sphinx.errors import ExtensionError
+from sphinx.util.docutils import switch_source_input
from sphinx.util.nodes import nested_parse_with_titles
import sphinx
-# Sphinx up to 1.6 uses AutodocReporter; 1.7 and later
-# use switch_source_input. Check borrowed from kerneldoc.py.
-Use_SSI = sphinx.__version__[:3] >= '1.7'
-if Use_SSI:
-from sphinx.util.docutils import switch_source_input
-else:
-from sphinx.ext.autodoc import AutodocReporter
__version__ = '1.0'
@@ -185,16 +179,9 @@ def run(self):
# of title_styles and section_level that kerneldoc.py does,
# because nested_parse_with_titles() does that for us.
def do_parse(self, result, node):
-if Use_SSI:
-with switch_source_input(self.state, result):
-nested_parse_with_titles(self.state, result, node)
-else:
-save = self.state.memo.reporter
-self.state.memo.reporter = AutodocReporter(result,
self.state.memo.reporter)
-try:
-nested_parse_with_titles(self.state, result, node)
-finally:
-self.state.memo.reporter = save
+with switch_source_input(self.state, result):
+nested_parse_with_titles(self.state, result, node)
+
def setup(app):
""" Register hxtool-doc directive with Sphinx"""
diff --git a/docs/sphinx/kerneldoc.py b/docs/sphinx/kerneldoc.py
index 72c403a7379..3aa972f2e89 100644
--- a/docs/sphinx/kerneldoc.py
+++ b/docs/sphinx/kerneldoc.py
@@ -38,20 +38,14 @@
from docutils.statemachine import ViewList
from docutils.parsers.rst import directives, Directive
-#
-# AutodocReporter is only good up to Sphinx 1.7
-#
import sphinx
+from sphinx.util import logging
+from sphinx.util.docutils import switch_source_input
-Use_SSI = sphinx.__version__[:3] >= '1.7'
-if Use_SSI:
-from sphinx.util.docutils import switch_source_input
-else:
-from sphinx.ext.autodoc import AutodocReporter
-
-import kernellog
__version__ = '1.0'
+logger = logging.getLogger('kerneldoc')
+
class KernelDocDirective(Directive):
"""Extract kernel-doc comments from the specified file"""
@@ -111,8 +105,7 @@ def run(self):
cmd += [filename]
try:
-kernellog.verbose(env.app,
- 'calling kernel-doc \'%s\'' % (" ".join(cmd)))
+logger.verbose('calling kernel-doc \'%s\'' % (" ".join(cmd)))
p = subprocess.Popen(cmd, stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
out, err = p.communicate()
@@ -122,8 +115,10 @@ def run(self):
if p.returncode != 0:
sys.stderr.write(err)
-kernellog.warn(env.app,
- 'kernel-doc \'%s\' failed with return code %d'
% (" ".join(cmd), p.returncode))
+logger.warning(
+'kernel-doc \'%s\' failed with return code %d' %
+(" ".join(cmd), p.returncode)
+)
return [nodes.error(None, nodes.paragraph(text = "kernel-doc
missing"))]
elif env.config.kerneldoc_verbosity > 0:
sys.stderr.write(err)
@@ -149,22 +144,13 @@ def run(self):
return node.children
except Exception as e: # pylint: disable=W0703
-kernellog.warn(env.app, 'kernel-doc \'%s\' processing failed with:
%s' %
+logger.warning('kernel-doc \'%s\' processing failed with: %s' %
(" ".join(cmd), str(e)))
return [nodes.error(None, nodes.paragraph(text = "kernel-doc
missing"))]
def do_parse(self, result, node):
-if Use_SSI:
-with switch_source_input(self.state, result):
-self.state.nested_parse(result, 0, node, match_titles=1)
-else:
-save = self.state.memo.title_styles,
self.state.memo.section_level, self.state.memo.reporter
-self.state.memo.reporter = AutodocReporter(result,
self.state.memo.reporter)
-self.state.memo.title_styles, self.state.memo.section_level = [], 0
-try:
-self.state.neste
Re: [PATCH 3/4] iotests: Change imports for Python 3.13
On Tue, Jul 2, 2024 at 7:52 AM Nir Soffer wrote: > On Thu, Jun 27, 2024 at 2:23 AM John Snow wrote: > > > > Python 3.13 isn't out yet, but it's in beta and Fedora is ramping up to > > make it the default system interpreter for Fedora 41. > > > > They moved our cheese for where ContextManager lives; add a conditional > > to locate it while we support both pre-3.9 and 3.13+. > > > > Signed-off-by: John Snow > > --- > > tests/qemu-iotests/testenv.py| 7 ++- > > tests/qemu-iotests/testrunner.py | 9 ++--- > > 2 files changed, 12 insertions(+), 4 deletions(-) > > > > diff --git a/tests/qemu-iotests/testenv.py > b/tests/qemu-iotests/testenv.py > > index 588f30a4f14..96d69e56963 100644 > > --- a/tests/qemu-iotests/testenv.py > > +++ b/tests/qemu-iotests/testenv.py > > @@ -25,7 +25,12 @@ > > import random > > import subprocess > > import glob > > -from typing import List, Dict, Any, Optional, ContextManager > > +from typing import List, Dict, Any, Optional > > + > > +if sys.version_info >= (3, 9): > > +from contextlib import AbstractContextManager as ContextManager > > +else: > > +from typing import ContextManager > > It can be cleaner to add a compat module hiding the details so the > entire project > can have a single instance of this. Other code will just use: > > from compat import ContextManager > If there were more than two uses, I'd consider it. As it stands, a compat.py module with just one import conditional in it doesn't seem worth the hassle. Are there more cases of compatibility goop inside iotests that need to be factored out to make it worth it? --js
Re: [PATCH] sphinx/qapidoc: Fix to generate doc for explicit, unboxed arguments
On Fri, Jun 28, 2024, 7:28 AM Markus Armbruster wrote:
> When a command's arguments are specified as an explicit type T,
> generated documentation points to the members of T.
>
> Example:
>
> ##
> # @announce-self:
> #
> # Trigger generation of broadcast RARP frames to update network
> [...]
> ##
> { 'command': 'announce-self', 'boxed': true,
> 'data' : 'AnnounceParameters'}
>
> generates
>
> "announce-self" (Command)
> -
>
> Trigger generation of broadcast RARP frames to update network
> [...]
>
> Arguments
> ~
>
> The members of "AnnounceParameters"
>
> Except when the command takes its arguments unboxed , i.e. it doesn't
> have 'boxed': true, we generate *nothing*. A few commands have a
> reference in their doc comment to compensate, but most don't.
>
> Example:
>
> ##
> # @blockdev-snapshot-sync:
> #
> # Takes a synchronous snapshot of a block device.
> #
> # For the arguments, see the documentation of BlockdevSnapshotSync.
> [...]
> ##
> { 'command': 'blockdev-snapshot-sync',
> 'data': 'BlockdevSnapshotSync',
> 'allow-preconfig': true }
>
> generates
>
> "blockdev-snapshot-sync" (Command)
> ~~
>
> Takes a synchronous snapshot of a block device.
>
> For the arguments, see the documentation of BlockdevSnapshotSync.
> [...]
>
> Same for event data.
>
> Fix qapidoc.py to generate the reference regardless of boxing. Delete
> now redundant references in the doc comments.
>
> Fixes: 4078ee5469e5 (docs/sphinx: Add new qapi-doc Sphinx extension)
> Cc: qemu-sta...@nongnu.org
> Signed-off-by: Markus Armbruster
>
LGTM
Didn't test because my branch is a mess at the moment but I don't see
anything obviously wrong.
If it breaks something I'm sure I'll find out, so
Reviewed-by: John Snow
---
> docs/sphinx/qapidoc.py | 12 +---
> qapi/block-core.json | 7 ---
> 2 files changed, 5 insertions(+), 14 deletions(-)
>
> diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> index f270b494f0..aacb2cd721 100644
> --- a/docs/sphinx/qapidoc.py
> +++ b/docs/sphinx/qapidoc.py
> @@ -219,15 +219,15 @@ def _nodes_for_enum_values(self, doc):
> section += dlnode
> return [section]
>
> -def _nodes_for_arguments(self, doc, boxed_arg_type):
> +def _nodes_for_arguments(self, doc, arg_type):
> """Return list of doctree nodes for the arguments section"""
> -if boxed_arg_type:
> +if arg_type and not arg_type.is_implicit():
> assert not doc.args
> section = self._make_section('Arguments')
> dlnode = nodes.definition_list()
> dlnode += self._make_dlitem(
> [nodes.Text('The members of '),
> - nodes.literal('', boxed_arg_type.name)],
> + nodes.literal('', arg_type.name)],
> None)
> section += dlnode
> return [section]
> @@ -332,8 +332,7 @@ def visit_command(self, name, info, ifcond, features,
> arg_type,
>allow_preconfig, coroutine):
> doc = self._cur_doc
> self._add_doc('Command',
> - self._nodes_for_arguments(doc,
> -arg_type if boxed else
> None)
> + self._nodes_for_arguments(doc, arg_type)
>+ self._nodes_for_features(doc)
>+ self._nodes_for_sections(doc)
>+ self._nodes_for_if_section(ifcond))
> @@ -341,8 +340,7 @@ def visit_command(self, name, info, ifcond, features,
> arg_type,
> def visit_event(self, name, info, ifcond, features, arg_type, boxed):
> doc = self._cur_doc
> self._add_doc('Event',
> - self._nodes_for_arguments(doc,
> -arg_type if boxed else
> None)
> + self._nodes_for_arguments(doc, arg_type)
>+ self._nodes_for_features(doc)
>+ self._nodes_for_sections(doc)
>+ self._nodes_for_if_section(ifcond))
> diff --git a/qapi/block-core.json b/qapi/block-core.json
> index df5e07debd..c5cb0c5d56 100644
> --- a/qapi/block-core.json
> +++ b/qapi/block-core.json
> @@ -1675,8 +1675,6 @@
> #
> # Takes a synchronous snapshot of a block device.
> #
> -# For the arguments, see the documentation of BlockdevSnapshotSync.
Re: [PATCH 3/4] iotests: Change imports for Python 3.13
Ping - happy to merge this series myself but didn't wanna change iotests without at least an ack from the lord of that castle. On Wed, Jun 26, 2024, 7:22 PM John Snow wrote: > Python 3.13 isn't out yet, but it's in beta and Fedora is ramping up to > make it the default system interpreter for Fedora 41. > > They moved our cheese for where ContextManager lives; add a conditional > to locate it while we support both pre-3.9 and 3.13+. > > Signed-off-by: John Snow > --- > tests/qemu-iotests/testenv.py| 7 ++- > tests/qemu-iotests/testrunner.py | 9 ++--- > 2 files changed, 12 insertions(+), 4 deletions(-) > > diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py > index 588f30a4f14..96d69e56963 100644 > --- a/tests/qemu-iotests/testenv.py > +++ b/tests/qemu-iotests/testenv.py > @@ -25,7 +25,12 @@ > import random > import subprocess > import glob > -from typing import List, Dict, Any, Optional, ContextManager > +from typing import List, Dict, Any, Optional > + > +if sys.version_info >= (3, 9): > +from contextlib import AbstractContextManager as ContextManager > +else: > +from typing import ContextManager > > DEF_GDB_OPTIONS = 'localhost:12345' > > diff --git a/tests/qemu-iotests/testrunner.py > b/tests/qemu-iotests/testrunner.py > index 7b322272e92..2e236c8fa39 100644 > --- a/tests/qemu-iotests/testrunner.py > +++ b/tests/qemu-iotests/testrunner.py > @@ -27,11 +27,14 @@ > import shutil > import sys > from multiprocessing import Pool > -from typing import List, Optional, Any, Sequence, Dict, \ > -ContextManager > - > +from typing import List, Optional, Any, Sequence, Dict > from testenv import TestEnv > > +if sys.version_info >= (3, 9): > +from contextlib import AbstractContextManager as ContextManager > +else: > +from typing import ContextManager > + > > def silent_unlink(path: Path) -> None: > try: > -- > 2.45.0 > >
Re: [PATCH v2 10/21] qapi: convert "Note" sections to plain rST
On Fri, Jun 28, 2024, 5:52 AM Markus Armbruster wrote: > John Snow writes: > > > We do not need a dedicated section for notes. By eliminating a specially > > parsed section, these notes can be treated as normal rST paragraphs in > > the new QMP reference manual, and can be placed and styled much more > > flexibly. > > > > Convert all existing "Note" and "Notes" sections to pure rST. As part of > > the conversion, capitalize the first letter of each sentence and add > > trailing punctuation where appropriate to ensure notes look sensible and > > consistent in rendered HTML documentation. Markup is also re-aligned to > > the de-facto standard of 3 spaces for directives. > > > > Update docs/devel/qapi-code-gen.rst to reflect the new paradigm, and > > update the QAPI parser to prohibit "Note" sections while suggesting a > > new syntax. The exact formatting to use is a matter of taste, but a good > > candidate is simply: > > > > .. note:: lorem ipsum ... > >... dolor sit amet ... > >... consectetur adipiscing elit ... > > > > ... but there are other choices, too. The Sphinx readthedocs theme > > offers theming for the following forms (capitalization unimportant); all > > are adorned with a (!) symbol () in the title bar for rendered HTML > > docs. > > > > See > > > https://sphinx-rtd-theme.readthedocs.io/en/stable/demo/demo.html#admonitions > > for examples of each directive/admonition in use. > > > > These are rendered in orange: > > > > .. Attention:: ... > > .. Caution:: ... > > .. WARNING:: ... > > > > These are rendered in red: > > > > .. DANGER:: ... > > .. Error:: ... > > > > These are rendered in green: > > > > .. Hint:: ... > > .. Important:: ... > > .. Tip:: ... > > > > These are rendered in blue: > > > > .. Note:: ... > > .. admonition:: custom title > > > >admonition body text > > > > This patch uses ".. note::" almost everywhere, with just two "caution" > > directives. Several instances of "Notes:" have been converted to merely > > ".. note::" where appropriate, but ".. admonition:: notes" is used in a > > few places where we had an ordered list of multiple notes that would not > > make sense as standalone/separate admonitions. > > I looked for hunks that don't 1:1 replace "Note:" or "Notes:" by > ".. note::." Findings: > > * Two hunks replace by ".. caution::" instead. Commit message got it. > Good. > > * Four hunks replace by ".. admonition:: notes", one of them as a test. > Commit message got it. Good. > > * Three hunks split "Notes:" into multiple ".. note::". Good, but could > be mentioned in commit message. > I meant to imply it when discussing when admonition was used, but yeah. > * Two hunks drop "Note:", changing it into paragraph. The paragraph > merges into the preceding "Example" section. Good, but should be > mentioned in the commit message, or turned into a separate patch. > Eh. we got enough commits. I think it's helpful to keep the whole conversion in one giant bang so that the diff is helpful in illustrating all of the different types of conversions. (In fact, even though I split out Example conversion for your sake in review, I think it'd be helpful to squash them together on merge for the same exact reason.) Let's just amend the commit message. > * One hunk adjusts a test case for the removal of the "Note:" tag. > Good, but could be mentioned in the commit message. > > Perhaps tweak the paragraph above: > > This patch uses ".. note::" almost everywhere, with just two "caution" > directives. Several instances of "Notes:" have been converted to > merely ".. note::", or multiple ".. note::" where appropriate. > ".. admonition:: notes" is used in a few places where we had an > ordered list of multiple notes that would not make sense as > standalone/separate admonitions. Two "Note:" following "Example:" > have been turned into ordinary paragraphs within the example. > > Okay? > Yep, suits me fine. > > NOTE: Because qapidoc.py does not attempt to preserve source ordering of > > sections, the conversion of Notes from a "tagged section" to an > > "untagged section" means that rendering order for some notes *may > > change* as a result of this patch. The forthcoming qapidoc.py rewrite > > strictly preserves source ordering in the rendered documentation, so > > this issue will be rectified in the new generator. > > > > Signed-off-by: John Snow > > Acked-by: Stefan Hajnoczi [for block*.json] > > I dislike the indentation changes, and may revert them in my tree. > Would you take a patch adjusting the indent later, or will you then tell me it's not worth the git blame fuzz? :) > Reviewed-by: Markus Armbruster > >
Re: [PATCH v2 07/21] docs/qapidoc: fix nested parsing under untagged sections
On Fri, Jun 28, 2024, 11:10 AM John Snow wrote: > > > On Fri, Jun 28, 2024, 3:55 AM Markus Armbruster wrote: > >> John Snow writes: >> >> > Sphinx does not like sections without titles, because it wants to >> > convert every section into a reference. When there is no title, it >> > struggles to do this and transforms the tree inproperly. >> > >> > Depending on the rST used, this may result in an assertion error deep in >> > the docutils HTMLWriter. >> > >> > (Observed when using ".. admonition:: Notes" under such a section - When >> > this is transformed with its own element, Sphinx is fooled into >> > believing this title belongs to the section and incorrect mutates the >> > docutils tree, leading to errors during rendering time.) >> > >> > When parsing an untagged section (free paragraphs), skip making a hollow >> > section and instead append the parse results to the prior section. >> > >> > Many Bothans died to bring us this information. >> > >> > Signed-off-by: John Snow >> > Acked-by: Markus Armbruster >> >> Generated HTML changes, but the diff is hard to review due to id >> attribute changes all over the place. >> >> Generated qemu-ga-ref.7 also changes: >> >> diff -rup old/qemu-ga-ref.7 new/qemu-ga-ref.7 >> --- old/qemu-ga-ref.7 2024-06-27 10:42:21.466096276 +0200 >> +++ new/qemu-ga-ref.7 2024-06-27 10:45:36.502414099 +0200 >> @@ -397,6 +397,7 @@ shutdown request, with no guarantee of s >> .B \fBmode\fP: \fBstring\fP (optional) >> \(dqhalt\(dq, \(dqpowerdown\(dq (default), or \(dqreboot\(dq >> .UNINDENT >> +.sp >> This command does NOT return a response on success. Success >> condition is indicated by the VM exiting with a zero exit status or, >> when running with \-\-no\-shutdown, by issuing the query\-status QMP >> @@ -1348,6 +1349,7 @@ the new password entry string, base64 en >> .B \fBcrypted\fP: \fBboolean\fP >> true if password is already crypt()d, false if raw >> .UNINDENT >> +.sp >> If the \fBcrypted\fP flag is true, it is the caller\(aqs >> responsibility to >> ensure the correct crypt() encryption scheme is used. This command >> does not attempt to interpret or report on the encryption scheme. >> >> We add vertical space. Visible when viewed with man. Looks like an >> improvement to me. >> >> Here's the first of these two spots in HTML: >> >> - >> -> class="docutils literal notranslate">> class="pre">guest-shutdown (Command)> href="#qapidoc-31" title="Permalink to this heading"> >> + >> +> class="docutils literal notranslate">> class="pre">guest-shutdown (Command)> href="#qapidoc-30" title="Permalink to this heading"> >> Initiate guest-activated shutdown. Note: this is an asynchronous >> shutdown request, with no guarantee of successful shutdown. >> >> @@ -502,22 +502,20 @@ shutdown request, with no guarantee of s >> >> >> >> - >> This command does NOT return a response on success. Success >> condition is indicated by the VM exiting with a zero exit status or, >> when running with –no-shutdown, by issuing the query-status QMP >> command to confirm the VM status is “shutdown”. >> - >> - >> -Since >> + >> +Since >> 0.15.0 >> >> >> >> The id changes muddy the waters. With them manually removed: >> >> >> > class="docutils literal notranslate">> class="pre">guest-shutdown (Command)> href="#qapidoc-31" title="Permalink to this heading"> >> Initiate guest-activated shutdown. Note: this is an asynchronous >> shutdown request, with no guarantee of successful shutdown. >> >> @@ -502,22 +502,20 @@ shutdown request, with no guarantee of s >> >> >> >> - >> This command does NOT return a response on success. Success >> condition is indicated by the VM exiting with a zero exit status or, >> when running with –no-shutdown, by issuing the query-status QMP >> command to confirm the VM status is “shutdown”. >> - >> >> Since >> 0.15.0 >> >> >> >> Makes no visual difference in my browser. >> >> Do these differences match your expectations? >> > > Yep! > > It does change the output just a little, but Sphinx really doesn't like > title-less sections. > > I thought the change looked fine, and I'm still planning on removing this > old generator anyway, so... > Oh, pro tip: try using the xml builder before and after for a cleaner comparison. --js >
Re: [PATCH v2 15/21] docs/qapidoc: create qmp-example directive
On Fri, Jun 28, 2024, 9:24 AM Markus Armbruster wrote:
> John Snow writes:
>
> > This is a directive that creates a syntactic sugar for creating
> > "Example" boxes very similar to the ones already used in the bitmaps.rst
> > document, please see e.g.
> >
> https://www.qemu.org/docs/master/interop/bitmaps.html#creation-block-dirty-bitmap-add
> >
> > In its simplest form, when a custom title is not needed or wanted, and
> > the example body is *solely* a QMP example:
> >
> > ```
> > .. qmp-example::
> >
> >{body}
> > ```
> >
> > is syntactic sugar for:
> >
> > ```
> > .. admonition:: Example:
> >
> >.. code-block:: QMP
> >
> > {body}
> > ```
> >
> > When a custom, plaintext title that describes the example is desired,
> > this form:
> >
> > ```
> > .. qmp-example::
> >:title: Defrobnification
> >
> >{body}
> > ```
> >
> > Is syntactic sugar for:
> >
> > ```
> > .. admonition:: Example: Defrobnification
> >
> >.. code-block:: QMP
> >
> > {body}
> > ```
> >
> > Lastly, when Examples are multi-step processes that require non-QMP
> > exposition, have lengthy titles, or otherwise involve prose with rST
> > markup (lists, cross-references, etc), the most complex form:
> >
> > ```
> > .. qmp-example::
> >:annotated:
> >
> >This example shows how to use `foo-command`::
> >
> > {body}
> > ```
> >
> > Is desugared to:
> >
> > ```
> > .. admonition:: Example:
> >
> >This example shows how to use `foo-command`::
> >
> > {body}
> >
> >For more information, please see `frobnozz`.
> > ```
>
^ Whoops, added prose in the desugar block without modifying the original.
> Can we combine the latter two? Like this:
>
> .. qmp-example::
> :title: Defrobnification
> :annotated:
>
> This example shows how to use `foo-command`::
>
>{body}
>
Yes! I only didn't use that form in the series because splitting longer
Examples into title and prose felt like an editorial decision, but
absolutely you can use both.
> > The primary benefit here being documentation source consistently using
> > the same directive for all forms of examples to ensure consistent visual
> > styling, and ensuring all relevant prose is visually grouped alongside
> > the code literal block.
> >
> > Note that as of this commit, the code-block rST syntax "::" does not
> > apply QMP highlighting; you would need to use ".. code-block:: QMP". The
> > very next commit changes this behavior to assume all "::" code blocks
> > within this directive are QMP blocks.
> >
> > Signed-off-by: John Snow
> > ---
> > docs/sphinx/qapidoc.py | 60 --
> > 1 file changed, 58 insertions(+), 2 deletions(-)
>
> No tests? Hmm, I see you convert existing tests in PATCH 19-21. While
> that works, test coverage now would make it easier to see how each patch
> affects doc generator output.
>
Mmm. Do you want me to move the test changes up to this patch ... ?
> > diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> > index 43dd99e21e6..a2fa05fc491 100644
> > --- a/docs/sphinx/qapidoc.py
> > +++ b/docs/sphinx/qapidoc.py
> > @@ -27,16 +27,19 @@
> > import os
> > import re
> > import textwrap
> > +from typing import List
> >
> > from docutils import nodes
> > -from docutils.parsers.rst import Directive, directives
> > +from docutils.parsers.rst import directives
> > from docutils.statemachine import ViewList
> > from qapi.error import QAPIError, QAPISemError
> > from qapi.gen import QAPISchemaVisitor
> > from qapi.schema import QAPISchema
> >
> > import sphinx
> > +from sphinx.directives.code import CodeBlock
> > from sphinx.errors import ExtensionError
> > +from sphinx.util.docutils import SphinxDirective
> > from sphinx.util.nodes import nested_parse_with_titles
> >
> >
> > @@ -494,7 +497,7 @@ def visit_module(self, name):
> > super().visit_module(name)
> >
> >
> > -class NestedDirective(Directive):
> > +class NestedDirective(SphinxDirective):
>
> What is this about?
>
Hmm. Strictly it's for access to sphinx configuration which I use only in
the next patch, but practically I suspect if I don't change it *here* that
the multiple inheritance from CodeBloc
Re: [PATCH v2 14/21] docs/qapidoc: factor out do_parse()
On Fri, Jun 28, 2024, 9:09 AM Markus Armbruster wrote: > John Snow writes: > > > Factor out the compatibility parser helper so it can be shared by other > > directives. > > Suggest "Factor out the compatibility parser helper into a base class, > so it can be shared by other directives." Sure. Haven't read the other mails yet. I'll make the change if you want a v3, otherwise feel free to edit. > > > > Signed-off-by: John Snow > > --- > > docs/sphinx/qapidoc.py | 64 +++--- > > 1 file changed, 35 insertions(+), 29 deletions(-) > > > > diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py > > index efcd84656fa..43dd99e21e6 100644 > > --- a/docs/sphinx/qapidoc.py > > +++ b/docs/sphinx/qapidoc.py > > @@ -494,7 +494,41 @@ def visit_module(self, name): > > super().visit_module(name) > > > > > > -class QAPIDocDirective(Directive): > > +class NestedDirective(Directive): > > +def run(self): > > +raise NotImplementedError > > Should this class be abstract? > It could be ... *sneezes* I plan to delete it by the end of the qapi-domain series anyway, or perhaps I could even delete it *before* with a dedicated "require sphinx >= 3.x" miniseries. Actually, that's probably a really good idea... > > + > > +def do_parse(self, rstlist, node): > > +""" > > +Parse rST source lines and add them to the specified node > > + > > +Take the list of rST source lines rstlist, parse them as > > +rST, and add the resulting docutils nodes as children of node. > > +The nodes are parsed in a way that allows them to include > > +subheadings (titles) without confusing the rendering of > > +anything else. > > +""" > > +# This is from kerneldoc.py -- it works around an API change in > > +# Sphinx between 1.6 and 1.7. Unlike kerneldoc.py, we use > > +# sphinx.util.nodes.nested_parse_with_titles() rather than the > > +# plain self.state.nested_parse(), and so we can drop the saving > > +# of title_styles and section_level that kerneldoc.py does, > > +# because nested_parse_with_titles() does that for us. > > +if USE_SSI: > > +with switch_source_input(self.state, rstlist): > > +nested_parse_with_titles(self.state, rstlist, node) > > +else: > > +save = self.state.memo.reporter > > +self.state.memo.reporter = AutodocReporter( > > +rstlist, self.state.memo.reporter > > +) > > +try: > > +nested_parse_with_titles(self.state, rstlist, node) > > +finally: > > +self.state.memo.reporter = save > > + > > + > > +class QAPIDocDirective(NestedDirective): > > """Extract documentation from the specified QAPI .json file""" > > > > required_argument = 1 > > @@ -532,34 +566,6 @@ def run(self): > > # so they are displayed nicely to the user > > raise ExtensionError(str(err)) from err > > > > -def do_parse(self, rstlist, node): > > -"""Parse rST source lines and add them to the specified node > > - > > -Take the list of rST source lines rstlist, parse them as > > -rST, and add the resulting docutils nodes as children of node. > > -The nodes are parsed in a way that allows them to include > > -subheadings (titles) without confusing the rendering of > > -anything else. > > -""" > > -# This is from kerneldoc.py -- it works around an API change in > > -# Sphinx between 1.6 and 1.7. Unlike kerneldoc.py, we use > > -# sphinx.util.nodes.nested_parse_with_titles() rather than the > > -# plain self.state.nested_parse(), and so we can drop the saving > > -# of title_styles and section_level that kerneldoc.py does, > > -# because nested_parse_with_titles() does that for us. > > -if USE_SSI: > > -with switch_source_input(self.state, rstlist): > > -nested_parse_with_titles(self.state, rstlist, node) > > -else: > > -save = self.state.memo.reporter > > -self.state.memo.reporter = AutodocReporter( > > -rstlist, self.state.memo.reporter > > -) > > -try: > > -nested_parse_with_titles(self.state, rstlist, node) > > -finally: > > -self.state.memo.reporter = save > > - > > > > def setup(app): > > """Register qapi-doc directive with Sphinx""" > > Reviewed-by: Markus Armbruster > >
Re: [PATCH v2 07/21] docs/qapidoc: fix nested parsing under untagged sections
On Fri, Jun 28, 2024, 3:55 AM Markus Armbruster wrote: > John Snow writes: > > > Sphinx does not like sections without titles, because it wants to > > convert every section into a reference. When there is no title, it > > struggles to do this and transforms the tree inproperly. > > > > Depending on the rST used, this may result in an assertion error deep in > > the docutils HTMLWriter. > > > > (Observed when using ".. admonition:: Notes" under such a section - When > > this is transformed with its own element, Sphinx is fooled into > > believing this title belongs to the section and incorrect mutates the > > docutils tree, leading to errors during rendering time.) > > > > When parsing an untagged section (free paragraphs), skip making a hollow > > section and instead append the parse results to the prior section. > > > > Many Bothans died to bring us this information. > > > > Signed-off-by: John Snow > > Acked-by: Markus Armbruster > > Generated HTML changes, but the diff is hard to review due to id > attribute changes all over the place. > > Generated qemu-ga-ref.7 also changes: > > diff -rup old/qemu-ga-ref.7 new/qemu-ga-ref.7 > --- old/qemu-ga-ref.7 2024-06-27 10:42:21.466096276 +0200 > +++ new/qemu-ga-ref.7 2024-06-27 10:45:36.502414099 +0200 > @@ -397,6 +397,7 @@ shutdown request, with no guarantee of s > .B \fBmode\fP: \fBstring\fP (optional) > \(dqhalt\(dq, \(dqpowerdown\(dq (default), or \(dqreboot\(dq > .UNINDENT > +.sp > This command does NOT return a response on success. Success > condition is indicated by the VM exiting with a zero exit status or, > when running with \-\-no\-shutdown, by issuing the query\-status QMP > @@ -1348,6 +1349,7 @@ the new password entry string, base64 en > .B \fBcrypted\fP: \fBboolean\fP > true if password is already crypt()d, false if raw > .UNINDENT > +.sp > If the \fBcrypted\fP flag is true, it is the caller\(aqs > responsibility to > ensure the correct crypt() encryption scheme is used. This command > does not attempt to interpret or report on the encryption scheme. > > We add vertical space. Visible when viewed with man. Looks like an > improvement to me. > > Here's the first of these two spots in HTML: > > - > - class="docutils literal notranslate"> class="pre">guest-shutdown (Command) href="#qapidoc-31" title="Permalink to this heading"> > + > + class="docutils literal notranslate"> class="pre">guest-shutdown (Command) href="#qapidoc-30" title="Permalink to this heading"> > Initiate guest-activated shutdown. Note: this is an asynchronous > shutdown request, with no guarantee of successful shutdown. > > @@ -502,22 +502,20 @@ shutdown request, with no guarantee of s > > > > - > This command does NOT return a response on success. Success > condition is indicated by the VM exiting with a zero exit status or, > when running with –no-shutdown, by issuing the query-status QMP > command to confirm the VM status is “shutdown”. > - > - > -Since > + > +Since > 0.15.0 > > > > The id changes muddy the waters. With them manually removed: > > > class="docutils literal notranslate"> class="pre">guest-shutdown (Command) href="#qapidoc-31" title="Permalink to this heading"> > Initiate guest-activated shutdown. Note: this is an asynchronous > shutdown request, with no guarantee of successful shutdown. > > @@ -502,22 +502,20 @@ shutdown request, with no guarantee of s > > > > - > This command does NOT return a response on success. Success > condition is indicated by the VM exiting with a zero exit status or, > when running with –no-shutdown, by issuing the query-status QMP > command to confirm the VM status is “shutdown”. > - > > Since > 0.15.0 > > > > Makes no visual difference in my browser. > > Do these differences match your expectations? > Yep! It does change the output just a little, but Sphinx really doesn't like title-less sections. I thought the change looked fine, and I'm still planning on removing this old generator anyway, so... >
Re: [PATCH v2 04/21] docs/qapidoc: delint a tiny portion of the module
On Fri, Jun 28, 2024, 3:29 AM Markus Armbruster wrote: > John Snow writes: > > > In a forthcoming series that adds a new QMP documentation generator, it > > will be helpful to have a linting baseline. However, there's no need to > > shuffle around the deck chairs too much, because most of this code will > > be removed once that new qapidoc generator (the "transmogrifier") is in > > place. > > > > To ease my pain: just turn off the black auto-formatter for most, but > > not all, of qapidoc.py. This will help ensure that *new* code follows a > > coding standard without bothering too much with cleaning up the existing > > code. > > > > Code that I intend to keep is still subject to the delinting beam. > > > > Signed-off-by: John Snow > > Reviewed-by: Markus Armbruster > > Not an objection, just so you know: I still see a few C0411 like 'third > party import "import sphinx" should be placed before ...' > > R-by stands. > Yeah, I think it depends on precisely where you run the script. I think because the folder is named "sphinx" that it confuses the tools in certain contexts. I'm not worried about it because we don't have an enforcement paradigm yet - I stick to my little self-test script just to make sure I'm being self-consistent, but I figured I'd worry about broader compatibility later when I reshuffle the deck chairs for qapi.
Re: [PATCH v2 05/21] qapi/parser: preserve indentation in QAPIDoc sections
On Thu, Jun 27, 2024, 2:25 AM Markus Armbruster wrote: > John Snow writes: > > > Change get_doc_indented() to preserve indentation on all subsequent text > > lines, and create a compatibility dedent() function for qapidoc.py that > > removes indentation the same way get_doc_indented() did. > > > > This is being done for the benefit of a new qapidoc generator which > > requires that indentation in argument and features sections are > > preserved. > > > > Prior to this patch, a section like this: > > > > ``` > > @name: lorem ipsum > >dolor sit amet > > consectetur adipiscing elit > > ``` > > > > would have its body text be parsed into: > > > > ``` > > lorem ipsum > > dolor sit amet > > consectetur adipiscing elit > > ``` > > > > We want to preserve the indentation for even the first body line so that > > the entire block can be parsed directly as rST. This patch would now > > parse that segment into: > > > > ``` > > lorem ipsum > >dolor sit amet > > consectetur adipiscing elit > > ``` > > > > This is helpful for formatting arguments and features as field lists in > > rST, where the new generator will format this information as: > > > > ``` > > :arg type name: lorem ipsum > >dolor sit amet > > consectetur apidiscing elit > > ``` > > > > ...and can be formed by the simple concatenation of the field list > > construct and the body text. The indents help preserve the continuation > > of a block-level element, and further allow the use of additional rST > > block-level constructs such as code blocks, lists, and other such > > markup. > > > > This understandably breaks the existing qapidoc.py; so a new function is > > added there to dedent the text for compatibility. Once the new generator > > is merged, this function will not be needed any longer and can be > > dropped. > > I'll restore this paragraph if you don't mind: > > I verified this patch changes absolutely nothing by comparing the > md5sums of the QMP ref html pages both before and after the change, so > it's certified inert. QAPI test output has been updated to reflect the > new strategy of preserving indents for rST. > Nope, misunderstood the instruction. Go right ahead. > > Signed-off-by: John Snow > > [Edited commit message and code comments per review --js] > > And I'll drop this line. > Vestigial remnants of a more optimistic time. > > Reviewed-by: Markus Armbruster > >
Re: [PATCH v2 03/21] docs/qapidoc: remove unused intersperse function
On Thu, Jun 27, 2024, 2:02 AM Markus Armbruster wrote: > John Snow writes: > > > This function has been unused since fd62bff901b. > > > > Signed-off-by: John Snow > > Reviewed-by: Markus Armbruster > > I assume you won't mind me adding a bit of polish: "since commit > fd62bff901b (sphinx/qapidoc: Drop code to generate doc for simple union > tag)". > Please be my guest! --js (Looking at it, that's a very weird phrase, isn't it?) >
[PATCH 4/4] python: enable testing for 3.13
Python 3.13 is in beta and Fedora 41 is preparing to make it the default system interpreter; enable testing for it. (In the event problems develop prior to release, it should only impact the check-python-tox job, which is not run by default and is allowed to fail.) Signed-off-by: John Snow --- python/setup.cfg | 3 ++- tests/docker/dockerfiles/python.docker | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/python/setup.cfg b/python/setup.cfg index 8ebd345d7ed..3b4e2cc5501 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -19,6 +19,7 @@ classifiers = Programming Language :: Python :: 3.10 Programming Language :: Python :: 3.11 Programming Language :: Python :: 3.12 +Programming Language :: Python :: 3.13 Typing :: Typed [options] @@ -184,7 +185,7 @@ multi_line_output=3 # of python available on your system to run this test. [tox:tox] -envlist = py38, py39, py310, py311, py312 +envlist = py38, py39, py310, py311, py312, py313 skip_missing_interpreters = true [testenv] diff --git a/tests/docker/dockerfiles/python.docker b/tests/docker/dockerfiles/python.docker index a3c1321190c..8f0af9ef25f 100644 --- a/tests/docker/dockerfiles/python.docker +++ b/tests/docker/dockerfiles/python.docker @@ -14,6 +14,7 @@ ENV PACKAGES \ python3.10 \ python3.11 \ python3.12 \ +python3.13 \ python3.8 \ python3.9 -- 2.45.0
[PATCH 2/4] python: Do not use pylint 3.2.4 with python 3.8
There is a bug in this version, see: https://github.com/pylint-dev/pylint/issues/9751 Signed-off-by: John Snow --- python/setup.cfg | 1 + 1 file changed, 1 insertion(+) diff --git a/python/setup.cfg b/python/setup.cfg index 48668609d3e..8ebd345d7ed 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -41,6 +41,7 @@ devel = isort >= 5.1.2 mypy >= 1.4.0 pylint >= 2.17.3 +pylint != 3.2.4; python_version<"3.9" tox >= 3.18.0 urwid >= 2.1.2 urwid-readline >= 0.13 -- 2.45.0
[PATCH 0/4] Python: Add 3.13 support, play linter whackamole
Fix some regressions in check-python-tox that have crept in since Pylint 3.x, and add Python 3.13 support to the pipeline. GitLab pipeline (before I fixed the missing DCO, but let's be honest, it can't possibly be worth re-running so many tests for just that): https://gitlab.com/jsnow/qemu/-/pipelines/1349737188 John Snow (4): python: linter changes for pylint 3.x python: Do not use pylint 3.2.4 with python 3.8 iotests: Change imports for Python 3.13 python: enable testing for 3.13 python/qemu/machine/machine.py | 1 + python/qemu/utils/qemu_ga_client.py| 2 +- python/setup.cfg | 4 +++- tests/docker/dockerfiles/python.docker | 1 + tests/qemu-iotests/testenv.py | 7 ++- tests/qemu-iotests/testrunner.py | 9 ++--- 6 files changed, 18 insertions(+), 6 deletions(-) -- 2.45.0
[PATCH 1/4] python: linter changes for pylint 3.x
New bleeding edge versions, new nits to iron out. This addresses the 'check-python-tox' optional GitLab test, while 'check-python-minreqs' saw no regressions, since it's frozen on an older version of pylint. Fixes: qemu/machine/machine.py:345:52: E0606: Possibly using variable 'sock' before assignment (possibly-used-before-assignment) qemu/utils/qemu_ga_client.py:168:4: R1711: Useless return at end of function or method (useless-return) Signed-off-by: John Snow --- python/qemu/machine/machine.py | 1 + python/qemu/utils/qemu_ga_client.py | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index f648f6af451..ebb58d5b68c 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -335,6 +335,7 @@ def binary(self) -> str: def _pre_launch(self) -> None: if self._qmp_set: +sock = None if self._monitor_address is None: self._sock_pair = socket.socketpair() os.set_inheritable(self._sock_pair[0].fileno(), True) diff --git a/python/qemu/utils/qemu_ga_client.py b/python/qemu/utils/qemu_ga_client.py index 9a665e6e990..cf0fcf9a8bb 100644 --- a/python/qemu/utils/qemu_ga_client.py +++ b/python/qemu/utils/qemu_ga_client.py @@ -174,7 +174,7 @@ def suspend(self, mode: str) -> None: # On error exception will raise except asyncio.TimeoutError: # On success command will timed out -return +pass def shutdown(self, mode: str = 'powerdown') -> None: if mode not in ['powerdown', 'halt', 'reboot']: -- 2.45.0
[PATCH 3/4] iotests: Change imports for Python 3.13
Python 3.13 isn't out yet, but it's in beta and Fedora is ramping up to make it the default system interpreter for Fedora 41. They moved our cheese for where ContextManager lives; add a conditional to locate it while we support both pre-3.9 and 3.13+. Signed-off-by: John Snow --- tests/qemu-iotests/testenv.py| 7 ++- tests/qemu-iotests/testrunner.py | 9 ++--- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py index 588f30a4f14..96d69e56963 100644 --- a/tests/qemu-iotests/testenv.py +++ b/tests/qemu-iotests/testenv.py @@ -25,7 +25,12 @@ import random import subprocess import glob -from typing import List, Dict, Any, Optional, ContextManager +from typing import List, Dict, Any, Optional + +if sys.version_info >= (3, 9): +from contextlib import AbstractContextManager as ContextManager +else: +from typing import ContextManager DEF_GDB_OPTIONS = 'localhost:12345' diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py index 7b322272e92..2e236c8fa39 100644 --- a/tests/qemu-iotests/testrunner.py +++ b/tests/qemu-iotests/testrunner.py @@ -27,11 +27,14 @@ import shutil import sys from multiprocessing import Pool -from typing import List, Optional, Any, Sequence, Dict, \ -ContextManager - +from typing import List, Optional, Any, Sequence, Dict from testenv import TestEnv +if sys.version_info >= (3, 9): +from contextlib import AbstractContextManager as ContextManager +else: +from typing import ContextManager + def silent_unlink(path: Path) -> None: try: -- 2.45.0
[PATCH v2 05/21] qapi/parser: preserve indentation in QAPIDoc sections
Change get_doc_indented() to preserve indentation on all subsequent text
lines, and create a compatibility dedent() function for qapidoc.py that
removes indentation the same way get_doc_indented() did.
This is being done for the benefit of a new qapidoc generator which
requires that indentation in argument and features sections are
preserved.
Prior to this patch, a section like this:
```
@name: lorem ipsum
dolor sit amet
consectetur adipiscing elit
```
would have its body text be parsed into:
```
lorem ipsum
dolor sit amet
consectetur adipiscing elit
```
We want to preserve the indentation for even the first body line so that
the entire block can be parsed directly as rST. This patch would now
parse that segment into:
```
lorem ipsum
dolor sit amet
consectetur adipiscing elit
```
This is helpful for formatting arguments and features as field lists in
rST, where the new generator will format this information as:
```
:arg type name: lorem ipsum
dolor sit amet
consectetur apidiscing elit
```
...and can be formed by the simple concatenation of the field list
construct and the body text. The indents help preserve the continuation
of a block-level element, and further allow the use of additional rST
block-level constructs such as code blocks, lists, and other such
markup.
This understandably breaks the existing qapidoc.py; so a new function is
added there to dedent the text for compatibility. Once the new generator
is merged, this function will not be needed any longer and can be
dropped.
Signed-off-by: John Snow
[Edited commit message and code comments per review --js]
Reviewed-by: Markus Armbruster
---
docs/sphinx/qapidoc.py | 27 ++-
scripts/qapi/parser.py | 4 ++--
tests/qapi-schema/doc-good.out | 32
3 files changed, 40 insertions(+), 23 deletions(-)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index 659e507353a..f9683444b14 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -26,6 +26,7 @@
import os
import re
+import textwrap
from docutils import nodes
from docutils.parsers.rst import Directive, directives
@@ -53,6 +54,19 @@
__version__ = "1.0"
+def dedent(text: str) -> str:
+# Adjust indentation to make description text parse as paragraph.
+
+lines = text.splitlines(True)
+if re.match(r"\s+", lines[0]):
+# First line is indented; description started on the line after
+# the name. dedent the whole block.
+return textwrap.dedent(text)
+
+# Descr started on same line. Dedent line 2+.
+return lines[0] + textwrap.dedent("".join(lines[1:]))
+
+
# Disable black auto-formatter until re-enabled:
# fmt: off
@@ -164,7 +178,7 @@ def _nodes_for_members(self, doc, what, base=None,
branches=None):
term = self._nodes_for_one_member(section.member)
# TODO drop fallbacks when undocumented members are outlawed
if section.text:
-defn = section.text
+defn = dedent(section.text)
else:
defn = [nodes.Text('Not documented')]
@@ -202,7 +216,7 @@ def _nodes_for_enum_values(self, doc):
termtext.extend(self._nodes_for_ifcond(section.member.ifcond))
# TODO drop fallbacks when undocumented members are outlawed
if section.text:
-defn = section.text
+defn = dedent(section.text)
else:
defn = [nodes.Text('Not documented')]
@@ -237,7 +251,7 @@ def _nodes_for_features(self, doc):
dlnode = nodes.definition_list()
for section in doc.features.values():
dlnode += self._make_dlitem(
-[nodes.literal('', section.member.name)], section.text)
+[nodes.literal('', section.member.name)], dedent(section.text))
seen_item = True
if not seen_item:
@@ -260,9 +274,12 @@ def _nodes_for_sections(self, doc):
continue
snode = self._make_section(section.tag)
if section.tag and section.tag.startswith('Example'):
-snode += self._nodes_for_example(section.text)
+snode += self._nodes_for_example(dedent(section.text))
else:
-self._parse_text_into_node(section.text, snode)
+self._parse_text_into_node(
+dedent(section.text) if section.tag else section.text,
+snode,
+)
nodelist.append(snode)
return nodelist
diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index 7b13a583ac1..1ef1f85b028 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -448,7 +448,7 @@ def get_doc_indented(self, doc: 'QAPIDoc') -> Optional[str]:
indent = must_match(r'\s*', line).end()
if not indent:
return line
[PATCH v2 11/21] qapi: update prose in note blocks
Where I've noticed, rephrase the note to read more fluently. Signed-off-by: John Snow Reviewed-by: Markus Armbruster --- qapi/block-core.json | 4 ++-- qga/qapi-schema.json | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index cacedfb771c..9ef23ec02ae 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -6048,9 +6048,9 @@ # # @name: the name of the internal snapshot to be created # -# .. note:: In transaction, if @name is empty, or any snapshot matching +# .. note:: In a transaction, if @name is empty or any snapshot matching #@name exists, the operation will fail. Only some image formats -#support it, for example, qcow2, and rbd. +#support it; for example, qcow2, and rbd. # # Since: 1.7 ## diff --git a/qga/qapi-schema.json b/qga/qapi-schema.json index 57598331c5c..1273d85bb5f 100644 --- a/qga/qapi-schema.json +++ b/qga/qapi-schema.json @@ -480,7 +480,7 @@ # # Returns: Number of file systems thawed by this call # -# .. note:: If return value does not match the previous call to +# .. note:: If the return value does not match the previous call to #guest-fsfreeze-freeze, this likely means some freezable filesystems #were unfrozen before this call, and that the filesystem state may #have changed before issuing this command. -- 2.45.0
[PATCH v2 10/21] qapi: convert "Note" sections to plain rST
We do not need a dedicated section for notes. By eliminating a specially parsed section, these notes can be treated as normal rST paragraphs in the new QMP reference manual, and can be placed and styled much more flexibly. Convert all existing "Note" and "Notes" sections to pure rST. As part of the conversion, capitalize the first letter of each sentence and add trailing punctuation where appropriate to ensure notes look sensible and consistent in rendered HTML documentation. Markup is also re-aligned to the de-facto standard of 3 spaces for directives. Update docs/devel/qapi-code-gen.rst to reflect the new paradigm, and update the QAPI parser to prohibit "Note" sections while suggesting a new syntax. The exact formatting to use is a matter of taste, but a good candidate is simply: .. note:: lorem ipsum ... ... dolor sit amet ... ... consectetur adipiscing elit ... ... but there are other choices, too. The Sphinx readthedocs theme offers theming for the following forms (capitalization unimportant); all are adorned with a (!) symbol () in the title bar for rendered HTML docs. See https://sphinx-rtd-theme.readthedocs.io/en/stable/demo/demo.html#admonitions for examples of each directive/admonition in use. These are rendered in orange: .. Attention:: ... .. Caution:: ... .. WARNING:: ... These are rendered in red: .. DANGER:: ... .. Error:: ... These are rendered in green: .. Hint:: ... .. Important:: ... .. Tip:: ... These are rendered in blue: .. Note:: ... .. admonition:: custom title admonition body text This patch uses ".. note::" almost everywhere, with just two "caution" directives. Several instances of "Notes:" have been converted to merely ".. note::" where appropriate, but ".. admonition:: notes" is used in a few places where we had an ordered list of multiple notes that would not make sense as standalone/separate admonitions. NOTE: Because qapidoc.py does not attempt to preserve source ordering of sections, the conversion of Notes from a "tagged section" to an "untagged section" means that rendering order for some notes *may change* as a result of this patch. The forthcoming qapidoc.py rewrite strictly preserves source ordering in the rendered documentation, so this issue will be rectified in the new generator. Signed-off-by: John Snow Acked-by: Stefan Hajnoczi [for block*.json] --- docs/devel/qapi-code-gen.rst | 7 +- qapi/block-core.json | 28 +++--- qapi/block.json | 2 +- qapi/char.json| 12 +-- qapi/control.json | 17 ++-- qapi/dump.json| 2 +- qapi/introspect.json | 6 +- qapi/machine-target.json | 26 +++--- qapi/machine.json | 47 +- qapi/migration.json | 12 +-- qapi/misc.json| 88 +-- qapi/net.json | 6 +- qapi/pci.json | 8 +- qapi/qdev.json| 28 +++--- qapi/qom.json | 17 ++-- qapi/rocker.json | 16 ++-- qapi/run-state.json | 18 ++-- qapi/sockets.json | 10 +-- qapi/stats.json | 22 ++--- qapi/transaction.json | 8 +- qapi/ui.json | 29 +++--- qapi/virtio.json | 12 +-- qga/qapi-schema.json | 48 +- scripts/qapi/parser.py| 15 tests/qapi-schema/doc-empty-section.err | 2 +- tests/qapi-schema/doc-empty-section.json | 2 +- tests/qapi-schema/doc-good.json | 4 +- tests/qapi-schema/doc-good.out| 8 +- tests/qapi-schema/doc-good.txt| 10 +-- .../qapi-schema/doc-interleaved-section.json | 2 +- 30 files changed, 260 insertions(+), 252 deletions(-) diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst index cee43222f19..ae97b335cbf 100644 --- a/docs/devel/qapi-code-gen.rst +++ b/docs/devel/qapi-code-gen.rst @@ -995,14 +995,13 @@ line "Features:", like this:: # @feature: Description text A tagged section begins with a paragraph that starts with one of the -following words: "Note:"/"Notes:", "Since:", "Example:"/"Examples:", -"Returns:", "Errors:", "TODO:". It ends with the start of a new -section. +following words: "Since:", "Example:"/"Examples:", "Returns:", +"Errors:", "TODO:". It ends with the start of a new sectio
[PATCH v2 21/21] qapi: remove "Example" doc section
Fully eliminate the "Example" sections in QAPI doc blocks now that they
have all been converted to arbitrary rST syntax using the
".. qmp-example::" directive. Update tests to match.
Migrating to the new syntax
---
The old "Example:" or "Examples:" section syntax is now caught as an
error, but "Example::" is stil permitted as explicit rST syntax for an
un-lexed, generic preformatted text block.
('Example' is not special in this case, any sentence that ends with "::"
will start an indented code block in rST.)
Arbitrary rST for Examples are now possible, but it's strongly
recommended that documentation authors use the ".. qmp-example::"
directive for consistent visual formatting in rendered HTML docs. The
":title:" directive option may be used to add extra information into the
title bar for the example. The ":annotated:" option can be used to write
arbitrary rST instead, with nested "::" blocks applying QMP formatting
where desired.
Other choices available are ".. code-block:: QMP" which will not create
an "Example:" box, or the short-form "::" code-block syntax which will
not apply QMP highlighting when used outside of the qmp-example
directive.
Why?
This patch has several benefits:
1. Example sections can now be written more arbitrarily, mixing
explanatory paragraphs and code blocks however desired.
2. Example sections can now use fully arbitrary rST.
3. All code blocks are now lexed and validated as QMP; increasing
usability of the docs and ensuring validity of example snippets.
(To some extent - This patch only gaurantees it lexes correctly, not
that it's valid under the JSON or QMP grammars. It will catch most
small mistakes, however.)
4. Each qmp-example can be titled or annotated independently without
bypassing the QMP lexer/validator.
(i.e. code blocks are now for *code* only, so we don't have to
sacrifice exposition for having lexically valid examples.)
NOTE: As with the "Notes" conversion patch, this patch (and those
preceding) may change the rendering order for Examples in the
current generator. The forthcoming qapidoc rewrite will fix this
by always generating documentation in source order.
Signed-off-by: John Snow
---
docs/devel/qapi-code-gen.rst| 15 +++
scripts/qapi/parser.py | 10 +-
tests/qapi-schema/doc-good.json | 19 ---
tests/qapi-schema/doc-good.out | 26 ++
tests/qapi-schema/doc-good.txt | 23 ++-
5 files changed, 56 insertions(+), 37 deletions(-)
diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index ae97b335cbf..493e2b8f8df 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -995,8 +995,8 @@ line "Features:", like this::
# @feature: Description text
A tagged section begins with a paragraph that starts with one of the
-following words: "Since:", "Example:"/"Examples:", "Returns:",
-"Errors:", "TODO:". It ends with the start of a new section.
+following words: "Since:", "Returns:", "Errors:", "TODO:". It ends with
+the start of a new section.
The second and subsequent lines of tagged sections must be indented
like this::
@@ -1020,10 +1020,9 @@ detailing a relevant error condition. For example::
A "Since: x.y.z" tagged section lists the release that introduced the
definition.
-An "Example" or "Examples" section is rendered entirely
-as literal fixed-width text. "TODO" sections are not rendered at all
-(they are for developers, not users of QMP). In other sections, the
-text is formatted, and rST markup can be used.
+"TODO" sections are not rendered at all (they are for developers, not
+users of QMP). In other sections, the text is formatted, and rST markup
+can be used.
For example::
@@ -1058,11 +1057,11 @@ For example::
#
# Since: 0.14
#
- # Example:
+ # .. qmp-example::
#
# -> { "execute": "query-blockstats" }
# <- {
- # ... lots of output ...
+ # ...
#}
##
{ 'command': 'query-blockstats',
diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index 6ad5663e545..adc85b5b394 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -553,7 +553,7 @@ def get_doc(self) -> 'QAPIDoc':
# Note: "sections" with two colons are left alone as
# rST markup and not interpreted as a section heading.
-# TODO: Remove this error sometime in 2025 or so
+# TODO: Remove these errors sometime in 2025 or so
# after we've fully transitioned to t
[PATCH v2 13/21] qapi/parser: don't parse rST markup as section headers
The double-colon synax is rST formatting that precedes a literal code
block. We do not want to capture these as QAPI-specific sections.
Coerce blocks that start with e.g. "Example::" to be parsed as untagged
paragraphs instead of special tagged sections.
Signed-off-by: John Snow
---
scripts/qapi/parser.py | 9 +++--
tests/qapi-schema/doc-good.json | 3 +++
tests/qapi-schema/doc-good.out | 3 +++
tests/qapi-schema/doc-good.txt | 3 +++
4 files changed, 16 insertions(+), 2 deletions(-)
diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index 0a13f0f541a..6ad5663e545 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -544,10 +544,15 @@ def get_doc(self) -> 'QAPIDoc':
line = self.get_doc_indented(doc)
no_more_args = True
elif match := re.match(
-r'(Returns|Errors|Since|Notes?|Examples?|TODO): *',
-line):
+r'(Returns|Errors|Since|Notes?|Examples?|TODO)'
+r'(?!::): *',
+line,
+):
# tagged section
+# Note: "sections" with two colons are left alone as
+# rST markup and not interpreted as a section heading.
+
# TODO: Remove this error sometime in 2025 or so
# after we've fully transitioned to the new qapidoc
# generator.
diff --git a/tests/qapi-schema/doc-good.json b/tests/qapi-schema/doc-good.json
index 32ff910b4f8..107123f8a8d 100644
--- a/tests/qapi-schema/doc-good.json
+++ b/tests/qapi-schema/doc-good.json
@@ -181,6 +181,9 @@
# - *verbatim*
# - {braces}
#
+# Note::
+#Ceci n'est pas une note
+#
# Since: 2.10
##
{ 'command': 'cmd',
diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out
index 631dc9f8dad..bd876b6542d 100644
--- a/tests/qapi-schema/doc-good.out
+++ b/tests/qapi-schema/doc-good.out
@@ -190,6 +190,9 @@ frobnicate
section=Examples
- *verbatim*
- {braces}
+section=None
+Note::
+ Ceci n'est pas une note
section=Since
2.10
doc symbol=cmd-boxed
diff --git a/tests/qapi-schema/doc-good.txt b/tests/qapi-schema/doc-good.txt
index d8bfa742c2f..30d457e5488 100644
--- a/tests/qapi-schema/doc-good.txt
+++ b/tests/qapi-schema/doc-good.txt
@@ -231,6 +231,9 @@ Examples
- *verbatim*
- {braces}
+Note::
+ Ceci n'est pas une note
+
Since
~
--
2.45.0
[PATCH v2 17/21] docs/sphinx: add CSS styling for qmp-example directive
From: Harmonie Snow
Add CSS styling for qmp-example directives to increase readability and
consistently style all example blocks.
Signed-off-by: Harmonie Snow
Signed-off-by: John Snow
---
docs/sphinx-static/theme_overrides.css | 46 ++
1 file changed, 46 insertions(+)
diff --git a/docs/sphinx-static/theme_overrides.css
b/docs/sphinx-static/theme_overrides.css
index c70ef951286..f3223af62b2 100644
--- a/docs/sphinx-static/theme_overrides.css
+++ b/docs/sphinx-static/theme_overrides.css
@@ -87,6 +87,52 @@ div[class^="highlight"] pre {
padding-bottom: 1px;
}
+/* qmp-example directive styling */
+
+.rst-content .admonition-example {
+background-color: #fcfcfc;
+padding: 0px;
+}
+
+.rst-content .admonition-example > .admonition-title {
+background-color: #338254;
+margin: 0px;
+}
+
+.rst-content .admonition-example > div[class^=highlight] {
+border-top: 1px solid #d7f0d7;
+border-bottom: 1px solid #d7f0d7;
+border-left: 3px solid #d7f0d7;
+border-right: 3px solid #d7f0d7;
+}
+
+.rst-content .admonition-example .highlight {
+background: linear-gradient(#d9f1d9 1%, #ecf8ec 10%,
+#ecf8ec 90%, #d9f1d9 99%);
+}
+
+.rst-content .admonition-example > .admonition-title:before {
+content: "里";
+}
+
+.rst-content .admonition-example > .admonition-title p {
+margin: 0px;
+}
+
+.rst-content .admonition-example p {
+padding: 15px 10px;
+margin: 0px;
+background-color: #fafafa;
+border-left: 1px solid #ededed;
+border-right: 1px solid #ededed;
+}
+
+.rst-content .admonition-example > .admonition {
+background-color: #fcfcfc;
+}
+
+/* end qmp-example styling */
+
@media screen {
/* content column
--
2.45.0
[PATCH v2 18/21] qapi: convert "Example" sections without titles
Use the no-option form of ".. qmp-example::" to convert any Examples
that do not have any form of caption or explanation whatsoever.
See commit-3: "docs/qapidoc: create qmp-example directive", for a
detailed explanation of this custom directive syntax.
See commit+3: "qapi: remove "Example" doc section" for a detailed
explanation of why.
Signed-off-by: John Snow
---
qapi/acpi.json | 4 +--
qapi/block-core.json | 64 +---
qapi/block.json | 18 ++-
qapi/char.json | 24 +--
qapi/control.json| 8 ++---
qapi/dump.json | 8 ++---
qapi/machine-target.json | 2 +-
qapi/machine.json| 38
qapi/migration.json | 58 ++--
qapi/misc-target.json| 22 +++---
qapi/misc.json | 32 ++--
qapi/net.json| 20 +++--
qapi/pci.json| 2 +-
qapi/qdev.json | 10 ---
qapi/qom.json| 8 ++---
qapi/replay.json | 8 ++---
qapi/rocker.json | 8 ++---
qapi/run-state.json | 30 +--
qapi/tpm.json| 6 ++--
qapi/trace.json | 4 +--
qapi/transaction.json| 2 +-
qapi/ui.json | 34 ++---
qapi/vfio.json | 2 +-
qapi/virtio.json | 2 +-
qapi/yank.json | 4 +--
25 files changed, 216 insertions(+), 202 deletions(-)
diff --git a/qapi/acpi.json b/qapi/acpi.json
index aa4dbe57943..045dab6228b 100644
--- a/qapi/acpi.json
+++ b/qapi/acpi.json
@@ -111,7 +111,7 @@
#
# Since: 2.1
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-acpi-ospm-status" }
# <- { "return": [ { "device": "d1", "slot": "0", "slot-type": "DIMM",
"source": 1, "status": 0},
@@ -131,7 +131,7 @@
#
# Since: 2.1
#
-# Example:
+# .. qmp-example::
#
# <- { "event": "ACPI_DEVICE_OST",
# "data": { "info": { "device": "d1", "slot": "0",
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 9ef23ec02ae..4e0f0395146 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -764,7 +764,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-block" }
# <- {
@@ -1168,7 +1168,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-blockstats" }
# <- {
@@ -1461,7 +1461,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "block_resize",
# "arguments": { "device": "scratch", "size": 1073741824 } }
@@ -1682,7 +1682,7 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "blockdev-snapshot-sync",
# "arguments": { "device": "ide-hd0",
@@ -1715,7 +1715,7 @@
#
# Since: 2.5
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "blockdev-add",
# "arguments": { "driver": "qcow2",
@@ -1861,7 +1861,7 @@
#
# Since: 1.3
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "block-commit",
# "arguments": { "device": "virtio0",
@@ -1899,7 +1899,7 @@
#
# Since: 1.6
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "drive-backup",
# "arguments": { "device": "drive0",
@@ -1925,7 +1925,7 @@
#
# Since: 2.3
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "blockdev-backup",
# "arguments": { "device": "src-id",
@@ -1949,7 +1949,7 @@
#
# Since: 2.0
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "query-named-block-nodes" }
# <- { "return": [ { "ro":false,
@@ -2130,7 +2130,7 @@
#
# Since: 1.3
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "drive-mirror",
# "arguments": { "device": "ide-hd0",
@@ -2307,7 +2307,7 @@
#
# Since: 2.4
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "block-dirty-bitmap-add",
# "arguments": { "node": "drive0", "name": "bitmap0" } }
@@ -2331,7 +2331,7 @@
#
# Since: 2.4
#
-# Example:
+# .. qmp-example::
#
# -> { "execute": "block-dirty-bitmap-remove",
# "arguments": { "no
[PATCH v2 12/21] qapi: add markup to note blocks
Generally, surround command-line options with ``literal`` markup to help
it stand out from prose in rendered HTML, and add cross-references to
replace "see also" messages.
References to types, values, and other QAPI definitions are not yet
adjusted here; they will be converted en masse in a subsequent patch
after the new QAPI doc generator is merged.
Signed-off-by: John Snow
Reviewed-by: Markus Armbruster
---
qapi/control.json | 4 ++--
qapi/misc.json | 8
qapi/qdev.json | 2 +-
qapi/run-state.json | 2 +-
qapi/sockets.json | 2 +-
qapi/ui.json| 2 +-
6 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/qapi/control.json b/qapi/control.json
index 59d5e00c151..fe2af45120b 100644
--- a/qapi/control.json
+++ b/qapi/control.json
@@ -24,8 +24,8 @@
#
# .. note:: This command is valid exactly when first connecting: it must
#be issued before any other command will be accepted, and will fail
-#once the monitor is accepting other commands. (see qemu
-#docs/interop/qmp-spec.rst)
+#once the monitor is accepting other commands.
+#(see :doc:`/interop/qmp-spec`)
#
# .. note:: The QMP client needs to explicitly enable QMP capabilities,
#otherwise all the QMP capabilities will be turned off by default.
diff --git a/qapi/misc.json b/qapi/misc.json
index 13ea82f5254..b04efbadec6 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -104,7 +104,7 @@
# Returns a list of information about each iothread.
#
# .. note:: This list excludes the QEMU main loop thread, which is not
-#declared using the -object iothread command-line option. It is
+#declared using the ``-object iothread`` command-line option. It is
#always the main thread of the process.
#
# Returns: a list of @IOThreadInfo for each iothread
@@ -138,8 +138,8 @@
#
# .. note:: This function will succeed even if the guest is already in
#the stopped state. In "inmigrate" state, it will ensure that the
-#guest remains paused once migration finishes, as if the -S option
-#was passed on the command line.
+#guest remains paused once migration finishes, as if the ``-S``
+#option was passed on the command line.
#
#In the "suspended" state, it will completely stop the VM and cause
#a transition to the "paused" state. (Since 9.0)
@@ -161,7 +161,7 @@
# .. note:: This command will succeed if the guest is currently running.
#It will also succeed if the guest is in the "inmigrate" state; in
#this case, the effect of the command is to make sure the guest
-#starts once migration finishes, removing the effect of the -S
+#starts once migration finishes, removing the effect of the ``-S``
#command line option if it was passed.
#
#If the VM was previously suspended, and not been reset or woken,
diff --git a/qapi/qdev.json b/qapi/qdev.json
index f5b35a814fe..d031fc3590d 100644
--- a/qapi/qdev.json
+++ b/qapi/qdev.json
@@ -59,7 +59,7 @@
#the 'docs/qdev-device-use.txt' file.
#
# 3. It's possible to list device properties by running QEMU with
-#the "-device DEVICE,help" command-line argument, where DEVICE
+#the ``-device DEVICE,help`` command-line argument, where DEVICE
#is the device's name.
#
# Example:
diff --git a/qapi/run-state.json b/qapi/run-state.json
index dc524234ace..252d7d6afa7 100644
--- a/qapi/run-state.json
+++ b/qapi/run-state.json
@@ -146,7 +146,7 @@
# @reason: The @ShutdownCause which resulted in the SHUTDOWN.
# (since 4.0)
#
-# .. note:: If the command-line option "-no-shutdown" has been
+# .. note:: If the command-line option ``-no-shutdown`` has been
#specified, qemu will not exit, and a STOP event will eventually
#follow the SHUTDOWN event.
#
diff --git a/qapi/sockets.json b/qapi/sockets.json
index 3970118bf4e..4d78d2ccb77 100644
--- a/qapi/sockets.json
+++ b/qapi/sockets.json
@@ -181,7 +181,7 @@
#
# .. note:: This type is deprecated in favor of SocketAddress. The
#difference between SocketAddressLegacy and SocketAddress is that
-#the latter has fewer {} on the wire.
+#the latter has fewer ``{}`` on the wire.
#
# Since: 1.3
##
diff --git a/qapi/ui.json b/qapi/ui.json
index a165e44..5bcccbfc930 100644
--- a/qapi/ui.json
+++ b/qapi/ui.json
@@ -1268,7 +1268,7 @@
# Since: 2.6
#
# .. note:: The consoles are visible in the qom tree, under
-#/backend/console[$index]. They have a device link and head
+#``/backend/console[$index]``. They have a device link and head
#property, so it is possible to map which console belongs to which
#device and display.
#
--
2.45.0
[PATCH v2 02/21] qapi: linter fixups
Fix minor irritants to pylint/flake8 et al.
(Yes, these need to be guarded by the Python tests. That's a work in
progress, a series that's quite likely to follow once I finish this
Sphinx project. Please pardon the temporary irritation.)
Signed-off-by: John Snow
Reviewed-by: Markus Armbruster
---
scripts/qapi/introspect.py | 8
scripts/qapi/schema.py | 6 +++---
scripts/qapi/visit.py | 5 +++--
3 files changed, 10 insertions(+), 9 deletions(-)
diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
index 86c075a6ad2..ac14b20f308 100644
--- a/scripts/qapi/introspect.py
+++ b/scripts/qapi/introspect.py
@@ -27,8 +27,8 @@
from .schema import (
QAPISchema,
QAPISchemaAlternatives,
-QAPISchemaBranches,
QAPISchemaArrayType,
+QAPISchemaBranches,
QAPISchemaBuiltinType,
QAPISchemaEntity,
QAPISchemaEnumMember,
@@ -233,9 +233,9 @@ def _use_type(self, typ: QAPISchemaType) -> str:
typ = type_int
elif (isinstance(typ, QAPISchemaArrayType) and
typ.element_type.json_type() == 'int'):
-type_intList = self._schema.lookup_type('intList')
-assert type_intList
-typ = type_intList
+type_intlist = self._schema.lookup_type('intList')
+assert type_intlist
+typ = type_intlist
# Add type to work queue if new
if typ not in self._used_types:
self._used_types.append(typ)
diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
index 721c470d2b8..d65c35f6ee6 100644
--- a/scripts/qapi/schema.py
+++ b/scripts/qapi/schema.py
@@ -730,6 +730,7 @@ def set_defined_in(self, name: str) -> None:
for v in self.variants:
v.set_defined_in(name)
+# pylint: disable=unused-argument
def check(
self, schema: QAPISchema, seen: Dict[str, QAPISchemaMember]
) -> None:
@@ -1166,7 +1167,7 @@ def _def_definition(self, defn: QAPISchemaDefinition) ->
None:
defn.info, "%s is already defined" % other_defn.describe())
self._entity_dict[defn.name] = defn
-def lookup_entity(self,name: str) -> Optional[QAPISchemaEntity]:
+def lookup_entity(self, name: str) -> Optional[QAPISchemaEntity]:
return self._entity_dict.get(name)
def lookup_type(self, name: str) -> Optional[QAPISchemaType]:
@@ -1302,11 +1303,10 @@ def _make_implicit_object_type(
name = 'q_obj_%s-%s' % (name, role)
typ = self.lookup_entity(name)
if typ:
-assert(isinstance(typ, QAPISchemaObjectType))
+assert isinstance(typ, QAPISchemaObjectType)
# The implicit object type has multiple users. This can
# only be a duplicate definition, which will be flagged
# later.
-pass
else:
self._def_definition(QAPISchemaObjectType(
name, info, None, ifcond, None, None, members, None))
diff --git a/scripts/qapi/visit.py b/scripts/qapi/visit.py
index e766acaac92..12f92e429f6 100644
--- a/scripts/qapi/visit.py
+++ b/scripts/qapi/visit.py
@@ -280,8 +280,9 @@ def gen_visit_alternate(name: str,
abort();
default:
assert(visit_is_input(v));
-error_setg(errp, "Invalid parameter type for '%%s', expected:
%(name)s",
- name ? name : "null");
+error_setg(errp,
+ "Invalid parameter type for '%%s', expected: %(name)s",
+ name ? name : "null");
/* Avoid passing invalid *obj to qapi_free_%(c_name)s() */
g_free(*obj);
*obj = NULL;
--
2.45.0
[PATCH v2 19/21] qapi: convert "Example" sections with titles
When an Example section has a brief explanation, convert it to a
qmp-example:: section using the :title: option.
Rule of thumb: If the title can fit on a single line and requires no rST
markup, it's a good candidate for using the :title: option of
qmp-example.
In this patch, trailing punctuation is removed from the title section
for consistent headline aesthetics. In just one case, specifics of the
example are removed to make the title read better.
See commit-4: "docs/qapidoc: create qmp-example directive", for a
detailed explanation of this custom directive syntax.
See commit+2: "qapi: remove "Example" doc section" for a detailed
explanation of why.
Signed-off-by: John Snow
---
qapi/block-core.json | 24
qapi/block.json | 13 ++---
qapi/migration.json | 25 ++---
qapi/qom.json| 8
qapi/ui.json | 11 ++-
qapi/virtio.json | 19 ++-
6 files changed, 52 insertions(+), 48 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 4e0f0395146..a371e3464d2 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -5885,9 +5885,8 @@
#
# Since: 2.7
#
-# Examples:
-#
-# 1. Add a new node to a quorum
+# .. qmp-example::
+#:title: Add a new node to a quorum
#
# -> { "execute": "blockdev-add",
# "arguments": {
@@ -5901,7 +5900,8 @@
# "node": "new_node" } }
# <- { "return": {} }
#
-# 2. Delete a quorum's node
+# .. qmp-example::
+#:title: Delete a quorum's node
#
# -> { "execute": "x-blockdev-change",
# "arguments": { "parent": "disk1",
@@ -5937,16 +5937,16 @@
#
# Since: 2.12
#
-# Examples:
-#
-# 1. Move a node into an IOThread
+# .. qmp-example::
+#:title: Move a node into an IOThread
#
# -> { "execute": "x-blockdev-set-iothread",
# "arguments": { "node-name": "disk1",
# "iothread": "iothread0" } }
# <- { "return": {} }
#
-# 2. Move a node into the main loop
+# .. qmp-example::
+#:title: Move a node into the main loop
#
# -> { "execute": "x-blockdev-set-iothread",
# "arguments": { "node-name": "disk1",
@@ -6022,16 +6022,16 @@
#
# Since: 2.0
#
-# Examples:
-#
-# 1. Read operation
+# .. qmp-example::
+#:title: Read operation
#
# <- { "event": "QUORUM_REPORT_BAD",
# "data": { "node-name": "node0", "sector-num": 345435,
"sectors-count": 5,
#"type": "read" },
# "timestamp": { "seconds": 1344522075, "microseconds": 745528 } }
#
-# 2. Flush operation
+# .. qmp-example::
+#:title: Flush operation
#
# <- { "event": "QUORUM_REPORT_BAD",
# "data": { "node-name": "node0", "sector-num": 0, "sectors-count":
2097120,
diff --git a/qapi/block.json b/qapi/block.json
index c8e52bc2d29..5ddd061e964 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -342,9 +342,8 @@
#
# Since: 2.5
#
-# Examples:
-#
-# 1. Change a removable medium
+# .. qmp-example::
+#:title: Change a removable medium
#
# -> { "execute": "blockdev-change-medium",
# "arguments": { "id": "ide0-1-0",
@@ -352,7 +351,8 @@
# "format": "raw" } }
# <- { "return": {} }
#
-# 2. Load a read-only medium into a writable drive
+# .. qmp-example::
+#:title: Load a read-only medium into a writable drive
#
# -> { "execute": "blockdev-change-medium",
# "arguments": { "id": "floppyA",
@@ -577,9 +577,8 @@
# "boundaries-write": [1000, 5000] } }
# <- { "return": {} }
#
-# Example:
-#
-# Remove all latency histograms:
+# .. qmp-example::
+#:title: Remove all latency histograms
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0" } }
diff --git a/qapi/migration.json b/qapi/migration.json
index a4391ea7e6f..37ce8afa380 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -287,14 +287,14 @@
#
# Since: 0.14
#
-# Examples:
-#
-# 1. Before the first migration
+# .. qmp-example::
+#:title: Before the first migration
#
# -> { "execute": "query-migrate" }
# <- { "return":
[PATCH v2 14/21] docs/qapidoc: factor out do_parse()
Factor out the compatibility parser helper so it can be shared by other directives. Signed-off-by: John Snow --- docs/sphinx/qapidoc.py | 64 +++--- 1 file changed, 35 insertions(+), 29 deletions(-) diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py index efcd84656fa..43dd99e21e6 100644 --- a/docs/sphinx/qapidoc.py +++ b/docs/sphinx/qapidoc.py @@ -494,7 +494,41 @@ def visit_module(self, name): super().visit_module(name) -class QAPIDocDirective(Directive): +class NestedDirective(Directive): +def run(self): +raise NotImplementedError + +def do_parse(self, rstlist, node): +""" +Parse rST source lines and add them to the specified node + +Take the list of rST source lines rstlist, parse them as +rST, and add the resulting docutils nodes as children of node. +The nodes are parsed in a way that allows them to include +subheadings (titles) without confusing the rendering of +anything else. +""" +# This is from kerneldoc.py -- it works around an API change in +# Sphinx between 1.6 and 1.7. Unlike kerneldoc.py, we use +# sphinx.util.nodes.nested_parse_with_titles() rather than the +# plain self.state.nested_parse(), and so we can drop the saving +# of title_styles and section_level that kerneldoc.py does, +# because nested_parse_with_titles() does that for us. +if USE_SSI: +with switch_source_input(self.state, rstlist): +nested_parse_with_titles(self.state, rstlist, node) +else: +save = self.state.memo.reporter +self.state.memo.reporter = AutodocReporter( +rstlist, self.state.memo.reporter +) +try: +nested_parse_with_titles(self.state, rstlist, node) +finally: +self.state.memo.reporter = save + + +class QAPIDocDirective(NestedDirective): """Extract documentation from the specified QAPI .json file""" required_argument = 1 @@ -532,34 +566,6 @@ def run(self): # so they are displayed nicely to the user raise ExtensionError(str(err)) from err -def do_parse(self, rstlist, node): -"""Parse rST source lines and add them to the specified node - -Take the list of rST source lines rstlist, parse them as -rST, and add the resulting docutils nodes as children of node. -The nodes are parsed in a way that allows them to include -subheadings (titles) without confusing the rendering of -anything else. -""" -# This is from kerneldoc.py -- it works around an API change in -# Sphinx between 1.6 and 1.7. Unlike kerneldoc.py, we use -# sphinx.util.nodes.nested_parse_with_titles() rather than the -# plain self.state.nested_parse(), and so we can drop the saving -# of title_styles and section_level that kerneldoc.py does, -# because nested_parse_with_titles() does that for us. -if USE_SSI: -with switch_source_input(self.state, rstlist): -nested_parse_with_titles(self.state, rstlist, node) -else: -save = self.state.memo.reporter -self.state.memo.reporter = AutodocReporter( -rstlist, self.state.memo.reporter -) -try: -nested_parse_with_titles(self.state, rstlist, node) -finally: -self.state.memo.reporter = save - def setup(app): """Register qapi-doc directive with Sphinx""" -- 2.45.0
[PATCH v2 20/21] qapi: convert "Example" sections with longer prose
These examples require longer explanations or have explanations that
require markup to look reasonable when rendered and so use the longer
form of the ".. qmp-example::" directive.
By using the :annotated: option, the content in the example block is
assumed *not* to be a code block literal and is instead parsed as normal
rST - with the exception that any code literal blocks after `::` will
assumed to be a QMP code literal block.
Note: There's one title-less conversion in this patch that comes along
for the ride because it's part of a larger "Examples" block that was
better to convert all at once.
See commit-5: "docs/qapidoc: create qmp-example directive", for a
detailed explanation of this custom directive syntax.
See commit+1: "qapi: remove "Example" doc section" for a detailed
explanation of why.
Signed-off-by: John Snow
---
qapi/block.json | 26 --
qapi/machine.json | 30 --
qapi/migration.json | 7 +--
qapi/virtio.json| 24 ++--
4 files changed, 59 insertions(+), 28 deletions(-)
diff --git a/qapi/block.json b/qapi/block.json
index 5ddd061e964..d95e9fd8140 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -545,31 +545,37 @@
#
# Since: 4.0
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# Set new histograms for all io types with intervals
-# [0, 10), [10, 50), [50, 100), [100, +inf):
+#Set new histograms for all io types with intervals
+#[0, 10), [10, 50), [50, 100), [100, +inf)::
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0",
# "boundaries": [10, 50, 100] } }
# <- { "return": {} }
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# Set new histogram only for write, other histograms will remain
-# not changed (or not created):
+#Set new histogram only for write, other histograms will remain
+#not changed (or not created)::
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0",
# "boundaries-write": [10, 50, 100] } }
# <- { "return": {} }
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# Set new histograms with the following intervals:
-# read, flush: [0, 10), [10, 50), [50, 100), [100, +inf)
-# write: [0, 1000), [1000, 5000), [5000, +inf)
+#Set new histograms with the following intervals:
+#
+#- read, flush: [0, 10), [10, 50), [50, 100), [100, +inf)
+#- write: [0, 1000), [1000, 5000), [5000, +inf)
+#
+#::
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0",
diff --git a/qapi/machine.json b/qapi/machine.json
index 83f60b319c7..0a5ffe652b7 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -1047,10 +1047,11 @@
#
# Since: 2.7
#
-# Examples:
+# .. qmp-example::
+#:annotated:
#
-# For pseries machine type started with -smp 2,cores=2,maxcpus=4
-# -cpu POWER8:
+#For pseries machine type started with
+#``-smp 2,cores=2,maxcpus=4 -cpu POWER8``::
#
# -> { "execute": "query-hotpluggable-cpus" }
# <- {"return": [
@@ -1060,7 +1061,10 @@
#"vcpus-count": 1, "qom-path": "/machine/unattached/device[0]"}
#]}
#
-# For pc machine type started with -smp 1,maxcpus=2:
+# .. qmp-example::
+#:annotated:
+#
+#For pc machine type started with ``-smp 1,maxcpus=2``::
#
# -> { "execute": "query-hotpluggable-cpus" }
# <- {"return": [
@@ -1075,8 +1079,11 @@
# }
#]}
#
-# For s390x-virtio-ccw machine type started with -smp 1,maxcpus=2
-# -cpu qemu (Since: 2.11):
+# .. qmp-example::
+#:annotated:
+#
+#For s390x-virtio-ccw machine type started with
+#``-smp 1,maxcpus=2 -cpu qemu`` (Since: 2.11)::
#
# -> { "execute": "query-hotpluggable-cpus" }
# <- {"return": [
@@ -1130,12 +1137,15 @@
#
# Since: 0.14
#
-# Example:
+# .. qmp-example::
+#:annotated:
#
-# -> { "execute": "balloon", "arguments": { "value": 536870912 } }
-# <- { "return": {} }
+#::
#
-# With a 2.5GiB guest this command inflated the ballon to 3GiB.
+# -> { "execute": "balloon", "arguments": { "value": 536870912 } }
+# <- { "return": {} }
+#
+#With a 2.5GiB guest this command inflated the ballon to 3GiB.
##
{ 'command': 'balloon', 'data': {'value': 'int'} }
diff --git a/qapi/migration.js
[PATCH v2 07/21] docs/qapidoc: fix nested parsing under untagged sections
Sphinx does not like sections without titles, because it wants to
convert every section into a reference. When there is no title, it
struggles to do this and transforms the tree inproperly.
Depending on the rST used, this may result in an assertion error deep in
the docutils HTMLWriter.
(Observed when using ".. admonition:: Notes" under such a section - When
this is transformed with its own element, Sphinx is fooled into
believing this title belongs to the section and incorrect mutates the
docutils tree, leading to errors during rendering time.)
When parsing an untagged section (free paragraphs), skip making a hollow
section and instead append the parse results to the prior section.
Many Bothans died to bring us this information.
Signed-off-by: John Snow
Acked-by: Markus Armbruster
---
docs/sphinx/qapidoc.py | 16 +++-
1 file changed, 11 insertions(+), 5 deletions(-)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index f9683444b14..efcd84656fa 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -272,14 +272,20 @@ def _nodes_for_sections(self, doc):
if section.tag and section.tag == 'TODO':
# Hide TODO: sections
continue
+
+if not section.tag:
+# Sphinx cannot handle sectionless titles;
+# Instead, just append the results to the prior section.
+container = nodes.container()
+self._parse_text_into_node(section.text, container)
+nodelist += container.children
+continue
+
snode = self._make_section(section.tag)
-if section.tag and section.tag.startswith('Example'):
+if section.tag.startswith('Example'):
snode += self._nodes_for_example(dedent(section.text))
else:
-self._parse_text_into_node(
-dedent(section.text) if section.tag else section.text,
-snode,
-)
+self._parse_text_into_node(dedent(section.text), snode)
nodelist.append(snode)
return nodelist
--
2.45.0
[PATCH v2 15/21] docs/qapidoc: create qmp-example directive
This is a directive that creates a syntactic sugar for creating
"Example" boxes very similar to the ones already used in the bitmaps.rst
document, please see e.g.
https://www.qemu.org/docs/master/interop/bitmaps.html#creation-block-dirty-bitmap-add
In its simplest form, when a custom title is not needed or wanted, and
the example body is *solely* a QMP example:
```
.. qmp-example::
{body}
```
is syntactic sugar for:
```
.. admonition:: Example:
.. code-block:: QMP
{body}
```
When a custom, plaintext title that describes the example is desired,
this form:
```
.. qmp-example::
:title: Defrobnification
{body}
```
Is syntactic sugar for:
```
.. admonition:: Example: Defrobnification
.. code-block:: QMP
{body}
```
Lastly, when Examples are multi-step processes that require non-QMP
exposition, have lengthy titles, or otherwise involve prose with rST
markup (lists, cross-references, etc), the most complex form:
```
.. qmp-example::
:annotated:
This example shows how to use `foo-command`::
{body}
```
Is desugared to:
```
.. admonition:: Example:
This example shows how to use `foo-command`::
{body}
For more information, please see `frobnozz`.
```
The primary benefit here being documentation source consistently using
the same directive for all forms of examples to ensure consistent visual
styling, and ensuring all relevant prose is visually grouped alongside
the code literal block.
Note that as of this commit, the code-block rST syntax "::" does not
apply QMP highlighting; you would need to use ".. code-block:: QMP". The
very next commit changes this behavior to assume all "::" code blocks
within this directive are QMP blocks.
Signed-off-by: John Snow
---
docs/sphinx/qapidoc.py | 60 --
1 file changed, 58 insertions(+), 2 deletions(-)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index 43dd99e21e6..a2fa05fc491 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -27,16 +27,19 @@
import os
import re
import textwrap
+from typing import List
from docutils import nodes
-from docutils.parsers.rst import Directive, directives
+from docutils.parsers.rst import directives
from docutils.statemachine import ViewList
from qapi.error import QAPIError, QAPISemError
from qapi.gen import QAPISchemaVisitor
from qapi.schema import QAPISchema
import sphinx
+from sphinx.directives.code import CodeBlock
from sphinx.errors import ExtensionError
+from sphinx.util.docutils import SphinxDirective
from sphinx.util.nodes import nested_parse_with_titles
@@ -494,7 +497,7 @@ def visit_module(self, name):
super().visit_module(name)
-class NestedDirective(Directive):
+class NestedDirective(SphinxDirective):
def run(self):
raise NotImplementedError
@@ -567,10 +570,63 @@ def run(self):
raise ExtensionError(str(err)) from err
+class QMPExample(CodeBlock, NestedDirective):
+"""
+Custom admonition for QMP code examples.
+
+When the :annotated: option is present, the body of this directive
+is parsed as normal rST instead. Code blocks must be explicitly
+written by the user, but this allows for intermingling explanatory
+paragraphs with arbitrary rST syntax and code blocks for more
+involved examples.
+
+When :annotated: is absent, the directive body is treated as a
+simple standalone QMP code block literal.
+"""
+
+required_argument = 0
+optional_arguments = 0
+has_content = True
+option_spec = {
+"annotated": directives.flag,
+"title": directives.unchanged,
+}
+
+def admonition_wrap(self, *content) -> List[nodes.Node]:
+title = "Example:"
+if "title" in self.options:
+title = f"{title} {self.options['title']}"
+
+admon = nodes.admonition(
+"",
+nodes.title("", title),
+*content,
+classes=["admonition", "admonition-example"],
+)
+return [admon]
+
+def run_annotated(self) -> List[nodes.Node]:
+content_node: nodes.Element = nodes.section()
+self.do_parse(self.content, content_node)
+return content_node.children
+
+def run(self) -> List[nodes.Node]:
+annotated = "annotated" in self.options
+
+if annotated:
+content_nodes = self.run_annotated()
+else:
+self.arguments = ["QMP"]
+content_nodes = super().run()
+
+return self.admonition_wrap(*content_nodes)
+
+
def setup(app):
"""Register qapi-doc directive with Sphinx"""
app.add_config_value("qapidoc_srctree", None, "env")
app.add_directive("qapi-doc", QAPIDocDirective)
+app.add_directive("qmp-example", QMPExample)
return {
"version": __version__,
--
2.45.0
[PATCH v2 08/21] qapi: fix non-compliant JSON examples
The new QMP documentation generator wants to parse all examples as
"QMP". We have an existing QMP lexer in docs/sphinx/qmp_lexer.py (Seen
in-use here: https://qemu-project.gitlab.io/qemu/interop/bitmaps.html)
that allows the use of "->", "<-" and "..." tokens to denote QMP
protocol flow with elisions, but otherwise defers to the JSON lexer.
To utilize this lexer for the existing QAPI documentation, we need them
to conform to a standard so that they lex and render correctly. Once the
QMP lexer is active for examples, errant QMP/JSON will produce warning
messages and fail the build.
Fix any invalid JSON found in QAPI documentation (identified by
attempting to lex all examples as QMP; see subsequent
commits). Additionally, elisions must be standardized for the QMP lexer;
they must be represented as the value "...", so three examples have been
adjusted to support that format here.
Signed-off-by: John Snow
Reviewed-by: Markus Armbruster
---
qapi/control.json | 3 ++-
qapi/machine.json | 2 +-
qapi/migration.json | 2 +-
qapi/misc.json | 3 ++-
qapi/net.json | 6 +++---
qapi/rocker.json| 2 +-
qapi/ui.json| 2 +-
7 files changed, 11 insertions(+), 9 deletions(-)
diff --git a/qapi/control.json b/qapi/control.json
index 6bdbf077c2e..10c906fa0e7 100644
--- a/qapi/control.json
+++ b/qapi/control.json
@@ -145,7 +145,8 @@
# },
# {
#"name":"system_powerdown"
-# }
+# },
+# ...
# ]
#}
#
diff --git a/qapi/machine.json b/qapi/machine.json
index 2fd3e9c3d5d..a982c945035 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -1057,7 +1057,7 @@
#"vcpus-count": 1 },
# { "props": { "core-id": 0 }, "type": "POWER8-spapr-cpu-core",
#"vcpus-count": 1, "qom-path": "/machine/unattached/device[0]"}
-#]}'
+#]}
#
# For pc machine type started with -smp 1,maxcpus=2:
#
diff --git a/qapi/migration.json b/qapi/migration.json
index 0f24206bce4..9ec9ef36c47 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -2106,7 +2106,7 @@
# Example:
#
# -> {"execute": "calc-dirty-rate", "arguments": {"calc-time": 1,
-# 'sample-pages': 512} }
+# "sample-pages": 512} }
# <- { "return": {} }
#
# Measure dirty rate using dirty bitmap for 500 milliseconds:
diff --git a/qapi/misc.json b/qapi/misc.json
index ec30e5c570a..4b41e15dcd4 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -287,7 +287,8 @@
#
# Example:
#
-# -> { "execute": "get-win32-socket", "arguments": { "info": "abcd123..",
fdname": "skclient" } }
+# -> { "execute": "get-win32-socket",
+# "arguments": { "info": "abcd123..", "fdname": "skclient" } }
# <- { "return": {} }
##
{ 'command': 'get-win32-socket', 'data': {'info': 'str', 'fdname': 'str'},
'if': 'CONFIG_WIN32' }
diff --git a/qapi/net.json b/qapi/net.json
index 0f5a259475e..c19df435a53 100644
--- a/qapi/net.json
+++ b/qapi/net.json
@@ -1003,9 +1003,9 @@
#
# Example:
#
-# <- { 'event': 'NETDEV_STREAM_DISCONNECTED',
-# 'data': {'netdev-id': 'netdev0'},
-# 'timestamp': {'seconds': 1663330937, 'microseconds': 526695} }
+# <- { "event": "NETDEV_STREAM_DISCONNECTED",
+# "data": {"netdev-id": "netdev0"},
+# "timestamp": {"seconds": 1663330937, "microseconds": 526695} }
##
{ 'event': 'NETDEV_STREAM_DISCONNECTED',
'data': { 'netdev-id': 'str' } }
diff --git a/qapi/rocker.json b/qapi/rocker.json
index 5635cf174fd..f5225eb62cc 100644
--- a/qapi/rocker.json
+++ b/qapi/rocker.json
@@ -250,7 +250,7 @@
# "action": {"goto-tbl": 10},
# "mask": {"in-pport": 4294901760}
# },
-# {...more...},
+# {...},
#]}
##
{ 'command': 'query-rocker-of-dpa-flows',
diff --git a/qapi/ui.json b/qapi/ui.json
index f610bce118a..c12f5292571 100644
--- a/qapi/ui.json
+++ b/qapi/ui.json
@@ -361,7 +361,7 @@
#"channel-id": 0,
#"tls": false
# },
-# [ ... more channels follow ... ]
+# ...
# ]
# }
#}
--
2.45.0
[PATCH v2 16/21] docs/qapidoc: add QMP highlighting to annotated qmp-example blocks
For any code literal blocks inside of a qmp-example directive, apply and
enforce the QMP lexer/highlighter to those blocks.
This way, you won't need to write:
```
.. qmp-example::
:annotated:
Blah blah
.. code-block:: QMP
-> { "lorem": "ipsum" }
```
But instead, simply:
```
.. qmp-example::
:annotated:
Blah blah::
-> { "lorem": "ipsum" }
```
Once the directive block is exited, whatever the previous default
highlight language was will be restored; localizing the forced QMP
lexing to exclusively this directive.
Signed-off-by: John Snow
---
docs/sphinx/qapidoc.py | 53 ++
1 file changed, 49 insertions(+), 4 deletions(-)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index a2fa05fc491..c8c404ad1b0 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -26,6 +26,7 @@
import os
import re
+import sys
import textwrap
from typing import List
@@ -37,6 +38,7 @@
from qapi.schema import QAPISchema
import sphinx
+from sphinx import addnodes
from sphinx.directives.code import CodeBlock
from sphinx.errors import ExtensionError
from sphinx.util.docutils import SphinxDirective
@@ -575,10 +577,10 @@ class QMPExample(CodeBlock, NestedDirective):
Custom admonition for QMP code examples.
When the :annotated: option is present, the body of this directive
-is parsed as normal rST instead. Code blocks must be explicitly
-written by the user, but this allows for intermingling explanatory
-paragraphs with arbitrary rST syntax and code blocks for more
-involved examples.
+is parsed as normal rST, but with any '::' code blocks set to use
+the QMP lexer. Code blocks must be explicitly written by the user,
+but this allows for intermingling explanatory paragraphs with
+arbitrary rST syntax and code blocks for more involved examples.
When :annotated: is absent, the directive body is treated as a
simple standalone QMP code block literal.
@@ -592,6 +594,33 @@ class QMPExample(CodeBlock, NestedDirective):
"title": directives.unchanged,
}
+def _highlightlang(self) -> addnodes.highlightlang:
+"""Return the current highlightlang setting for the document"""
+node = None
+doc = self.state.document
+
+if hasattr(doc, "findall"):
+# docutils >= 0.18.1
+for node in doc.findall(addnodes.highlightlang):
+pass
+else:
+for elem in doc.traverse():
+if isinstance(elem, addnodes.highlightlang):
+node = elem
+
+if node:
+return node
+
+# No explicit directive found, use defaults
+node = addnodes.highlightlang(
+lang=self.env.config.highlight_language,
+force=False,
+# Yes, Sphinx uses this value to effectively disable line
+# numbers and not 0 or None or -1 or something. ¯\_(ツ)_/¯
+linenothreshold=sys.maxsize,
+)
+return node
+
def admonition_wrap(self, *content) -> List[nodes.Node]:
title = "Example:"
if "title" in self.options:
@@ -606,8 +635,24 @@ def admonition_wrap(self, *content) -> List[nodes.Node]:
return [admon]
def run_annotated(self) -> List[nodes.Node]:
+lang_node = self._highlightlang()
+
content_node: nodes.Element = nodes.section()
+
+# Configure QMP highlighting for "::" blocks, if needed
+if lang_node["lang"] != "QMP":
+content_node += addnodes.highlightlang(
+lang="QMP",
+force=False, # "True" ignores lexing errors
+linenothreshold=lang_node["linenothreshold"],
+)
+
self.do_parse(self.content, content_node)
+
+# Restore prior language highlighting, if needed
+if lang_node["lang"] != "QMP":
+content_node += addnodes.highlightlang(**lang_node.attributes)
+
return content_node.children
def run(self) -> List[nodes.Node]:
--
2.45.0
[PATCH v2 00/21] qapi: convert "Note" and "Example" sections to rST
This series focuses primarily on converting our existing QAPI/QMP
documentation to remove special "Note" and "Example" sections in favor
of rST markup for the same.
This is being done primarily to reduce the number of specially parsed
QAPI sections we have in favor of allowing fully arbitrary rST markup
for greater flexibility and freedom in styling the rendered HTML
documentation.
(A tangible side benefit is that the new qapidoc generator has fewer
sections to reason about when it splices inherited documentation
together for the QMP reference manual; docs largely preserve the order
of documentation "as written" instead of needing to splice multiple
separate sections together. A goal of the new generator is to eventually
remove all tagged sections except for "members" and "features".)
V2:
- Lotsa stuff :{
- Added new example box styling and sphinx directive macro
- Addressed... most? list feedback. (But not the indent split.)
Known issues:
- The Example boxes are styled a little wonkily if prose follows the
code-block instead of precedes it, fixup will be tomorrow.
--js
Harmonie Snow (1):
docs/sphinx: add CSS styling for qmp-example directive
John Snow (20):
[DO-NOT-MERGE]: Add some ad-hoc linting helpers.
qapi: linter fixups
docs/qapidoc: remove unused intersperse function
docs/qapidoc: delint a tiny portion of the module
qapi/parser: preserve indentation in QAPIDoc sections
qapi/parser: fix comment parsing immediately following a doc block
docs/qapidoc: fix nested parsing under untagged sections
qapi: fix non-compliant JSON examples
qapi: nail down convention that Errors sections are lists
qapi: convert "Note" sections to plain rST
qapi: update prose in note blocks
qapi: add markup to note blocks
qapi/parser: don't parse rST markup as section headers
docs/qapidoc: factor out do_parse()
docs/qapidoc: create qmp-example directive
docs/qapidoc: add QMP highlighting to annotated qmp-example blocks
qapi: convert "Example" sections without titles
qapi: convert "Example" sections with titles
qapi: convert "Example" sections with longer prose
qapi: remove "Example" doc section
docs/devel/qapi-code-gen.rst | 25 +-
docs/sphinx-static/theme_overrides.css| 46
docs/sphinx/qapidoc.py| 258 +-
qapi/acpi.json| 4 +-
qapi/block-core.json | 116
qapi/block.json | 59 ++--
qapi/char.json| 36 ++-
qapi/control.json | 28 +-
qapi/dump.json| 10 +-
qapi/introspect.json | 6 +-
qapi/machine-target.json | 28 +-
qapi/machine.json | 117
qapi/migration.json | 104 +++
qapi/misc-target.json | 22 +-
qapi/misc.json| 123 +
qapi/net.json | 32 ++-
qapi/pci.json | 10 +-
qapi/qdev.json| 40 +--
qapi/qom.json | 33 ++-
qapi/replay.json | 8 +-
qapi/rocker.json | 26 +-
qapi/run-state.json | 48 ++--
qapi/sockets.json | 10 +-
qapi/stats.json | 22 +-
qapi/tpm.json | 6 +-
qapi/trace.json | 4 +-
qapi/transaction.json | 12 +-
qapi/ui.json | 76 +++---
qapi/vfio.json| 2 +-
qapi/virtio.json | 57 ++--
qapi/yank.json| 4 +-
qga/qapi-schema.json | 48 ++--
scripts/qapi-lint.sh | 61 +
scripts/qapi/Makefile | 5 +
scripts/qapi/introspect.py| 8 +-
scripts/qapi/parser.py| 38 ++-
scripts/qapi/schema.py| 6 +-
scripts/qapi/visit.py | 5 +-
tests/qapi-schema/doc-empty-section.err | 2 +-
tests/qapi-schema/doc-empty-section.json | 2 +-
tests/qapi-schema/doc-good.json | 28 +-
tests/qapi-schema/doc-good.out| 61 +++--
tests/qapi-schema/doc-good.txt| 36 ++-
.../qapi-schema/doc-interleaved-section.json | 2 +-
44 files changed, 1010 insertions(+), 664 deletions(-)
create mode 100755 scripts/qapi-lint.sh
create mode 100644 scripts/qapi/Makefile
--
2.45.0
[PATCH v2 09/21] qapi: nail down convention that Errors sections are lists
By unstated convention, Errors sections are rST lists. Document the convention, and make the one exception conform. Signed-off-by: John Snow Reviewed-by: Markus Armbruster --- docs/devel/qapi-code-gen.rst | 7 +++ qapi/transaction.json| 2 +- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst index f453bd35465..cee43222f19 100644 --- a/docs/devel/qapi-code-gen.rst +++ b/docs/devel/qapi-code-gen.rst @@ -1011,6 +1011,13 @@ like this:: "Returns" and "Errors" sections are only valid for commands. They document the success and the error response, respectively. +"Errors" sections should be formatted as an rST list, each entry +detailing a relevant error condition. For example:: + + # Errors: + # - If @device does not exist, DeviceNotFound + # - Any other error returns a GenericError. + A "Since: x.y.z" tagged section lists the release that introduced the definition. diff --git a/qapi/transaction.json b/qapi/transaction.json index 5749c133d4a..07afc269d54 100644 --- a/qapi/transaction.json +++ b/qapi/transaction.json @@ -235,7 +235,7 @@ # additional detail. # # Errors: -# Any errors from commands in the transaction +# - Any errors from commands in the transaction # # Note: The transaction aborts on the first failure. Therefore, there # will be information on only one failed operation returned in an -- 2.45.0
[PATCH v2 04/21] docs/qapidoc: delint a tiny portion of the module
In a forthcoming series that adds a new QMP documentation generator, it
will be helpful to have a linting baseline. However, there's no need to
shuffle around the deck chairs too much, because most of this code will
be removed once that new qapidoc generator (the "transmogrifier") is in
place.
To ease my pain: just turn off the black auto-formatter for most, but
not all, of qapidoc.py. This will help ensure that *new* code follows a
coding standard without bothering too much with cleaning up the existing
code.
Code that I intend to keep is still subject to the delinting beam.
Signed-off-by: John Snow
Reviewed-by: Markus Armbruster
---
docs/sphinx/qapidoc.py | 62 +-
1 file changed, 37 insertions(+), 25 deletions(-)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index 3c0565d0ceb..659e507353a 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -28,26 +28,33 @@
import re
from docutils import nodes
+from docutils.parsers.rst import Directive, directives
from docutils.statemachine import ViewList
-from docutils.parsers.rst import directives, Directive
-from sphinx.errors import ExtensionError
-from sphinx.util.nodes import nested_parse_with_titles
-import sphinx
-from qapi.gen import QAPISchemaVisitor
from qapi.error import QAPIError, QAPISemError
+from qapi.gen import QAPISchemaVisitor
from qapi.schema import QAPISchema
+import sphinx
+from sphinx.errors import ExtensionError
+from sphinx.util.nodes import nested_parse_with_titles
+
# Sphinx up to 1.6 uses AutodocReporter; 1.7 and later
# use switch_source_input. Check borrowed from kerneldoc.py.
-Use_SSI = sphinx.__version__[:3] >= '1.7'
-if Use_SSI:
+USE_SSI = sphinx.__version__[:3] >= "1.7"
+if USE_SSI:
from sphinx.util.docutils import switch_source_input
else:
-from sphinx.ext.autodoc import AutodocReporter
+from sphinx.ext.autodoc import ( # pylint: disable=no-name-in-module
+AutodocReporter,
+)
-__version__ = '1.0'
+__version__ = "1.0"
+
+
+# Disable black auto-formatter until re-enabled:
+# fmt: off
class QAPISchemaGenRSTVisitor(QAPISchemaVisitor):
@@ -441,6 +448,10 @@ def get_document_nodes(self):
return self._top_node.children
+# Turn the black formatter on for the rest of the file.
+# fmt: on
+
+
class QAPISchemaGenDepVisitor(QAPISchemaVisitor):
"""A QAPI schema visitor which adds Sphinx dependencies each module
@@ -448,34 +459,34 @@ class QAPISchemaGenDepVisitor(QAPISchemaVisitor):
that the generated documentation output depends on the input
schema file associated with each module in the QAPI input.
"""
+
def __init__(self, env, qapidir):
self._env = env
self._qapidir = qapidir
def visit_module(self, name):
if name != "./builtin":
-qapifile = self._qapidir + '/' + name
+qapifile = self._qapidir + "/" + name
self._env.note_dependency(os.path.abspath(qapifile))
super().visit_module(name)
class QAPIDocDirective(Directive):
"""Extract documentation from the specified QAPI .json file"""
+
required_argument = 1
optional_arguments = 1
-option_spec = {
-'qapifile': directives.unchanged_required
-}
+option_spec = {"qapifile": directives.unchanged_required}
has_content = False
def new_serialno(self):
"""Return a unique new ID string suitable for use as a node's ID"""
env = self.state.document.settings.env
-return 'qapidoc-%d' % env.new_serialno('qapidoc')
+return "qapidoc-%d" % env.new_serialno("qapidoc")
def run(self):
env = self.state.document.settings.env
-qapifile = env.config.qapidoc_srctree + '/' + self.arguments[0]
+qapifile = env.config.qapidoc_srctree + "/" + self.arguments[0]
qapidir = os.path.dirname(qapifile)
try:
@@ -513,13 +524,14 @@ def do_parse(self, rstlist, node):
# plain self.state.nested_parse(), and so we can drop the saving
# of title_styles and section_level that kerneldoc.py does,
# because nested_parse_with_titles() does that for us.
-if Use_SSI:
+if USE_SSI:
with switch_source_input(self.state, rstlist):
nested_parse_with_titles(self.state, rstlist, node)
else:
save = self.state.memo.reporter
self.state.memo.reporter = AutodocReporter(
-rstlist, self.state.memo.reporter)
+rstlist, self.state.memo.reporter
+)
try:
nested_parse_with_titles(self.state, rstlist, node)
finally:
@@ -527,12 +539,12 @@ def do_parse(self, rstlist, node):
def setup(app):
-"""
[PATCH v2 01/21] [DO-NOT-MERGE]: Add some ad-hoc linting helpers.
These aren't ready for upstream inclusion, because they do not properly manage version dependencies, execution environment and so on. These are just the tools I use in my Own Special Environment :tm: for testing and debugging. They've been tested only on Fedora 38 for right now, which means: Python 3.11.4-1.fc38 pylint 2.17.4-2.fc38 mypy 1.4.0-1.fc38 isort 5.12.0-1.fc38 flake8 5.0.3-2.fc38 "Soon" :tm: I'll move the qapi generator code under the python/ directory to take advantage of the more robust linting infrastructure there, but that's going to happen after this series. Sorry about that! Signed-off-by: John Snow --- scripts/qapi-lint.sh | 61 +++ scripts/qapi/Makefile | 5 2 files changed, 66 insertions(+) create mode 100755 scripts/qapi-lint.sh create mode 100644 scripts/qapi/Makefile diff --git a/scripts/qapi-lint.sh b/scripts/qapi-lint.sh new file mode 100755 index 000..732716f2444 --- /dev/null +++ b/scripts/qapi-lint.sh @@ -0,0 +1,61 @@ +#!/usr/bin/env bash +set -e + +if [[ -f qapi/.flake8 ]]; then +echo "flake8 --config=qapi/.flake8 qapi/" +flake8 --config=qapi/.flake8 qapi/ +fi +if [[ -f qapi/pylintrc ]]; then +echo "pylint --rcfile=qapi/pylintrc qapi/" +pylint --rcfile=qapi/pylintrc qapi/ +fi +if [[ -f qapi/mypy.ini ]]; then +echo "mypy --config-file=qapi/mypy.ini qapi/" +mypy --config-file=qapi/mypy.ini qapi/ +fi + +if [[ -f qapi/.isort.cfg ]]; then +pushd qapi +echo "isort -c ." +isort -c . +popd +fi + +if [[ -f ../docs/sphinx/qapi-domain.py ]]; then +pushd ../docs/sphinx + +echo "mypy --strict qapi-domain.py" +mypy --strict qapi-domain.py +echo "flake8 qapi-domain.py --max-line-length=99" +flake8 qapi-domain.py --max-line-length=99 +echo "isort qapi-domain.py" +isort qapi-domain.py +echo "black --check qapi-domain.py" +black --check qapi-domain.py + +popd +fi + +if [[ -f ../docs/sphinx/qapidoc.py ]]; then +pushd ../docs/sphinx + +echo "pylint --rc-file ../../scripts/qapi/pylintrc qapidoc.py" +PYTHONPATH=../scripts/ pylint \ + --rc-file ../../scripts/qapi/pylintrc \ + qapidoc.py +echo "flake8 qapidoc.py --max-line-length=80" +flake8 qapidoc.py --max-line-length=80 +echo "isort qapidoc.py" +isort qapidoc.py +echo "black --line-length 80 --check qapidoc.py" +black --line-length 80 --check qapidoc.py + +popd +fi + +pushd ../build +make -j13 +make check-qapi-schema +make docs +make sphinxdocs +popd diff --git a/scripts/qapi/Makefile b/scripts/qapi/Makefile new file mode 100644 index 000..314e8a5505e --- /dev/null +++ b/scripts/qapi/Makefile @@ -0,0 +1,5 @@ +check: + isort -c . + flake8 . + cd .. && pylint --rcfile=qapi/pylintrc qapi + cd .. && mypy -p qapi --config-file=qapi/mypy.ini -- 2.45.0
[PATCH v2 03/21] docs/qapidoc: remove unused intersperse function
This function has been unused since fd62bff901b. Signed-off-by: John Snow --- docs/sphinx/qapidoc.py | 10 -- 1 file changed, 10 deletions(-) diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py index f270b494f01..3c0565d0ceb 100644 --- a/docs/sphinx/qapidoc.py +++ b/docs/sphinx/qapidoc.py @@ -50,16 +50,6 @@ __version__ = '1.0' -# Function borrowed from pydash, which is under the MIT license -def intersperse(iterable, separator): -"""Yield the members of *iterable* interspersed with *separator*.""" -iterable = iter(iterable) -yield next(iterable) -for item in iterable: -yield separator -yield item - - class QAPISchemaGenRSTVisitor(QAPISchemaVisitor): """A QAPI schema visitor which generates docutils/Sphinx nodes -- 2.45.0
[PATCH v2 06/21] qapi/parser: fix comment parsing immediately following a doc block
If a comment immediately follows a doc block, the parser doesn't ignore
that token appropriately. Fix that.
e.g.
> ##
> # = Hello World!
> ##
>
> # I'm a comment!
will break the parser, because it does not properly ignore the comment
token if it immediately follows a doc block.
Fixes: 3d035cd2cca6 (qapi: Rewrite doc comment parser)
Signed-off-by: John Snow
Reviewed-by: Markus Armbruster
---
scripts/qapi/parser.py | 2 +-
tests/qapi-schema/doc-good.json | 2 ++
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index 1ef1f85b028..c3d20cc01bc 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -583,7 +583,7 @@ def get_doc(self) -> 'QAPIDoc':
line = self.get_doc_line()
first = False
-self.accept(False)
+self.accept()
doc.end()
return doc
diff --git a/tests/qapi-schema/doc-good.json b/tests/qapi-schema/doc-good.json
index de38a386e8f..8b39eb946af 100644
--- a/tests/qapi-schema/doc-good.json
+++ b/tests/qapi-schema/doc-good.json
@@ -55,6 +55,8 @@
# - {braces}
##
+# Not a doc comment
+
##
# @Enum:
#
--
2.45.0
Re: [PATCH 13/13] qapi: convert "Example" sections to rST
On Wed, Jun 26, 2024, 1:18 AM Markus Armbruster wrote:
> John Snow writes:
>
> > Eliminate the "Example" sections in QAPI doc blocks, converting them
> > into QMP example code blocks. This is generally done in this patch by
> > converting "Example:" or "Examples:" lines into ".. code-block:: QMP"
> > lines.
>
> [...]
>
> > diff --git a/qapi/migration.json b/qapi/migration.json
> > index 85a14bb4308..849358b6387 100644
> > --- a/qapi/migration.json
> > +++ b/qapi/migration.json
>
> [...]
>
> > @@ -336,7 +338,35 @@
> > # }
> > #}
> > #
> > -# 5. Migration is being performed and XBZRLE is active:
> > +# .. code-block:: QMP
> > +#:caption: Example: Migration is being performed and XBZRLE is
> active
> > +#
> > +# -> { "execute": "query-migrate" }
> > +# <- {
> > +# "return":{
> > +# "status":"active",
> > +# "total-time":12345,
> > +# "setup-time":12345,
> > +# "expected-downtime":12345,
> > +# "ram":{
> > +# "total":1057024,
> > +# "remaining":1053304,
> > +# "transferred":3720,
> > +# "duplicate":123,
> > +# "normal":123,
> > +# "normal-bytes":123456,
> > +# "dirty-sync-count":15
> > +# },
> > +# "disk":{
> > +# "total":20971520,
> > +# "remaining":20880384,
> > +# "transferred":91136
> > +# }
> > +# }
> > +#}
> > +#
> > +# .. code-block:: QMP
> > +#:caption: Example: Migration is being performed and XBZRLE is
> active
> > #
> > # -> { "execute": "query-migrate" }
> > # <- {
>
> Example accidentally duplicated.
>
Fixed this yesterday, oopsie. I think this was a rebase goof.
>
> [...]
>
> > diff --git a/tests/qapi-schema/doc-good.json
> b/tests/qapi-schema/doc-good.json
> > index 4b338cc0186..2774a7ce14d 100644
> > --- a/tests/qapi-schema/doc-good.json
> > +++ b/tests/qapi-schema/doc-good.json
> > @@ -46,11 +46,13 @@
> > #
> > # Duis aute irure dolor
> > #
> > -# Example:
> > +# .. code-block:: QMP
> > +#:caption: Example:
>
> See [*] below.
>
> > #
> > # -> in
> > # <- out
> > -# Examples:
> > +# .. code-block::
> > +#
>
> Likewise.
>
> > # - *verbatim*
> > # - {braces}
> > ##
> > @@ -172,12 +174,13 @@
> > #
> > # Duis aute irure dolor
> > #
> > -# Example:
> > +# .. code-block::
> > #
> > # -> in
> > # <- out
> > #
> > -# Examples:
> > +# .. code-block::
> > +#
> > # - *verbatim*
> > # - {braces}
> > #
> > @@ -196,7 +199,7 @@
> > # @cmd-feat1: a feature
> > # @cmd-feat2: another feature
> > #
> > -# Example:
> > +# .. code-block::
> > #
> > # -> in
> > #
> > diff --git a/tests/qapi-schema/doc-good.out
> b/tests/qapi-schema/doc-good.out
> > index 2c9b4e419cb..347b9cb7134 100644
> > --- a/tests/qapi-schema/doc-good.out
> > +++ b/tests/qapi-schema/doc-good.out
> > @@ -93,11 +93,13 @@ Notes:
> >
> > Duis aute irure dolor
> >
> > -Example:
> > +.. code-block:: QMP
> > + :caption: Example:
>
> [*] This demonstrates the "Example: ..." is *not* recognized as a
> "Example" section before the patch (compare to the change we get for
> recognized sections below).
>
> I pointed out the same issue for "Note" in review of PATCH 9, and
> suggested ways to resolve it. Pick a way there, and use it here as well
>
ACK
> >
> > -> in
> > <- out
> > -Examples:
> > +.. code-block::
> > +
> > - *verbatim*
> > - {braces}
> > doc symbol=Enum
> > @@ -184,10 +186,14 @@ frobnicate
> > - Ut enim ad minim veniam
> >
> > Duis aute irure dolor
> > -section=Example
> > +
> > +.. code-block::
> > +
> > -> in
> > <- out
> > -section=Examples
> > +
> > +.. code-block::
> >
Re: [PULL 39/42] i386: Add support for SUCCOR feature
On Thu, Jun 13, 2024 at 05:50:08PM +0800, Xiaoyao Li wrote:
> On 6/8/2024 4:34 PM, Paolo Bonzini wrote:
> > From: John Allen
> >
> > Add cpuid bit definition for the SUCCOR feature. This cpuid bit is required
> > to
> > be exposed to guests to allow them to handle machine check exceptions on AMD
> > hosts.
> >
> >
> > v2:
> >- Add "succor" feature word.
> >- Add case to kvm_arch_get_supported_cpuid for the SUCCOR feature.
> >
> > Reported-by: William Roche
> > Reviewed-by: Joao Martins
> > Signed-off-by: John Allen
> > Message-ID: <20240603193622.47156-3-john.al...@amd.com>
> > Signed-off-by: Paolo Bonzini
>
> [snip]
> ...
>
> > diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
> > index 55a9e8a70cf..56d8e2a89ec 100644
> > --- a/target/i386/kvm/kvm.c
> > +++ b/target/i386/kvm/kvm.c
> > @@ -532,6 +532,8 @@ uint32_t kvm_arch_get_supported_cpuid(KVMState *s,
> > uint32_t function,
> >*/
> > cpuid_1_edx = kvm_arch_get_supported_cpuid(s, 1, 0, R_EDX);
> > ret |= cpuid_1_edx & CPUID_EXT2_AMD_ALIASES;
> > +} else if (function == 0x8007 && reg == R_EBX) {
> > +ret |= CPUID_8000_0007_EBX_SUCCOR;
>
> IMO, this is incorrect.
>
> Why make it supported unconditionally? Shouldn't there be a KVM patch to
> report it in KVM_GET_SUPPORTED_CPUID?
>
> If make it supported unconditionally, all VMs boot with "-cpu host/max" will
> see the CPUID bits, even if it is Intel VMs.
Paolo,
This change in kvm_arch_get_supported_cpuid was added based on your
suggestion here:
https://lore.kernel.org/all/d4c1bb9b-8438-ed00-c79d-e8ad2a7e4...@redhat.com/
Is there something missing from the patch needed to prevent the bits
from being seen on Intel VMs?
I am planning to send a patch early this week to report the bits for KVM
and a patch that removes the above change for qemu. Is there another way
you would prefer to handle it?
Thanks,
John
>
>
> > } else if (function == KVM_CPUID_FEATURES && reg == R_EAX) {
> > /* kvm_pv_unhalt is reported by GET_SUPPORTED_CPUID, but it can't
> >* be enabled without the in-kernel irqchip
>
Re: [PATCH 05/20] qapi/parser: adjust info location for doc body section
On Mon, May 27, 2024 at 7:58 AM Markus Armbruster wrote:
> John Snow writes:
>
> > On Thu, May 16, 2024, 1:58 AM Markus Armbruster
> wrote:
> >
> >> John Snow writes:
> >>
> >> > Instead of using the info object for the doc block as a whole, update
> >> > the info pointer for each call to ensure_untagged_section when the
> >> > existing section is otherwise empty. This way, Sphinx error
> information
> >> > will match precisely to where the text actually starts.
> >> >
> >> > Signed-off-by: John Snow
> >> > ---
> >> > scripts/qapi/parser.py | 9 +++--
> >> > 1 file changed, 7 insertions(+), 2 deletions(-)
> >> >
> >> > diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
> >> > index 8cdd5334ec6..41b9319e5cb 100644
> >> > --- a/scripts/qapi/parser.py
> >> > +++ b/scripts/qapi/parser.py
> >> > @@ -662,8 +662,13 @@ def end(self) -> None:
> >> >
> >> > def ensure_untagged_section(self, info: QAPISourceInfo) -> None:
> >> > if self.all_sections and not self.all_sections[-1].tag:
> >> > -# extend current section
> >> > -self.all_sections[-1].text += '\n'
> >>
> >> Before, we always append a newline.
> >>
> >> > +section = self.all_sections[-1]
> >> > +# Section is empty so far; update info to start *here*.
> >> > +if not section.text:
> >> > +section.info = info
> >> > +else:
> >> > +# extend current section
> >> > +self.all_sections[-1].text += '\n'
> >>
> >> Afterwards, we append it only when the section already has some text.
> >>
> >> The commit message claims the patch only adjusts section.info. That's
> a
> >> lie :)
> >>
> >
> > Well. It wasn't intentional, so it wasn't a lie... it was just wrong :)
> >
> >
> >> I believe the change makes no difference because .end() strips leading
> >> and trailing newline.
> >>
> >> > return
> >> > # start new section
> >> > section = self.Section(info)
> >>
> >> You could fix the commit message, but I think backing out the
> >> no-difference change is easier. The appended patch works in my testing.
> >>
> >> Next one. Your patch changes the meaning of section.info. Here's its
> >> initialization:
> >>
> >> class Section:
> >> # pylint: disable=too-few-public-methods
> >> def __init__(self, info: QAPISourceInfo,
> >> tag: Optional[str] = None):
> >> ---># section source info, i.e. where it begins
> >> self.info = info
> >> # section tag, if any ('Returns', '@name', ...)
> >> self.tag = tag
> >> # section text without tag
> >> self.text = ''
> >>
> >> The comment is now wrong. Calls for a thorough review of .info's uses.
> >>
> >
> > Hmm... Did I really change its meaning? I guess it's debatable what
> "where
> > it begins" means. Does the tagless section start...
> >
> > ## <-- Here?
> > # Hello! <-- Or here?
> > ##
> >
> > I assert the *section* starts wherever the first line of text it contains
> > starts. Nothing else makes any sense.
> >
> > There is value in recording where the doc block starts, but that's not a
> > task for the *section* info.
> >
> > I don't think I understand your feedback.
>
> This was before my vacation, and my memory is foggy, ... I may have
> gotten confused back then. Let me have a fresh look now.
>
> self.info gets initialized in Section.__init__() to whatever info it
> gets passed.
>
> Your patch makes .ensure_untagged_section() overwrite this Section.info
> when it extends an untagged section that is still empty. Hmmm. I'd
> prefer .info to remain constant after initialization.
>
but, we don't have the right info when we initialize the entire QAPIDoc
object, because the section hasn't truly actually started yet, so I don't
think I can actually achieve your preference.
>
> I figure this overwrite can happen only when extenting the body section
> QAPIDoc.__init__() creates. In that case, it adjusts .info from
> beginning of doc comment to first non-bla
Re: [PATCH 09/13] qapi: convert "Note" sections to plain rST
On Fri, Jun 21, 2024 at 8:23 AM Markus Armbruster wrote: > John Snow writes: > > > On Thu, Jun 20, 2024 at 11:46 AM John Snow wrote: > > > >> > >> > >> On Thu, Jun 20, 2024, 9:35 AM Markus Armbruster > wrote: > >> > >>> Markus Armbruster writes: > >>> > >>> > John Snow writes: > >>> > >>> [...] > >>> > >>> >> diff --git a/qga/qapi-schema.json b/qga/qapi-schema.json > >>> >> index b3de1fb6b3a..57598331c5c 100644 > >>> >> --- a/qga/qapi-schema.json > >>> >> +++ b/qga/qapi-schema.json > >>> > >>> [...] > >>> > >>> >> @@ -631,8 +632,8 @@ > >>> >> # Errors: > >>> >> # - If hybrid suspend is not supported, Unsupported > >>> >> # > >>> >> -# Notes: It's strongly recommended to issue the guest-sync command > >>> >> -# before sending commands when the guest resumes > >>> >> +# .. note:: It's strongly recommended to issue the guest-sync > command > >>> >> +#before sending commands when the guest resumes. > >>> >> # > >>> >> # Since: 1.1 > >>> >> ## > >>> >> @@ -1461,16 +1462,15 @@ > >>> >> # * POSIX: as defined by os-release(5) > >>> >> # * Windows: contains string "server" or "client" > >>> >> # > >>> >> -# Notes: On POSIX systems the fields @id, @name, @pretty-name, > >>> >> -# @version, @version-id, @variant and @variant-id follow the > >>> >> -# definition specified in os-release(5). Refer to the manual > page > >>> >> -# for exact description of the fields. Their values are taken > >>> >> -# from the os-release file. If the file is not present in the > >>> >> -# system, or the values are not present in the file, the fields > >>> >> -# are not included. > >>> >> +# .. note:: On POSIX systems the fields @id, @name, @pretty-name, > >>> >> +#@version, @version-id, @variant and @variant-id follow the > >>> >> +#definition specified in os-release(5). Refer to the manual > page > >>> for > >>> >> +#exact description of the fields. Their values are taken from > the > >>> >> +#os-release file. If the file is not present in the system, or > >>> the > >>> >> +#values are not present in the file, the fields are not > included. > >>> >> # > >>> >> -# On Windows the values are filled from information gathered > from > >>> >> -# the system. > >>> >> +#On Windows the values are filled from information gathered > from > >>> >> +#the system. > >>> > > >>> > Please don't change the indentation here. I get the same output with > >>> > > >>> > @@ -1461,7 +1462,7 @@ > >>> ># * POSIX: as defined by os-release(5) > >>> ># * Windows: contains string "server" or "client" > >>> ># > >>> > -# Notes: On POSIX systems the fields @id, @name, @pretty-name, > >>> > +# .. note:: On POSIX systems the fields @id, @name, @pretty-name, > >>> ># @version, @version-id, @variant and @variant-id follow the > >>> ># definition specified in os-release(5). Refer to the manual > page > >>> ># for exact description of the fields. Their values are taken > >>> > >>> I'm blind. Actually, you change indentation of subsequent lines from 4 > >>> to 3 *everywhere*. I guess you do that to make subsequent lines line > up > >>> with the directive, here "note". > >>> > >>> Everywhere else, we indent such lines by 4. Hmm. How terrible would > it > >>> be not to mess with the alignment? > >>> > >>> If we want to use 3 for directives, is it worth pointing out in the > >>> commit message? > >>> > >>> [...] > >>> > >> > >> Let me look up some conventions and see what's the most prominent... as > >> well as testing what emacs does by default (or if emacs can be > configured > >> to do what we want with in-tree style config. Warning
Re: [PATCH 09/13] qapi: convert "Note" sections to plain rST
On Fri, Jun 21, 2024 at 8:08 AM Markus Armbruster wrote:
> John Snow writes:
>
> > We do not need a dedicated section for notes. By eliminating a specially
> > parsed section, these notes can be treated as normal rST paragraphs in
> > the new QMP reference manual, and can be placed and styled much more
> > flexibly.
> >
> > Convert all existing "Note" and "Notes" sections to pure rST. As part of
> > the conversion, capitalize the first letter of each sentence and add
> > trailing punctuation where appropriate to ensure notes look sensible and
> > consistent in rendered HTML documentation.
> >
> > Update docs/devel/qapi-code-gen.rst to reflect the new paradigm, and ...
> >
> > ... Update the QAPI parser to prohibit "Note" sections while suggesting
> > a new syntax. The exact formatting to use is a matter of taste, but a
> > good candidate is simply:
> >
> > .. note:: lorem ipsum ...
> >
> > ... but there are other choices, too. The Sphinx readthedocs theme
> > offers theming for the following forms (capitalization unimportant); all
> > are adorned with a (!) symbol in the title bar for rendered HTML docs.
> >
> > See
> >
> https://sphinx-rtd-theme.readthedocs.io/en/stable/demo/demo.html#admonitions
> > for examples of each directive/admonition in use.
> >
> > These are rendered in orange:
> >
> > .. Attention:: ...
> > .. Caution:: ...
> > .. WARNING:: ...
> >
> > These are rendered in red:
> >
> > .. DANGER:: ...
> > .. Error:: ...
> >
> > These are rendered in green:
> >
> > .. Hint:: ...
> > .. Important:: ...
> > .. Tip:: ...
> >
> > These are rendered in blue:
> >
> > .. Note:: ...
> > .. admonition:: custom title
> >
> >admonition body text
> >
> > This patch uses ".. note::" almost everywhere, with just two "caution"
> > directives. ".. admonition:: notes" is used in a few places where we had
> > an ordered list of multiple notes that would not make sense as
> > standalone/separate admonitions.
> >
> > Signed-off-by: John Snow
> > Acked-by: Stefan Hajnoczi [for block*.json]
>
> [...]
>
> > diff --git a/qapi/qom.json b/qapi/qom.json
> > index 8bd299265e3..5bfa0ded42c 100644
> > --- a/qapi/qom.json
> > +++ b/qapi/qom.json
> > @@ -195,12 +195,12 @@
> > #
> > # @typename: the type name of an object
> > #
> > -# Note: objects can create properties at runtime, for example to
> > -# describe links between different devices and/or objects. These
> > -# properties are not included in the output of this command.
> > -#
> > # Returns: a list of ObjectPropertyInfo describing object properties
> > #
> > +# .. note:: Objects can create properties at runtime, for example to
> > +#describe links between different devices and/or objects. These
> > +#properties are not included in the output of this command.
> > +#
> > # Since: 2.12
> > ##
>
> You move the note. Commit message doesn't tell why.
>
> > { 'command': 'qom-list-properties',
>
> [...]
>
"v2" of this series now declines to move the note in this patch and instead
moves it in a separate patch that also enforces source order more strictly
so that the move can be explained in detail.
Rendering order diverges from source order briefly as a result; I will
mention that in the commit message instead.
(I don't think it's easy or worth doing to re-order the patches such that
source and render order never diverge; too much engineering for so
temporary a minor issue. Not to mention the source and render order is
already divergent in many places, so I don't think it's a regression so
much as it is a temporary lateralgression?)
Re: [PATCH 04/13] qapi/parser: preserve indentation in QAPIDoc sections
On Fri, Jun 21, 2024 at 2:38 AM Markus Armbruster wrote: > John Snow writes: > > > On Thu, Jun 20, 2024, 11:07 AM Markus Armbruster > wrote: > > > >> John Snow writes: > >> > >> > On Wed, Jun 19, 2024, 8:03 AM Markus Armbruster > wrote: > >> > > >> >> John Snow writes: > >> >> > >> >> > Change get_doc_indented() to preserve indentation on all > subsequent text > >> >> > lines, and create a compatibility dedent() function for qapidoc.py > to > >> >> > remove that indentation. This is being done for the benefit of a > new > >> >> > >> >> Suggest "remove indentation the same way get_doc_indented() did." > >> >> > >> > > >> > Aight. > >> > > >> > > >> >> > qapidoc generator which requires that indentation in argument and > >> >> > features sections are preserved. > >> >> > > >> >> > Prior to this patch, a section like this: > >> >> > > >> >> > ``` > >> >> > @name: lorem ipsum > >> >> >dolor sit amet > >> >> > consectetur adipiscing elit > >> >> > ``` > >> >> > > >> >> > would have its body text be parsed as: > >> >> > >> >> Suggest "parsed into". > >> >> > >> > > >> > Why? (I mean, I'll do it, but I don't see the semantic difference > >> > personally) > >> > > >> > >> "Parse as " vs. "Parse into ". > >> > >> >> > (first and final newline only for presentation) > >> >> > > >> >> > ``` > >> >> > lorem ipsum > >> >> > dolor sit amet > >> >> > consectetur adipiscing elit > >> >> > ``` > >> >> > > >> >> > We want to preserve the indentation for even the first body line > so that > >> >> > the entire block can be parsed directly as rST. This patch would > now > >> >> > parse that segment as: > >> >> > >> >> If you change "parsed as" to "parsed into" above, then do it here, > too. > >> >> > >> >> > > >> >> > ``` > >> >> > lorem ipsum > >> >> >dolor sit amet > >> >> > consectetur adipiscing elit > >> >> > ``` > >> >> > > >> >> > This is helpful for formatting arguments and features as field > lists in > >> >> > rST, where the new generator will format this information as: > >> >> > > >> >> > ``` > >> >> > :arg type name: lorem ipsum > >> >> >dolor sit amet > >> >> > consectetur apidiscing elit > >> >> > ``` > >> >> > > >> >> > ...and can be formed by the simple concatenation of the field list > >> >> > construct and the body text. The indents help preserve the > continuation > >> >> > of a block-level element, and further allow the use of additional > rST > >> >> > block-level constructs such as code blocks, lists, and other such > >> >> > markup. Avoiding reflowing the text conditionally also helps > preserve > >> >> > source line context for better rST error reporting from sphinx > through > >> >> > generated source, too. > >> >> > >> >> What do you mean by "reflowing"? > >> >> > >> > > >> > Poorly phrased, was thinking about emacs too much. I mean munging the > text > >> > post-hoc for the doc generator such that newlines are added or > removed in > >> > the process of re-formatting text to get the proper indentation for > the new > >> > rST form. > >> > > >> > In prototyping, this got messy very quickly and was difficult to > correlate > >> > source line numbers across the transformation. > >> > > >> > It was easier to just not munge the text at all instead of munging it > and > >> > then un-munging it. > >> > > >> > (semantic satiation: munge munge munge munge.) > >> > >> Is this about a possible alternative solution you explored? Keeping > >> .get_doc_indented() as is, and then try to undo its damage? > >> > > > > precisamente. That solution was categorically worse. > > Since .get_doc_indented() removes N spaces of indentation, we'd want to > add back N spaces of indentation. But we can't know N, so I guess we'd > make do with an arbitrary number. Where would reflowing come it? > > I'd like you to express more clearly that you're talking about an > alternative you rejected. Perhaps like this: > > block-level constructs such as code blocks, lists, and other such > markup. > > The alternative would be to somehow undo .get_doc_indented()'s > indentation changes in the new generator. Much messier. > > Feel free to add more detail to the last paragraph. > Eh, I just deleted it. I recall running into troubles but I can't articulate the precise conditions because as you point out, it's a doomed strategy for other reasons - you can't reconstruct the proper indentation. This patch is still the correct way to go, so I don't have to explain my failures at length in the commit message ... I just like giving people clues for *why* I decided to implement things a certain way, because I often find that more instructive than the "how". In this case, the "why" is probably more properly summarized as "it's a total shitshow in that direction, trust me" --js
