On Tue, Jul 9, 2024 at 6:52 AM Markus Armbruster <arm...@redhat.com> wrote:
> John Snow <js...@redhat.com> writes:
>
> > Fully eliminate the "Example" sections in QAPI doc blocks now that they
> > have all been converted to arbitrary rST syntax using the
> > ".. qmp-example::" directive. Update tests to match.
> >
> > Migrating to the new syntax
> > ---------------------------
> >
> > The old "Example:" or "Examples:" section syntax is now caught as an
> > error, but "Example::" is stil permitted as explicit rST syntax for an
> > un-lexed, generic preformatted text block.
> >
> > ('Example' is not special in this case, any sentence that ends with "::"
> > will start an indented code block in rST.)
> >
> > Arbitrary rST for Examples is now possible, but it's strongly
> > recommended that documentation authors use the ".. qmp-example::"
> > directive for consistent visual formatting in rendered HTML docs. The
> > ":title:" directive option may be used to add extra information into the
> > title bar for the example. The ":annotated:" option can be used to write
> > arbitrary rST instead, with nested "::" blocks applying QMP formatting
> > where desired.
> >
> > Other choices available are ".. code-block:: QMP" which will not create
> > an "Example:" box, or the short-form "::" code-block syntax which will
> > not apply QMP highlighting when used outside of the qmp-example
> > directive.
> >
> > Why?
> > ----
> >
> > This patch has several benefits:
> >
> > 1. Example sections can now be written more arbitrarily, mixing
> > explanatory paragraphs and code blocks however desired.
> >
> > 2. Example sections can now use fully arbitrary rST.
> >
> > 3. All code blocks are now lexed and validated as QMP; increasing
> > usability of the docs and ensuring validity of example snippets.
> >
> > (To some extent - This patch only gaurantees it lexes correctly, not
> > that it's valid under the JSON or QMP grammars. It will catch most
> > small mistakes, however.)
> >
> > 4. Each qmp-example can be titled or annotated independently without
> > bypassing the QMP lexer/validator.
> >
> > (i.e. code blocks are now for *code* only, so we don't have to
> > sacrifice exposition for having lexically valid examples.)
> >
> > NOTE: As with the "Notes" conversion patch,
>
> Commit d461c279737 (qapi: convert "Note" sections to plain rST).
>
Didn't have a stable commit ID at the time, will work it in if/when the
notes patches hit main.
>
> > this patch (and those
> > preceding) may change the rendering order for Examples in the
>
> The three preceding ones, to be precise.
>
> > current generator. The forthcoming qapidoc rewrite will fix this
> > by always generating documentation in source order.
>
> Conversions from "Example" section to plain reST may change order. This
> patch converts a test, and the preceding three convert the real uses.
>
> Does any of the patches actually change order?
>
I do not actually know ...! It has the *potential* in the same exact way
that the notes patch did, but I don't actually know if it *did*. My hunch
is "no" because there's only one intermediate section we identified with
the notes series, but I didn't exhaustively prove it. That's why I used the
"may" weasel wording.
>
> > Signed-off-by: John Snow <js...@redhat.com>
> > ---
> > docs/devel/qapi-code-gen.rst | 58 ++++++++++++++++++++++++++++-----
> > scripts/qapi/parser.py | 10 +++++-
> > tests/qapi-schema/doc-good.json | 19 +++++++----
> > tests/qapi-schema/doc-good.out | 26 ++++++++++-----
> > tests/qapi-schema/doc-good.txt | 23 ++++++-------
> > 5 files changed, 98 insertions(+), 38 deletions(-)
> >
> > diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
> > index ae97b335cbf..2e10a3cbd69 100644
> > --- a/docs/devel/qapi-code-gen.rst
> > +++ b/docs/devel/qapi-code-gen.rst
> > @@ -899,7 +899,7 @@ Documentation markup
> > ~~~~~~~~~~~~~~~~~~~~
> >
> > Documentation comments can use most rST markup. In particular,
> > -a ``::`` literal block can be used for examples::
> > +a ``::`` literal block can be used for pre-formatted text::
> >
> > # ::
> > #
> > @@ -995,8 +995,8 @@ line "Features:", like this::
> > # @feature: Description text
> >
> > A tagged section begins with a paragraph that starts with one of the
> > -following words: "Since:", "Example:"/"Examples:", "Returns:",
> > -"Errors:", "TODO:". It ends with the start of a new section.
> > +following words: "Since:", "Returns:", "Errors:", "TODO:". It ends with
> > +the start of a new section.
> >
> > The second and subsequent lines of tagged sections must be indented
> > like this::
> > @@ -1020,13 +1020,53 @@ detailing a relevant error condition. For
> example::
> > A "Since: x.y.z" tagged section lists the release that introduced the
> > definition.
> >
> > -An "Example" or "Examples" section is rendered entirely
> > -as literal fixed-width text. "TODO" sections are not rendered at all
> > -(they are for developers, not users of QMP). In other sections, the
> > -text is formatted, and rST markup can be used.
> > +"TODO" sections are not rendered at all (they are for developers, not
>
> Drop "at all"?
>
Sure.
>
> > +users of QMP). In other sections, the text is formatted, and rST markup
> > +can be used.
> > +
> > +QMP Examples can be added by using the ``.. qmp-example::``
> > +directive. In its simplest form, this can be used to contain a single
> > +QMP code block which accepts standard JSON syntax with additional server
> > +directionality indicators (``->`` and ``<-``), and elisions (``...``).
> > +
> > +Optionally, a plaintext title may be provided by using the ``:title:``
> > +directive option. If the title is omitted, the example title will
> > +default to "Example:".
> > +
> > +A simple QMP example::
> > +
> > + # .. qmp-example::
> > + # :title: Using query-block
> > + #
> > + # -> { "execute": "query-block" }
> > + # <- { ... }
> > +
> > +More complex or multi-step examples where exposition is needed before or
> > +between QMP code blocks can be created by using the ``:annotated:``
> > +directive option. When using this option, nested QMP code blocks must be
> > +entered explicitly with rST's ``::`` syntax.
>
Telling on myself: you can use .. code-block:: QMP too, but I figured
recommending "::" was shorter and sweeter. There are lots of minutiae here
for people who haven't spent a long time reading and writing rST, so I
tried to keep it short.
> > +
> > +Highlighting in non-QMP languages can be accomplished by using the
> > +``.. code-block:: lang`` directive, and non-highlighted text can be
> > +achieved by omitting the language argument.
> >
> > For example::
> >
> > + # .. qmp-example::
> > + # :annotated:
> > + # :title: A more complex demonstration
> > + #
> > + # This is a more complex example that can use
> > + # ``arbitrary rST syntax`` in its exposition::
> > + #
> > + # -> { "execute": "query-block" }
> > + # <- { ... }
> > + #
> > + # Above, lengthy output has been omitted for brevity.
> > +
> > +
> > +Examples of complete definition documentation::
> > +
> > ##
> > # @BlockStats:
> > #
> > @@ -1058,11 +1098,11 @@ For example::
> > #
> > # Since: 0.14
> > #
> > - # Example:
> > + # .. qmp-example::
> > #
> > # -> { "execute": "query-blockstats" }
> > # <- {
> > - # ... lots of output ...
> > + # ...
> > # }
> > ##
> > { 'command': 'query-blockstats',
> > diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
> > index 6ad5663e545..adc85b5b394 100644
> > --- a/scripts/qapi/parser.py
> > +++ b/scripts/qapi/parser.py
> > @@ -553,7 +553,7 @@ def get_doc(self) -> 'QAPIDoc':
> > # Note: "sections" with two colons are left alone as
> > # rST markup and not interpreted as a section
> heading.
> >
> > - # TODO: Remove this error sometime in 2025 or so
> > + # TODO: Remove these errors sometime in 2025 or so
> > # after we've fully transitioned to the new qapidoc
> > # generator.
> >
> > @@ -567,6 +567,14 @@ def get_doc(self) -> 'QAPIDoc':
> > )
> > raise QAPIParseError(self, emsg)
> >
> > + if 'Example' in match.group(1):
> > + emsg = (
> > + f"The '{match.group(1)}' section is no
> longer "
> > + "supported. Please use the '..
> qmp-example::' "
> > + "directive, or other suitable markup
> instead."
> > + )
> > + raise QAPIParseError(self, emsg)
> > +
> > doc.new_tagged_section(self.info, match.group(1))
> > text = line[match.end():]
> > if text:
> > diff --git a/tests/qapi-schema/doc-good.json
> b/tests/qapi-schema/doc-good.json
> > index 107123f8a8d..c71d65cd51f 100644
> > --- a/tests/qapi-schema/doc-good.json
> > +++ b/tests/qapi-schema/doc-good.json
> > @@ -172,12 +172,17 @@
> > #
> > # Duis aute irure dolor
> > #
> > -# Example:
> > +# .. qmp-example::
> > +# :title: Ideal fast-food burger situation
> > #
> > -# -> in
> > -# <- out
> > +# -> "in"
> > +# <- "out"
>
> Heh, trickery to make the text right of -> and <- JSON.
>
O:-)
It's maybe *slightly* bad form, but it does help illustrate how the new
directive is transformed when using text output modes in the test suite, so
I kept it. The new directive will simply not allow malformed JSON, so this
seemed like the simplest way to cheese that.
>
> > #
> > -# Examples:
> > +# Examples::
> > +#
> > +# - Not a QMP code block
> > +# - Merely a preformatted code block literal
> > +# It isn't even an rST list.
> > # - *verbatim*
> > # - {braces}
>
(And here, we test the use of non-QMP code block literals, esp. after the
qmp-example directive, proving that language settings have been restored to
defaults.)
> > #
> > @@ -199,11 +204,11 @@
> > # @cmd-feat1: a feature
> > # @cmd-feat2: another feature
> > #
> > -# Example:
> > +# .. qmp-example::
> > #
> > -# -> in
> > +# -> "this example"
> > #
> > -# <- out
> > +# <- "has no title"
>
> Same trickery.
>
Do you want that changed ... ?
>
> > ##
> > { 'command': 'cmd-boxed', 'boxed': true,
> > 'data': 'Object',
> > diff --git a/tests/qapi-schema/doc-good.out
> b/tests/qapi-schema/doc-good.out
> > index bd876b6542d..eee18cd436a 100644
> > --- a/tests/qapi-schema/doc-good.out
> > +++ b/tests/qapi-schema/doc-good.out
> > @@ -184,13 +184,21 @@ frobnicate
> > - Ut enim ad minim veniam
> >
> > Duis aute irure dolor
> > - section=Example
> > - -> in
> > - <- out
> > - section=Examples
> > +
> > +.. qmp-example::
> > + :title: Ideal fast-food burger situation
> > +
> > + -> "in"
> > + <- "out"
> > +
> > +Examples::
> > +
> > + - Not a QMP code block
> > + - Merely a preformatted code block literal
> > + It isn't even an rST list.
> > - *verbatim*
> > - {braces}
> > - section=None
> > +
> > Note::
> > Ceci n'est pas une note
> > section=Since
> > @@ -202,10 +210,12 @@ If you're bored enough to read this, go see a
> video of boxed cats
> > a feature
> > feature=cmd-feat2
> > another feature
> > - section=Example
> > - -> in
> > + section=None
> > +.. qmp-example::
> >
> > - <- out
> > + -> "this example"
> > +
> > + <- "has no title"
> > doc symbol=EVT_BOXED
> > body=
> >
> > diff --git a/tests/qapi-schema/doc-good.txt
> b/tests/qapi-schema/doc-good.txt
> > index 30d457e5488..cb37db606a6 100644
> > --- a/tests/qapi-schema/doc-good.txt
> > +++ b/tests/qapi-schema/doc-good.txt
> > @@ -217,17 +217,16 @@ Notes:
> >
> > Duis aute irure dolor
> >
> > +Example: Ideal fast-food burger situation:
>
No comment on the american making fast food burger jokes? :-(
> >
> > -Example
> > -~~~~~~~
> > + -> "in"
> > + <- "out"
> >
> > - -> in
> > - <- out
> > -
> > -
> > -Examples
> > -~~~~~~~~
> > +Examples:
> >
> > + - Not a QMP code block
> > + - Merely a preformatted code block literal
> > + It isn't even an rST list.
> > - *verbatim*
> > - {braces}
> >
> > @@ -261,13 +260,11 @@ Features
> > "cmd-feat2"
> > another feature
> >
> > +Example::
> >
> > -Example
> > -~~~~~~~
> > + -> "this example"
> >
> > - -> in
> > -
> > - <- out
> > + <- "has no title"
> >
> >
> > "EVT_BOXED" (Event)
>
Does this adequately resolve your qualms about .txt rendering for examples?
--js
