Re: [PATCH v3 00/12] Convert QAPI doc comments to generate rST instead of texinfo

2020-03-06 Thread Peter Maydell
On Tue, 25 Feb 2020 at 14:04, Peter Maydell  wrote:
>
> This series switches all our QAPI doc comments over from
> texinfo format to rST.

> Git branch of this series also available at
> https://git.linaro.org/people/peter.maydell/qemu-arm.git sphinx-conversions

I've just updated this git branch with a rebase on top of:
 * current master, which has the qemu-doc conversion
 * the minor-cleanups patchset I just posted

No serious changes to the meat of the series:
 * some new text in migration.json needed indenting to
   match this series' stricter requirements
 * fixups of (textual) makefile conflicts
 * five new patches at the end which delete all the
   texinfo machinery since once the qapi generation
   is converted we don't need texinfo at all

I'll probably send a new set of patches to the list next
week; this set should still be fine for review though.

thanks
-- PMM



Re: [PATCH v3 00/12] Convert QAPI doc comments to generate rST instead of texinfo

2020-02-26 Thread Markus Armbruster
Peter Maydell  writes:

> This series switches all our QAPI doc comments over from
> texinfo format to rST.
>
> I've pushed out a v3 because a lot of v2 is now in master and
> it seems like it might be easier (and less intimidating :-))
> to review a patchset that's only got the remaining work and
> which is based on current master.

Appreciated!

> Changes v2->v3:
>  * all the "preliminary tidy up of existing doc comment" patches
>are now in master -- thanks!
>  * rebased on current master (there were some minor conflicts with
>the just-committed creation of the tools manual)
>
> We have 3 weeks left til softfreeze, so I'm still hopeful we
> can land this in this release cycle, though I suppose it would
> not be a disaster if it landed in the next. (I'm guessing we
> will not complete conversion of the other bits of Texinfo to
> rST this cycle, anyway.)

I'll try to review in time.  Plenty of competition, so no promises.

[...]




[PATCH v3 00/12] Convert QAPI doc comments to generate rST instead of texinfo

2020-02-25 Thread Peter Maydell
This series switches all our QAPI doc comments over from
texinfo format to rST.

I've pushed out a v3 because a lot of v2 is now in master and
it seems like it might be easier (and less intimidating :-))
to review a patchset that's only got the remaining work and
which is based on current master.

Changes v2->v3:
 * all the "preliminary tidy up of existing doc comment" patches
   are now in master -- thanks!
 * rebased on current master (there were some minor conflicts with
   the just-committed creation of the tools manual)

We have 3 weeks left til softfreeze, so I'm still hopeful we
can land this in this release cycle, though I suppose it would
not be a disaster if it landed in the next. (I'm guessing we
will not complete conversion of the other bits of Texinfo to
rST this cycle, anyway.)

(All the text below is the same as the v2 cover letter, if
you've already read that.)


Changes v1->v2:
 * rebased (a few minor conflicts fixed)
 * I have fixed the failures to pass "make check"
 * minor tweaks to commit messages etc (noted in individual patches)
 * the old patch 12 ('qapi: Explicitly put "foo: dropped in n.n" notes
   into Notes section') has been deleted
 * patch 18 ('qapi: Delete all the "foo: dropped in n.n" notes')
 * I have not made the change to be more consistent about treating
   an apparent heading-comment with trailing lines of rST the same
   way as we would treat one with leading lines of rST, just because
   the whole area of how we handle headings is up in the air anyway.
   If we decide the approach here is basically right I'll make this
   change in a v3; otherwise it's likely to be moot anyway.
 * I have also not added a patch that rewraps long lines added
   by some of the earlier doc-comment adjustments; I figure we
   can come back and do that later.
 * I haven't (yet) written an extra patch that tries to guess
   what might be a good sphinx-build binary to use (none of my
   systems put it anywhere except 'sphinx-build')

The basic approach is somewhat similar to how we deal with kerneldoc
and hxtool: we have a custom Sphinx extension which is passed a
filename which is the json file it should run the QAPI parser over and
generate documentation for. Unlike 'kerneldoc' but somewhat like
hxtool, I have chosed to generate documentation by generating a tree
of docutils nodes, rather than by generating rST source that is then
fed to the rST parser to generate docutils nodes.  Individual lumps of
doc comment go to the rST parser, but the structured parts we render
directly. This makes it easier to get the structure and heading level
nesting correct.

Rather than trying to exactly handle all the existing comments I have
opted (as Markus suggested) to tweak them where this seemed more
sensible than contorting the rST generator to deal with
weirdnesses. The principal changes are:
 * whitespace is now significant, and multiline definitions must have
   their second and subsequent lines indented to match the first line
 * general rST format markup is permitted, not just the small set of
   markup the old texinfo generator handled. For most things (notably
   bulleted and itemized lists) the old format is the same as rST was.
 * Specific things that might trip people up:
   - instead of *bold* and _italic_ rST has **bold** and *italic*
   - lists need a preceding and following blank line
   - a lone literal '*' will need to be backslash-escaped to
 avoid a rST syntax error
 * the old leading '|' for example (literal text) blocks is replaced
   by the standard rST '::' literal block.
 * headings and subheadings must now be in a freeform documentation
   comment of their own
 * we support arbitrary levels of sub- and sub-sub-heading, not just a
   main and sub-heading like the old texinfo generator
 * as a special case, @foo is retained and is equivalent to ``foo``
Moving on to the actual code changes:
 * we start by changing the existing parser code to be more careful
   with leading whitespace: instead of stripping it all, it strips
   only the amount required for indented multiline definitions, and
   complains if it finds an unexpected de-indent. The texinfo
   generator code is updated to do whitespace stripping itself, so
   there is no change to the generated texi source.
 * then we add the new qapidoc Sphinx extension, which is not yet used
   by anything. This is a 500 line script, all added in one patch. I
   can split it if people think that would help, but I'm not sure I
   see a good split point.
 * then we can convert the two generated reference documents, one at a
   time. This is mostly just updating makefile rules and the like.
 * after that we can do some minor tweaks to doc comments that would
   have confused the texinfo parser: changing our two instances of
   '|'-markup literal blocks to rST '::' literal blocks, and adding
   some headings to the GA reference so the rST interop manual ToC
   looks better.
 * finally, we can delete the old texinfo machinery and update