Re: [petsc-dev] User(s) manual sections field in manual pages?

2019-06-17 Thread Jed Brown via petsc-dev
Patrick Sanan  writes:

>> It ought, I suppose, be possible to write a plugin that adds links
>> automagically to all keywords in formatted source, but I don't know the
>> details of how these are written.
>>
> Sounds like Jed's suggesting that this could be done with a script similar
> to the one that exists now for the manual. How about
> 1. Use pandoc to convert (a section of) the dev manual to the new .rst
> format
> 2. Modify the existing script to add links from (temporary copies of) the
> new .rst to the existing man pages (still using the existing htmlmap file)
> 3. Generate an example of what the new html dev manual would like
> This would
> - probably reveal some gotchas about this approach
> - be a good step along the way to a useful first result, which is the
> manual and dev manual on the web

I think this would be a good start.


Re: [petsc-dev] User(s) manual sections field in manual pages?

2019-06-17 Thread Jed Brown via petsc-dev
Lawrence Mitchell  writes:

>> On 17 Jun 2019, at 19:50, Jed Brown via petsc-dev  
>> wrote:
>
> [...]
>> 
>> As for strategy, we have these primary forms of documentation:
>> 
>> * Users Manual
>> 
>>  A lot of value here is in cross-references between the manual and man
>>  pages.  PETSc has a special pass where it creates links from to all
>>  the man pages every time a keyword appears, including in listings.
>>  I'd rather not be verbose about it in rst source as in
>>  :c:func:`~.PetscScalar`.  
>
> Yes, this is a disadvantage of OOTB sphinx (unless we've misconfigured
> it). You have to explicitly mark up the links in the long-form
> documentation that you would like to link. I guess we could do it
> after the fact, but have not done so. As a result, we're rather
> inconsistent about it. This is "fine" if you're reading the whole
> manual with a Python terminal up and running to check docstrings, but
> hardly seamless.

The noisiness of the native RST source representation is also pretty
gross in my opinion.

>>  Also, the Firedrake manual (and I think
>>  Sphinx as a whole) doesn't hyperlink keywords appearing in source.
>>  So, for example, there is no link from CircleManifoldMesh to a man
>>  page.
>> 
>>  
>> https://www.firedrakeproject.org/extruded-meshes.html#generating-extruded-meshes-in-firedrake
>
> It ought, I suppose, be possible to write a plugin that adds links 
> automagically to all keywords in formatted source, but I don't know the 
> details of how these are written.

Right, we have to learn how to add such things.  If you know where to
look, it would be helpful.

>> * Manual pages
>> 
>>  I need to learn a bit more about the numpydoc and breathe
>>  implementations to be able to write a Sowing parser for it.
>
> Would this be changing the format for man page docs to numpydoc? I don't know 
> enough about the current setup one-way or another to know if 

I guess that could be considered, but I meant parsing Sowing to talk to
Sphinx in the same way that numpydoc parses its format for Sphinx.  I
haven't found a nice tutorial on this so I figured I would work it out
from the implementations of numpydoc/breathe.

>>> For instance, can we start by just generating stub .rst-formatted man
>>> pages, which include a link to the existing man pages?
>> 
>> I guess, but all the value is in cross-references.  Also, I don't care
>> to have an rst-formatted representation appear in a file.
>
> This is certainly a disadvantage of our approach: docstrings as formatted in 
> the python repl are ugly if they contain much linking (and/or maths). Having 
> a setup that minimises the amount of noise in the text format is probably 
> preferable.

I care about the source format too because that's what people need to
edit.


Re: [petsc-dev] User(s) manual sections field in manual pages?

2019-06-17 Thread Lawrence Mitchell via petsc-dev



> On 17 Jun 2019, at 19:50, Jed Brown via petsc-dev  
> wrote:

[...]
> 
> As for strategy, we have these primary forms of documentation:
> 
> * Users Manual
> 
>  A lot of value here is in cross-references between the manual and man
>  pages.  PETSc has a special pass where it creates links from to all
>  the man pages every time a keyword appears, including in listings.
>  I'd rather not be verbose about it in rst source as in
>  :c:func:`~.PetscScalar`.  

Yes, this is a disadvantage of OOTB sphinx (unless we've misconfigured it). You 
have to explicitly mark up the links in the long-form documentation that you 
would like to link. I guess we could do it after the fact, but have not done 
so. As a result, we're rather inconsistent about it. This is "fine" if you're 
reading the whole manual with a Python terminal up and running to check 
docstrings, but hardly seamless.

>  Also, the Firedrake manual (and I think
>  Sphinx as a whole) doesn't hyperlink keywords appearing in source.
>  So, for example, there is no link from CircleManifoldMesh to a man
>  page.
> 
>  
> https://www.firedrakeproject.org/extruded-meshes.html#generating-extruded-meshes-in-firedrake

It ought, I suppose, be possible to write a plugin that adds links 
automagically to all keywords in formatted source, but I don't know the details 
of how these are written.

> * Manual pages
> 
>  I need to learn a bit more about the numpydoc and breathe
>  implementations to be able to write a Sowing parser for it.

Would this be changing the format for man page docs to numpydoc? I don't know 
enough about the current setup one-way or another to know if 

>> For instance, can we start by just generating stub .rst-formatted man
>> pages, which include a link to the existing man pages?
> 
> I guess, but all the value is in cross-references.  Also, I don't care
> to have an rst-formatted representation appear in a file.

This is certainly a disadvantage of our approach: docstrings as formatted in 
the python repl are ugly if they contain much linking (and/or maths). Having a 
setup that minimises the amount of noise in the text format is probably 
preferable.

Lawrence



Re: [petsc-dev] User(s) manual sections field in manual pages?

2019-06-17 Thread Jed Brown via petsc-dev
Patrick Sanan  writes:

> What are the options for incrementalism with this general approach? Does it
> make much sense to say "let's make sure the manual is online first, and
> then later we can update the man pages and tutorials?".

Note: The above "incrementalism" is different from my previous comment
about "incremental builds", which refers to being able to quickly update
docs after making a change, similar to what we get when building the
library.


As for strategy, we have these primary forms of documentation:

* Users Manual

  A lot of value here is in cross-references between the manual and man
  pages.  PETSc has a special pass where it creates links from to all
  the man pages every time a keyword appears, including in listings.
  I'd rather not be verbose about it in rst source as in
  :c:func:`~.PetscScalar`.  Also, the Firedrake manual (and I think
  Sphinx as a whole) doesn't hyperlink keywords appearing in source.
  So, for example, there is no link from CircleManifoldMesh to a man
  page.

  
https://www.firedrakeproject.org/extruded-meshes.html#generating-extruded-meshes-in-firedrake

  PETSc's mapnameslatex.py script takes less than one second so this
  isn't a performance issue.

* Developers Manual

  This might be a good place to start because it is smaller and updated
  less frequently.  The same issues with auto cross-references apply.

* Manual pages

  I need to learn a bit more about the numpydoc and breathe
  implementations to be able to write a Sowing parser for it.

* Examples

  Easy in principle.  We strip CHKERRQ, hyperlink all the functions, and
  provide references from the man pages.

* HTML website-only documents (e.g., change logs, faq.html)

  Pandoc seems to do fine out of the box with converting these to
  rst/markdown (we can filter out the sidebar because it would be
  generated by Sphinx).  It's also the least interesting so I suggest
  not prioritizing it.

> For instance, can we start by just generating stub .rst-formatted man
> pages, which include a link to the existing man pages?

I guess, but all the value is in cross-references.  Also, I don't care
to have an rst-formatted representation appear in a file.