Re: [sage-devel] Re: Place for "internal" documentation of a class

[Nathann Cohen]

Hello !

> Despite from that I hope you understand what I have in mind with developer
> documentation.

I totally agree with you on this respect. The documentation that Jori
wanted to write totally fits with your example: there is a class that
we did not write ourselves and that we do not understand totally (even
after working on it), and it seemed adequate to "take notes" about
what we know of the implementation. For ourselves in the future, and
for others.

Perhaps that meant as Volker said that "the code should have been
written differently from the start", and in order to rewrite it one
needs first to understand what it does.

Nathann

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Re: Place for "internal" documentation of a class

[Jakob Kroeker]

> Again it depends on the case. Not every algorithm needs to be documented, 
> some things are necessarily assumed to be known to anybody who wants to 
> make relevant changes. I don't think Sage explains the algorithm for 
> solving CRT anywhere...
>

Agree. I did not have in mind well-known algorithms.

See for example the compete documentation issue overview of Singulars 
'factory', 
https://github.com/Singular/Sources/issues/637

More than 60 percent of the source code is not documented at all.


How is the class hierarchy not best described by the code? If you can't be 
> bothered to read the code you can also use Python's inspection abilities.
>

You score a point  here (use Python's inspection abilities)

Despite from that I hope you understand what I have in mind with developer 
documentation.
I really hope that I'm wrong, but I predict that in case that it is missing 
or not adequate, 
in ten or twenty years sage may get serious problems with rotten and 
unmaintainable source code,
because at that point many current developers will have left the sage 
project and fresh developers may fail to understand the old sources.

And because in history predictors or carriers of bad news were burned, I 
will shut up now on that topic.


Jakob


Am Montag, 5. Januar 2015 22:41:32 UTC+1 schrieb Volker Braun:
>
> On Monday, January 5, 2015 5:45:21 PM UTC+1, Jakob Kroeker wrote:
>
>> class diagrams,  state transition diagrams, internal code 
>> interdependencies/implemented design patterns, call graphs
>
>
> How is the class hierarchy not best described by the code? If you can't be 
> bothered to read the code you can also use Python's inspection abilities.
>
> Adding a link/reference to the underlying algorithm is definitely 
>>> desirable.
>>
>> Despite from the fact that (I guess) this basic rule is obvious, in 
>> reality its often violated, at least in Singular code.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Re: Place for "internal" documentation of a class

[Volker Braun]

On Monday, January 5, 2015 5:45:21 PM UTC+1, Jakob Kroeker wrote:

> class diagrams,  state transition diagrams, internal code 
> interdependencies/implemented design patterns, call graphs


How is the class hierarchy not best described by the code? If you can't be 
bothered to read the code you can also use Python's inspection abilities.

Adding a link/reference to the underlying algorithm is definitely desirable.
>
> Despite from the fact that (I guess) this basic rule is obvious, in 
> reality its often violated, at least in Singular code.
>

Again it depends on the case. Not every algorithm needs to be documented, 
some things are necessarily assumed to be known to anybody who wants to 
make relevant changes. I don't think Sage explains the algorithm for 
solving CRT anywhere...

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Re: Place for "internal" documentation of a class

[Jakob Kroeker]

>
> I don't understand what you mean by "design". If you code isn't readable 
> then that can't be fixed by documentation, only by writing clear code.
>


The program architecture, the grand picture (if not obvious): class 
diagrams,  state transition diagrams, internal code 
interdependencies/implemented design patterns, call graphs;
 everything a developer cannot extract by a code inspection without major 
effort in comparison to reading the developer documentation.

AFAIK, Nathann Cohen did hit this issue when working on the poset 
implementation.


Adding a link/reference to the underlying algorithm is definitely desirable.
>

Despite from the fact that (I guess) this basic rule is obvious, in reality 
its often violated, at least in Singular code.
For me that is a reason to extend the review checklist by this point. What 
do you think?

Of course we cannot put everything on the checklist. In case it turns out 
mandatory to put on the checklist that the people should 
breath, I probably would give up and move away ;-)


Am Montag, 5. Januar 2015 16:00:54 UTC+1 schrieb Volker Braun:
>
> On Monday, January 5, 2015 3:15:07 PM UTC+1, Jakob Kroeker wrote:
>>
>> Does the 'why' implicate documentation for an abstract description of the 
>>> implementation design
>>
>>
> I don't understand what you mean by "design". If you code isn't readable 
> then that can't be fixed by documentation, only by writing clear code.
>  
>
>> (reference to) underlying algorithms?
>>>
>>
> Adding a link/reference to the underlying algorithm is definitely 
> desirable.
>
>  

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Re: Place for "internal" documentation of a class

[Volker Braun]

On Monday, January 5, 2015 3:15:07 PM UTC+1, Jakob Kroeker wrote:
>
> Does the 'why' implicate documentation for an abstract description of the 
>> implementation design
>
>
I don't understand what you mean by "design". If you code isn't readable 
then that can't be fixed by documentation, only by writing clear code.
 

> (reference to) underlying algorithms?
>>
>
Adding a link/reference to the underlying algorithm is definitely desirable.


-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Re: Place for "internal" documentation of a class

[Jakob Kroeker]

>
> * Document the why, not the how
>

Does the 'why' implicate documentation for an abstract description of the 
implementation design (if not obvious) and the (reference to) underlying 
algorithms?

I just looked at the sage review checklist and did not find and advice to 
check for appropriate developer documentation (e.g. class design or 
reference to implemented algorithm)

If that is missing it may and will cost for a new developer/maintainer  
unreasonably  big effort  to understand a design or  reconstruct an 
algorithm from the source,
what is often mandatory to maintain or extend existing implementation.

Jakob

Am Montag, 5. Januar 2015 11:36:23 UTC+1 schrieb Volker Braun:
>
> The __init__ docstring is used as the class docstring if there is no 
> separate class docstring, this is what I would recommend if you want it 
> seen. Alternatively you can force sphinx to show it with a .. automethod: 
> instruction.
>
> As for what is appropriate to document, the usual adages apply:
>
> * Document the why, not the how
> * Place the documentation as close to the relevant code as possible.
>
> Anything that affects only a single method has no place outside of that 
> method. If it can be written as a code comment, then that is preferrable. 
> Only very high-level documentation should ever be placed in the class 
> scope, e.g. "Performance-critical: This class implements geobuckets for 
> non-communative polynomials". 
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Re: Place for "internal" documentation of a class

[kcrisman]

> The __init__ docstring is used as the class docstring if there is no 
> separate class docstring, this is what I would recommend if you want it 
> seen. Alternatively you can force sphinx to show it with a .. automethod: 
> instruction.
>
>
Perhaps there should be a mention of this in the developer guide (if it 
isn't already). 

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Re: Place for "internal" documentation of a class

[Volker Braun]

The __init__ docstring is used as the class docstring if there is no 
separate class docstring, this is what I would recommend if you want it 
seen. Alternatively you can force sphinx to show it with a .. automethod: 
instruction.

As for what is appropriate to document, the usual adages apply:

* Document the why, not the how
* Place the documentation as close to the relevant code as possible.

Anything that affects only a single method has no place outside of that 
method. If it can be written as a code comment, then that is preferrable. 
Only very high-level documentation should ever be placed in the class 
scope, e.g. "Performance-critical: This class implements geobuckets for 
non-communative polynomials". 


-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Re: Place for "internal" documentation of a class

[Travis Scrimshaw]

On Sunday, January 4, 2015 2:20:56 PM UTC-8, Andrew wrote:
>
> I agree with Nathan that it would be good to have "internal" documentation 
> more accessible (and to write more of it). I don't see any issues with 
> "scaring" ordinary users as this material should be in the developers 
> manual rather than the reference manual.
>

   I don't think it will scare an ordinary user, but obfuscate the 
documentation the typical user will want to look at.

>
> At the moment we are writing a lot of documentation that sphinx simply 
> ignores. As Travis says, we can read this documentation in the source files 
> but as we are all spending a lot of time writing this documentation why not 
> take more advantage of it? Is there any reason not to have an "expanded" 
> developers guide that includes all of the ReST documentation available in 
> the source? Alternatively, it shouldn't be hard to have sphinx add the 
> documentation for the __init__ method of a class as a special "Technical 
> specification" section of the class documentation.
>

   Although perhaps the issue is that we don't intrinsically have separate 
developers and users documentation. I would not be opposed to having a link 
or some way of also including all hidden methods in the reference manual.

>
>
> On Monday, 5 January 2015 07:03:04 UTC+11, Jori Mantysalo wrote:
>>
>> Is there "internal" documentation in Sage? If so, where it is?   
>
>
>> I mean that if a developer wants to know how some class is made (and 
>> maybe 
>> WHY, if there are nontrivial choises for data structure), should he or 
>> she 
>> always found it with ".__init__?"? Or by looking at start of 
>> source 
>> code file, or from the end of file? 
>>
>>I think a comment in the code right above the class definition is a 
good place to put this if you feel this deserves a comment, in the sense 
that this is not clear from the code/superclass(es). However if the choice 
of data structure affects potential performance, and hence the utility, and 
there are other classes with the same top-level capabilities which can be 
contrasted, then I feel there should be some information in the class doc.

   I'm starting to feel like I'm beating a (very) dead horse. Maybe I'm 
being paranoid or "more closed-minded than usual", but too much information 
causes problems too. We have different tools (docstrings, comments, and the 
code itself) for different purposes and I think we should try to keep 
things separated as much as possible.

Best,
Travis

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Re: Place for "internal" documentation of a class

[Andrew]

I agree with Nathan that it would be good to have "internal" documentation 
more accessible (and to write more of it). I don't see any issues with 
"scaring" ordinary users as this material should be in the developers 
manual rather than the reference manual.

At the moment we are writing a lot of documentation that sphinx simply 
ignores. As Travis says, we can read this documentation in the source files 
but as we are all spending a lot of time writing this documentation why not 
take more advantage of it? Is there any reason not to have an "expanded" 
developers guide that includes all of the ReST documentation available in 
the source? Alternatively, it shouldn't be hard to have sphinx add the 
documentation for the __init__ method of a class as a special "Technical 
specification" section of the class documentation.

Andrew


On Monday, 5 January 2015 07:03:04 UTC+11, Jori Mantysalo wrote:
>
> On Sat, 27 Dec 2014, Travis Scrimshaw wrote: 
>
> > If you're developing, you're...not...looking at the code? That's how I'm 
> > interpreting your comments. 
>
> Is there "internal" documentation in Sage? If so, where it is? 
>
> I mean that if a developer wants to know how some class is made (and maybe 
> WHY, if there are nontrivial choises for data structure), should he or she 
> always found it with ".__init__?"? Or by looking at start of source 
> code file, or from the end of file? 
>
> -- 
> Jori Mäntysalo 
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Re: Place for "internal" documentation of a class

[Jori Mantysalo]

On Sat, 27 Dec 2014, Travis Scrimshaw wrote:


If you're developing, you're...not...looking at the code? That's how I'm
interpreting your comments.


Is there "internal" documentation in Sage? If so, where it is?

I mean that if a developer wants to know how some class is made (and maybe 
WHY, if there are nontrivial choises for data structure), should he or she 
always found it with ".__init__?"? Or by looking at start of source 
code file, or from the end of file?


--
Jori Mäntysalo

Re: [sage-devel] Re: Place for "internal" documentation of a class

[Nathann Cohen]

> Is it possible to make sphinx generate html code that is "hidden" with a
> toggle button when it encounter a "Technical details::" section? That could
> be a good compromise solution.

Funny that you ask. This is what I wrote to Sphinx-devel 10 days ago:
https://groups.google.com/d/topic/sphinx-users/iu23DxbJmc4/discussion

Nathann

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Re: Place for "internal" documentation of a class

[mmarco]

Is it possible to make sphinx generate html code that is "hidden" with a 
toggle button when it encounter a "Technical details::" section? That could 
be a good compromise solution.

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Re: Place for "internal" documentation of a class

[Jori Mantysalo]

There is still one possibility: Having a file for *only* internal 
documentation or technical details. Then 
http://www.sagemath.org/doc/reference/combinat/posets.html would contain 
something like "Implementation details" as a list item.


Also some user might benefit from technical details. Some time ago there 
was no is_connected() for posets. However, knowing that poset "really" is 
a digraph the user might easily use a is_connected() from graphs. Maybe?


--
Jori Mäntysalo

Re: [sage-devel] Re: Place for "internal" documentation of a class

[Nathann Cohen]

> If you're developing, you're...not...looking at the code? That's how I'm
> interpreting your comments. I don't see how it is possible to develop
> without looking at the code.
>
> To reiterate, public documentation should be there for the user, internal
> documentation (i.e. private methods) and comments are there for the
> developers.

Travis, pardon me for saying so but recently I got the impression that
your comments were a bit more close-minded than usual.

What I said above, and the reason why Jori initiated this thread, is
that documentation in the code is less readable than documentation in
its HTML form. This is why our documentation is also compiled in html:
because we can make drawings, use "itemize" environments, have text in
bold, etc...

By adding technical documentation at module level (under a section
like "Technical details" or "Implementation"), we do not scare users
away by providing useless information as we *claim* explicitly that
this is only technical details.

On the other hand, the developers (who are also users), may be
interested to look at that. Even a user who just wonder "how it is
implemented" may be interested in that.

For "advertising purposes", it also lessens the barrier between "user"
and "developers" as we try to encourage everybody to dig in and see
how it works -- and become a contributor later.

We need it to be more explicit. I have been working on Posets for a
while, and Jori has been working on Posets for a while, and we never
noticed this doc. Even for us, who are developers, documentation of
the __init__ method was not sufficienty visible, and so we began to
write a documentation that was... already there !

I would also be thankful if you eventually agreed that some people,
like Jori and I, could expect to see this documentation advertised
better, like at module level. As a result, and even though it may not
be what you would do yourself, you could acknowledge that it cannot
really hurt anybody in the end, and that it would solve the problem
that Jori and I, and possibly others, met in the past or will meet in
the future.

Regards,

Nathann

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Re: Place for "internal" documentation of a class

[Travis Scrimshaw]

If you're developing, you're...not...looking at the code? That's how I'm 
interpreting your comments. I don't see how it is possible to develop 
without looking at the code.

To reiterate, public documentation should be there for the user, internal 
documentation (i.e. private methods) and comments are there for the 
developers.

Travis


On Saturday, December 27, 2014 7:08:14 PM UTC-8, Nathann Cohen wrote:
>
> > That is few and far between, and putting it at the module or class 
> levels 
> > are quite intrusive. 
>
> In a section named "technical details", you would find this intrusive ? 
>
> > Having the background technical information 
> > (especially that about design patterns used or easily deciphered from 
> the 
> > code) adds useless clutter that benefits nobody. 
>
> We need a way to convey that information to developers, and in this 
> specific case it happened that the developers did not see it. We need 
> to make it more visible than by letting it stay in the code. 
>
> Nathann 
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Re: Place for "internal" documentation of a class

[Nathann Cohen]

> That is few and far between, and putting it at the module or class levels
> are quite intrusive.

In a section named "technical details", you would find this intrusive ?

> Having the background technical information
> (especially that about design patterns used or easily deciphered from the
> code) adds useless clutter that benefits nobody.

We need a way to convey that information to developers, and in this
specific case it happened that the developers did not see it. We need
to make it more visible than by letting it stay in the code.

Nathann

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Re: Place for "internal" documentation of a class

[Travis Scrimshaw]

>
> Well, our users can become developpers too. Plus we can show this in a 
> non-intrusive way. It is a bit hard to make all the doc "clean" when 
> it is just code. 
>

That is few and far between, and putting it at the module or class levels 
are quite intrusive.

>
> Look at this: the doc is meant to developers only, and users will not 
> read it anyway. Everybody wins. 
>
>
> http://www.sagemath.org/doc/reference/graphs/sage/graphs/base/static_sparse_graph.html
>  
> 
>  
>
>
> There it is describing the *use* of the class, and as such, it is not for 
developers as it is describing different backends (although I think anyone 
reading beyond the first two parts will be looking at the functions 
specified or the code). Having the background technical information 
(especially that about design patterns used or easily deciphered from the 
code) adds useless clutter that benefits nobody.
 
Best,
Travis

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Re: Place for "internal" documentation of a class

[Nathann Cohen]

Yo !

>-1 as it is internal workings and would add useless information to the
> average user. If you need to know those details, then you should be looking
> at the file/code. IMO the public documentation should be about the
> mathematical structure and how to use the class in Sage.

Well, our users can become developpers too. Plus we can show this in a
non-intrusive way. It is a bit hard to make all the doc "clean" when
it is just code.

Look at this: the doc is meant to developers only, and users will not
read it anyway. Everybody wins.

http://www.sagemath.org/doc/reference/graphs/sage/graphs/base/static_sparse_graph.html

Nathann

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.