On 28 July 2010 20:14, Johan Hake <[email protected]> wrote: > On Sunday July 25 2010 13:25:25 Kristian Ølgaard wrote: >> On 25 July 2010 20:29, Johan Hake <[email protected]> wrote: >> > On Saturday July 24 2010 09:48:44 Kristian Ølgaard wrote: >> >> On 22 July 2010 20:10, Johan Hake <[email protected]> wrote: >> >> > On Monday July 19 2010 12:54:44 Kristian Oelgaard wrote: >> >> >> On 23 June 2010 15:35, Kristian Oelgaard <[email protected]> > wrote: >> >> >> > On 23 June 2010 10:55, Kristian Oelgaard <[email protected]> >> > >> > wrote: >> >> >> >> On 22 June 2010 19:03, Johan Hake <[email protected]> wrote: >> >> >> >>> On Tuesday June 22 2010 08:28:37 Kristian Oelgaard wrote: >> >> >> >>>> I've started writing the programmer's reference for FEniCS. >> >> >> >>>> One of the features that we decided on was that doc-strings for >> >> >> >>>> PyDOLFIN should be written and maintained as part of the >> >> >> >>>> documentation project and then added to the dolfin module on >> >> >> >>>> import. >> >> >> >>>> >> >> >> >>>> I thought about doing this in the following way: >> >> >> >>>> >> >> >> >>>> 1) Create a pseudo module 'dolfin-doc' which is a copy of the >> >> >> >>>> classes and functions in the 'real' dolfin module only it >> >> >> >>>> contains no code at all, just doc-strings. (This approach will >> >> >> >>>> also make it easy to create a script to check if all functions >> >> >> >>>> are documented or if any docs are obsolete). >> >> >> >>> >> >> >> >>> Sounds good. I first thought of a structure (other than a dummy >> >> >> >>> class) that just mimics the class hierarchy, but in some way that >> >> >> >>> is what you actually suggests and it is probably as easy as >> >> >> >>> anything else. >> >> >> >>> >> >> >> >>>> 2) Use the autodoc functionality of Sphinx to create parts of >> >> >> >>>> the documentation for functions and classes >> >> >> >>>> >> >> >> >>>> 3) Manually add additional information (in the reST file) and >> >> >> >>>> links to other parts of the documentation like demos etc. This >> >> >> >>>> will not be available using help() in the Python interpreter. >> >> >> >>>> >> >> >> >>>> 4) In the dolfin.__init__.py function import the 'dolfin-doc' >> >> >> >>>> module and copy the doc-strings from all classes and functions >> >> >> >>>> to the classes and functions in the real dolfin module as was >> >> >> >>>> suggested by Johan Hake. >> >> >> >>>> >> >> >> >>>> The problem with this approach is that assigning to __doc__ is >> >> >> >>>> not permitted for objects of 'type' type. >> >> >> >>>> >> >> >> >>> :( >> >> >> >>> >> >> >> >>> I did not anticipate this. Not sure why this is. I have got the >> >> >> >>> impression that numpy get around this. They use numpydoc to >> >> >> >>> dynamically add their documentation. It makes heavy use of >> >> >> >>> sphinx, but I couldn't figure how they get around that __doc__ >> >> >> >>> is read-only. >> >> >> >> >> >> >> >> To me it looks like numpydoc is a Sphinx extension that translates >> >> >> >> the Numpy docstrings into something that Sphinx can understand, >> >> >> >> not the other way around which is what we want. >> >> >> >> >> >> >> >> http://projects.scipy.org/numpy/browser/trunk/doc/sphinxext/README >> >> >> >> .tx t >> >> >> >> >> >> >> >> So I think our best bet is to proceed with your suggestions below. >> >> >> >> >> >> >> >> Kristian >> >> >> >> >> >> >> >>> While it might be cool to look into what NumPy have done, (they >> >> >> >>> also define a pseudo classes, which they populate with >> >> >> >>> docstrings, (look into phantom_import.py), and they also define >> >> >> >>> some nice format for the reST used in the docstrings), I suggest >> >> >> >>> two things we can do: >> >> >> >>> >> >> >> >>> 1) SWIG can generate docstrings. We do that allready using parsed >> >> >> >>> doxygen documentation. All of this is gathered in docstrings.i. I >> >> >> >>> suggest generating such a file from our documentation. We need to >> >> >> >>> turn of %feature("autodoc","1") in dolfin.i to get rid of the >> >> >> >>> long and sometimes faulty generated signatures. >> >> >> > >> >> >> > I turns out that it's only the __doc__ of the class I can't assign >> >> >> > to, not the __doc__ of member functions (and regular functions). A >> >> >> > simpler solution (at least for me) is to parse the cpp.py module >> >> >> > once generated and substitute all docstrings of classes with the >> >> >> > docstrings from the dolfindoc module rather than creating the >> >> >> > docstrings.i file. >> >> >> > >> >> >> > Then for the classes that we manually add we use the method you >> >> >> > described below, but only for class.__doc__ . >> >> >> > >> >> >> > class Foo(object): >> >> >> > __doc__ = dolfindoc.Foo.__doc__ >> >> >> > def bar(self): >> >> >> > "this doc string will be substituted with the >> >> >> > dolfindoc.Foo.__dict__["bar"].__doc__." >> >> >> > pass >> >> >> >> >> >> Unfortunately it turned out I was too quick to make this conclusion. >> >> >> For instance, >> >> >> for the Swig generated class Mesh in the module cpp I can't assign to >> >> >> >> >> >> Mesh.__doc__ >> >> >> >> >> >> because it is a 'type' object as pointed our earlier, I can assign >> >> >> to: >> >> >> >> >> >> Mesh.__dict__["__init__"].__doc__ >> >> >> >> >> >> but not >> >> >> >> >> >> Mesh.__dict__["size"].__doc__ >> >> >> >> >> >> along with practically all other member functions. >> >> >> >> >> >> The reason is that Swig, after the class definitions does this: >> >> >> >> >> >> Mesh.size = new_instancemethod(_cpp.Mesh_size,None,Mesh) >> >> >> >> >> >> to which no assignment is possible. >> >> >> 'AttributeError: attribute '__doc__' of 'instancemethod' objects is >> >> >> not writable' >> >> >> The code in Mesh.size is already >> >> >> >> >> >> def size(self, *args): >> >> >> return _cpp.Mesh_size(self, *args) >> >> >> >> >> >> so why Swig overrides the method I don't know. If it didn't I could >> >> >> assign to the docstring. >> >> >> >> >> >> Recall that my initial plan was to simply parse the cpp.py file and >> >> >> substitute the docstrings of all classes with what the pseudo module >> >> >> dolfindoc would define. However, that doesn't seem so practical >> >> >> anymore since I would need to also comment out all lines where >> >> >> instancemethods are being assigned to class members such that on >> >> >> import (in dolfin/__init__.py) I can loop classes and assign to >> >> >> function docstrings. This is of course possible but seems a bit >> >> >> awkward and I don't know what implications the commenting out of >> >> >> instancemethod assignment will have. >> >> >> As I see it, two solutions remain. >> >> >> >> >> >> 1) Take the pseudo module dolfindoc from the FEniCS documentation and >> >> >> create a docstring.i generator. The script >> >> >> dolfin/dolfin/swig/generator.py should then try to import the >> >> >> dolfindoc module, if successful use that to create docstring.i, >> >> >> otherwise use the Doxy2SWIG generator. >> >> > >> >> > I thought that you landed on this alternative. The point is to >> >> > generate a dosctrings.i file from the FEniCS documentation. Then for >> >> > the extended Python layer we can use a generated Python module to >> >> > assign the docstrings: >> >> > >> >> > class VariationalProblem(...): >> >> > ... >> >> > __doc__ generated_docstring_module.VariationalProblem.__doc__ >> >> >> >> This approach is getting more and more complicated; it turns out we >> >> also need to generate all the >> >> _post.i files to get the docstrings correct. >> > >> > Do yo meen the extended python functions that resides in these files? >> >> Yes, they will need a docstring too. But since Swig doesn't mess with >> these particular (member-) functions, it is actually possible to >> assign to __doc__ dynamically (on import) so maybe that would be an >> easier option. > > How would you do that? I tried > > %extend dolfin::GenericVector > { > %pythoncode > %{ > def data(self): > generated_docstring_module.GenericVector.data.__doc__ > return self._data() > %} > } > > But it was only possible to have a "real" string, not a variable that was a > str. Not sure why...
Me neither, what I thought about doing was to generate the contents of the *_post.i files from some source and then insert the docstring as a literal string in the *_post.i file. But I decided that was too much work. So what I will end up doing is to assign to the __doc__ members of these functions in the dolfin/__init__.py (on import). This is possible because Swig doesn't mess around with these functions but just adds them to the module. If the *_post.i contains other than simple functions/member functions we'll have to generate the *_post.i files from some source. Kristian >> >> At this point it seems a >> >> lot easier to me to simply extend the Python layer with everything we >> >> want to have detailed documentation for and distribute the >> >> dolfindocstrings module with DOLFIN so we can do: >> >> >> >> class VariationalProblem(...): >> >> ... >> >> __doc__ generated_docstring_module.VariationalProblem.__doc__ >> >> >> >> this way we handle the docstrings in a uniform way, else we need to >> >> generate docstrings.i for some functions, *_post.i files for other >> >> functions and STILL do the assign to __doc__ trick. >> >> >> >> For functions that we don't extend in the Python layer we still have >> >> the oneliner docs from the header files extended with some random >> >> output from Swig. >> > >> > How do we handle the documentation of the pure cpp classes, for example >> > cpp.Mesh? >> >> For all functions/classes we put a docstring in the docstringsmodule >> and distribute this module with DOLFIN. > > Ok > >> The current approach that we're pursuing is: >> For the classes in cpp, we generate the docstrings.i, >> For add on functions in *_post.i we need to either generate *_post.i >> or assign to __doc__ of those particular functions on import. >> For classes functions in the Python layer we just assign to __doc__ in >> the definition of classes/functions. > > Ok > >> or as I suggested: >> >> Extend the Python layer with whatever classes/functions that we want >> to have documented in detail and assign to __doc__ in the definitions. > > Ok > > Johan > >> Kristian >> >> > Johan >> > >> >> Kristian >> >> >> >> >> One question w.r.t this approach, when is docstrings.i being >> >> >> generated? I see it is distributed with DOLFIN but shouldn't it be >> >> >> generated everytime DOLFIN is rebuild with enablePython, or does >> >> >> enableDocs have to be switched on as well? >> >> > >> >> > It is generated by running dolfin/swig/generate.py. It takes quite a >> >> > long time so I think it is good to pre-generate this in the >> >> > distribution. >> >> > >> >> >> 2) Extend our Python layer with all functions and classes that we >> >> >> want to use the dolfindoc docstrings for. >> >> >> These should just be empty and redirect calls to classes and >> >> >> functions of the cpp. In dolfin/mesh/mesh.py: >> >> >> >> >> >> class Mesh(cpp.Mesh): >> >> >> def size(self): >> >> >> try: >> >> >> import dolfindoc.mesh.mesh >> >> >> __doc__ = dolfindoc.mesh.mesh.Mesh.__doc__ >> >> >> except: >> >> >> __doc__ = cpp.Mesh.__doc__ >> >> >> return cpp.Mesh.size() >> >> >> >> >> >> An additional benefit of this approach is that the module structure >> >> >> can be identical to what we have in the _real_ DOLFIN, not as it is >> >> >> now where everything is dumped in the dolfin.cpp module. >> >> >> I don't know how much overhead this will create, alternatively we can >> >> >> skip the try/except clause and simply have the documentation as a >> >> >> dependency, or not add the docstrings for memberfunctions and add >> >> >> them later on import as was the original idea. >> >> > >> >> > This sounds cumbersome and in the example above will the try: except >> >> > clause be called everytime size is called. >> >> > >> >> > Johan >> >> > >> >> >> Suggestions and comments are more than welcome! >> >> >> >> >> >> Kristian >> >> >> >> >> >> > then in dolfin/__init__.py we load the classes as we do now from >> >> >> > the dolfin module, and then iterate over all functions and member >> >> >> > functions and substitute docstrings from the dolfindoc module. >> >> >> > >> >> >> > Kristian >> >> >> > >> >> >> >>> 2) The added python classes and methods can be documented using >> >> >> >>> your suggested approach, but instead of adding the docstring >> >> >> >>> after class creation, do it during class (method or function) >> >> >> >>> creation, a la: >> >> >> >>> >> >> >> >>> class Foo(object): >> >> >> >>> __doc__ = docstrings.Foo.__doc__ >> >> >> >>> ... >> >> >> >>> >> >> >> >>> where docstrings is the generated module containing the >> >> >> >>> docstrings. >> >> >> >>> >> >> >> >>> Johan >> >> >> >>> >> >> >> >>>> In other words we can't assign to the __doc__ of >> >> >> >>>> >> >> >> >>>> class Foo(object): >> >> >> >>>> "Foo doc" >> >> >> >>>> pass >> >> >> >>>> >> >> >> >>>> Which is a new-style class and found in UFL and the SWIG code in >> >> >> >>>> DOLFIN. >> >> >> >>>> >> >> >> >>>> It works fine for >> >> >> >>>> >> >> >> >>>> def some_function(v): >> >> >> >>>> "function doc" >> >> >> >>>> return 2*v >> >> >> >>>> >> >> >> >>>> and >> >> >> >>>> >> >> >> >>>> class Bar: >> >> >> >>>> "Bar doc" >> >> >> >>>> pass >> >> >> >>>> >> >> >> >>>> which is the old-style class often found in FFC. >> >> >> >>>> >> >> >> >>>> Does anyone have a solution or comments to the above approach, >> >> >> >>>> or maybe we can do it in a completely different way. >> >> >> >>>> >> >> >> >>>> I read about some workaround for the 'assign to __doc__' >> >> >> >>>> problem, but it doesn't seem that nice and it might be a >> >> >> >>>> problem to incorporate in the SWIG generated code? >> >> >> >>>> >> >> >> >>>> http://stackoverflow.com/questions/71817/using-the-docstring-fro >> >> >> >>>> m-o ne- metho d-to-automatically-overwrite-that-of-another-me >> >> >> >>>> >> >> >> >>>> >> >> >> >>>> Kristian >> >> >> >>>> >> >> >> >>>> _______________________________________________ >> >> >> >>>> Mailing list: https://launchpad.net/~fenics >> >> >> >>>> Post to : [email protected] >> >> >> >>>> Unsubscribe : https://launchpad.net/~fenics >> >> >> >>>> More help : https://help.launchpad.net/ListHelp >> >> >> >> >> >> _______________________________________________ >> >> >> Mailing list: https://launchpad.net/~fenics >> >> >> Post to : [email protected] >> >> >> Unsubscribe : https://launchpad.net/~fenics >> >> >> More help : https://help.launchpad.net/ListHelp >> >> >> >> _______________________________________________ >> >> Mailing list: https://launchpad.net/~fenics >> >> Post to : [email protected] >> >> Unsubscribe : https://launchpad.net/~fenics >> >> More help : https://help.launchpad.net/ListHelp >> >> _______________________________________________ >> Mailing list: https://launchpad.net/~fenics >> Post to : [email protected] >> Unsubscribe : https://launchpad.net/~fenics >> More help : https://help.launchpad.net/ListHelp > _______________________________________________ Mailing list: https://launchpad.net/~fenics Post to : [email protected] Unsubscribe : https://launchpad.net/~fenics More help : https://help.launchpad.net/ListHelp
