On 7 September 2010 19:23, Anders Logg <[email protected]> wrote: > On Tue, Sep 07, 2010 at 10:19:15AM -0700, Johan Hake wrote: >> On Tuesday September 7 2010 10:08:34 Kristian Ølgaard wrote: >> > On 7 September 2010 18:51, Johan Hake <[email protected]> wrote: >> > > On Tuesday September 7 2010 09:24:40 Kristian Ølgaard wrote: >> > >> On 7 September 2010 17:59, Johan Hake <[email protected]> wrote: >> > >> > [snip] >> > >> > >> > >> >> > But how do we extract the different arguments? I suppose this is >> > >> >> > collected by Doxygen, and we just need to parse these and output >> > >> >> > them in a correct way? >> > >> >> >> > >> >> I don't think we need to parse the arguments and output them. We just >> > >> >> get the function name and if we have more than one set of arguments >> > >> >> i.e., a different signature we know that we have an overloaded method >> > >> >> and how to handle it. >> > >> > >> > >> > And I guess the almighty generate_cpp_documentation.py script are able >> > >> > to extract the argument information? >> > >> >> > >> No, but we should be able to figure this out from the signature (split >> > >> ',' in '()'). >> > > >> > > Ok! Anders mentioned this too. >> > > >> > >> >> The arguments should be described in the *Arguments* section of the >> > >> >> individual docstring with links to classes formatted like >> > >> >> _MeshEntity_, which we will substitute with :py:class:`MeshEntity` or >> > >> >> >> > >> >> :cpp:class:`MeshEntity` depending on which interface we document. >> > >> > >> > >> > Ok, but we only want that once for each method in python, even if it >> > >> > is overloaded? >> > >> >> > >> No, I think we need to document the argument list for every overloaded >> > >> version like it is done in docstrings.dolfin.cpp.Mesh. >> > > >> > > Agree, I think I misunderstood you. >> > > >> > >> >> Although I just realized that standard C++ stuff like double* which >> > >> >> end up as numpy.array etc. should probably be handled. >> > >> > >> > >> > Yes this part I am a little worried about... But maybe a god >> > >> > handwritten lookup table will do the trick? At least for 99% of the >> > >> > cases ;) >> > >> >> > >> I share your concern, but if, as you suggest, we'll be able to get God >> > >> to hand write our documentation I think we should be OK. :) >> > > >> > > Good to have God on our side! >> > > >> > >> (a lookup table would be my best bet at the moment) >> > > >> > > Ok! >> > > >> > >> >> On a related note: >> > >> >> int some_func() >> > >> >> and >> > >> >> const int some_func() const >> > >> >> are different in C++, but in Python we don't have const right? >> > >> >> This will simplify the documentation a lot. >> > >> > >> > >> > Yes, we tend to %ignore all const versions of different methods. >> > >> > >> > >> > [snap] >> > >> > >> > >> >> >> > * Extended methods needs to be handled in one of three ways: >> > >> >> >> > 1) Write the docstring directly into the foo_post.i file >> > >> >> >> > >> >> I like this option, if this is where we have the code for a function, >> > >> >> then this is where the docstring should be as it increases the >> > >> >> probability of the docstring being up to date. >> > >> > >> > >> > Ok, lets settle on this one. We also need to make sure that all >> > >> > %extended methods in the C++ layer gets a proper docstring. However I >> > >> > am not really sure how this can be done :P >> > >> >> > >> I'm not sure I follow this, which %extended methods do you mean? >> > > >> > > There are two ways to extend a class. >> > > >> > > 1) The C++ layer >> > > 2) The Python layer >> > > >> > > often we use 1) to create a protected helper method which is called using >> > > an extended method in the Python layer, 2). The latter can be properly >> > > documented directly. >> > > >> > > But some cases excists where we just extend the C++ layer, see for >> > > example the IMPLEMENT_VARIABLE_INTERFACE macro in shared_ptr_classes.i. >> > > These methods gets no docstrings and I am not sure it is possible to add >> > > them later. >> > >> > OK, docstrings for 2) should go in the code as we agreed, and I guess >> > 1) will fall under the 1% category which we may/may not be able to >> > handle in a clever way later. >> >> Ok. >> >> > >> > [snup] >> > >> > >> > >> >> > Why do we need to assign to these methods? They already get their >> > >> >> > docstrings from the docstrings.i file. However if we want to get >> > >> >> > rid of the new_instancemethod assignment above, we can just remove >> > >> >> > the >> > >> >> >> > >> >> Some history. >> > >> >> Initially, we wanted to have all docstrings separated from the DOLFIN >> > >> >> code and collected in the fenics-doc module. The easiest way to get >> > >> >> the >>> help(dolfin) docstring correct is to assign to __doc__ >> > >> >> dynamically. >> > >> >> If we could do this we wouldn't even need the docstrings.i file and >> > >> >> things would be simple. >> > >> >> However, we discovered that this was not possible, and because of >> > >> >> that we still need to generate the docstrings.i file. >> > >> >> Then, still assuming we wanted to separate docs from code and keeping >> > >> >> docstrings in fenics-doc, I thought it would be easier to generate >> > >> >> the docstrings.i file from the handwritten docstrings module in >> > >> >> fenics-doc. >> > >> >> Some methods don't get their docstrings from the docstrings.i file >> > >> >> though, so we still need to assign to __doc__ which is the easiest >> > >> >> thing to do. >> > >> >> Just recently we decided to extract the docstrings from the C++ >> > >> >> implementation thus moving the docs back into DOLFIN. This makes the >> > >> >> docstrings module almost superfluous with the only practical usage is >> > >> >> to have documentation for the extended methods defined in the _post.i >> > >> >> files but if we put the docstrings directly in the _post.i files we >> > >> >> no longer need it. >> > >> > >> > >> > Ok, then I do not see any reason for a separate docstring module, >> > >> > makes life a lite bit easier... >> > >> >> > >> Agree. >> > > >> > > Ok. >> > > >> > >> > [snep] >> > >> > >> > >> >> > I am confused. Do you suggest that we just document the extended >> > >> >> > Python layer directly in the python module as it is today? Why >> > >> >> > should we then dumpt the docstrings in a separate docstring >> > >> >> > module? So autodoc can have something to shew on? Couldn't autodoc >> > >> >> > just shew on the dolfin module directly? >> > >> >> >> > >> >> I'm confused too. :) I guess my head has not been properly reset >> > >> >> between the changes in documentation strategies. >> > >> >> The Sphinx autodoc can only handle one dolfin module, so we need to >> > >> >> either import the 'real' one or the docstrings dolfin module. >> > >> >> If we can completely remove the need for the docstrings module, then >> > >> >> we should of course include the 'real' one. >> > >> > >> > >> > Ok! >> > >> > >> > >> >> >> Then programmer's writing the Python >> > >> >> >> layer just need to document while they're coding, where they are >> > >> >> >> coding just like they do (or should anyways) for the C++ part. >> > >> >> > >> > >> >> > Still confused why we need a certain docstring module. >> > >> >> >> > >> >> Maybe we don't. >> > >> >> >> > >> >> >> > 2) for the extended Python layer in the cpp.py >> > >> >> >> > >> > >> >> >> > For the rest, and this will be the main part, we rely on parsed >> > >> >> >> > docstrings from the headers. >> > >> >> >> > >> > >> >> >> > The python programmers reference will then be generated based on >> > >> >> >> > the actual dolfin module using sphinx and autodoc. >> > >> >> >> >> > >> >> >> We could/should probably use either the dolfin module or the >> > >> >> >> generated docstring module to generate the relevant reST files. >> > >> >> >> Although we might need to run some cross-checks with the Doxygen >> > >> >> >> xml to get the correct file names where the classes are defined >> > >> >> >> in DOLFIN such that we retain the original DOLFIN source tree >> > >> >> >> structure. Otherwise all our documentation will end up in cpp.rst >> > >> >> >> which I would hate to navigate through as a user. >> > >> >> > >> > >> >> > This one got to technical for me. Do you say that there is no way >> > >> >> > to split the documentation into smaller parts without relying on >> > >> >> > the c++ module/file structure? >> > >> >> >> > >> >> But how would you split it? >> > >> > >> > >> > I do not know. But then I do not know what the generation step can >> > >> > take as different inputs. >> > >> >> > >> The write_python_documentation step should probably take the dolfin >> > >> module and the intermediate representation. >> > > >> > > What is the intermediate representation? >> > >> > It is whatever output we get from the extract_documentation script >> > which we'll add to the dolfin module. How it will look depends a bit >> > on what we need in the write_cpp_documentation and >> > write_python_documentation functions in fenics-doc. >> >> Ok. >> >> > >> >> It makes sense to keep the classes Mesh >> > >> >> and MeshEntity in the mesh/ part of the documentation. Unfortunately, >> > >> >> Swig doesn't add info to the classes in the cpp.py module about where >> > >> >> they were originally defined. This is why we need to pair it with >> > >> >> info from the xml output. >> > >> > >> > >> > Ok, but say we keep all documentation in one module. If you are able >> > >> > to pair the different classes or functions with a module name, or >> > >> > file name you are able to create documentation which is structured >> > >> > after this hierarchy? >> > >> >> > >> We need to figure out something, having everything in the cpp.py >> > >> module would create one big mess and it makes sense to follow the >> > >> DOLFIN C++ structure even for the Python interface. >> > > >> > > Ok, but we do not have everything in a big cpp file. Types get imported >> > > into the Dolfin namespace in __init__ mostly from cpp.py. >> > >> > My point is, there's no telling where the cpp.Mesh class was >> > originally defined. Everything from la to mesh to fem is dumped in the >> > cpp.py module. >> >> Ok, but don't you just need a way to associate the classes to different >> modules? I thought this was what you used the doxygen output to. If we >> instead >> use the module representation we should be able to do this association >> directly with just the dolfin tree as the assosiated types should reside in: >> >> dolfin.submodule.__dict__ >> >> > > Would it help to add the cpp imports to submodules instead of the main >> > > __init__ file? We already have the submodules: >> > > >> > > mesh, common, compilemodules, fem, adaptivity and function >> > > >> > > We could add: >> > > >> > > io, log, nls, ode, parameter, ale >> > > >> > > and drag in stuff from cpp.py in these modules instead. In this way we >> > > structure the main __init__ file better, and you might be able to >> > > structure the pythons reference manual better? >> > >> > I'm quite sure I tried something like this, and the problem was that >> > even if you do: >> > >> > from dolfin.cpp import Mesh >> > >> > in dolfin/mesh/__init__.py >> > >> > the Mesh class will still point to dolfin.cpp when you inspect it --> >> > difficult to create the proper structure for the *.rst files. >> >> Would something that I sceted above work? >> >> > I'll need to double check though that this is really the case. >> > And even if we can use the approach you outline above it means that we >> > have more stuff we need to manually maintain. >> >> Sure, but now everything is throwed into dolfin/__init__.py which is really a >> mess now. >> >> Johan > > This discussion is getting long and complex. I'm tempted to just let > the two of you sort it out between you and then ask for a 10-line > summary. > > And then, of course, suggest a completely different solution. :-)
That's what we're afraid of. In fact this is the reason we're still discussing, to avoid implementing something which you'll tell us to redo :) I think we're quite close to a converged solution were the last details concerns the structure of the Python documentation i.e., how to retrieve the original DOLFIN library structure from the cpp.py module. Just a crazy thought, is it possible to tell Swig to split the modules for us? Such that we get dolfin/mesh --> site-packages/dolfin/mesh. This will also speed up the compilation of the Python module if more than one process is used. Kristian > -- > Anders > > _______________________________________________ > Mailing list: https://launchpad.net/~fenics > Post to : [email protected] > Unsubscribe : https://launchpad.net/~fenics > More help : https://help.launchpad.net/ListHelp > _______________________________________________ Mailing list: https://launchpad.net/~fenics Post to : [email protected] Unsubscribe : https://launchpad.net/~fenics More help : https://help.launchpad.net/ListHelp
