On 21-03-16 10:48, Moritz Lennert wrote:
On 18/03/16 18:38, Pietro wrote:
On Fri, Mar 18, 2016 at 2:16 PM, Moritz Lennert
<mlenn...@club.worldonline.be> wrote:
On 18/03/16 12:58, Pietro wrote:
In your opinion is this true at the module level, or mostly for the
wxGUI ?
No in my opinion things are quite mixed also in C/python modules.
Let's start with a simple example: most of the GRASS modules, mix
nicely logic and cli, several of them have a single main function with
everything inside. I think could be useful to have a more clear
distinction between logic/algorithms and their public interface
(cli/gui).
Why ? I really like the fact that each module works kind of like a
high-level function with a defined public interface.
Yes, and I like too! Indeed I don't want to change this. What I would
like to change is to better distinguish this high-level
functionalities from the low level parts. So for instance just opening
randomly a GRASS core module: r.resamp.stats
https://trac.osgeo.org/grass/browser/grass/trunk/raster/r.resamp.stats/main.c
here are defined:
- static const struct menu, that probably could be useful also for
other modules so should go to the grass-lib
- static char *build_method_list(void), that could be generalized to
used also from other modules and should go to the grass-lib
- static int find_method(const char *name), could be also generalized
to be used by other modules and should go to the grass-lib
- static void resamp_unweighted(void), this function could be also
changed to be more general and moved to the grass-lib too
- static void resamp_weighted(void), same as before.
For each of the above function we can build tests to improve the
reliability, verify performance regression and so on.
So far you can access to this functions only from the CLI interface,
if we clear separate this two level then we can access to this using
CLI, but also using C/python/etc. So for instance If I need select an
option from a menu list on a new module I have to reinvent the wheel,
write my own buggy code and as GRASS developers we end up having
duplicate buggy code in each module.
The main function could stay or be rewritten in python, this is not
really relevant because it is just defining the CLI interface and
calling the functions and finally adding the history metadata and set
the color table.
If we clearly split these two things the GRASS modules
became just an interface to some functions inside the GRASS libraries.
Many modules (if not most) are already that: a combination of GRASS
library
function calls in order to achieve the specific goal the module is
set out
for.
yes they are calling GRASS library functions, but they are also adding
functionalities that (imho) should be included to the GRASS library.
Because they could be useful not only for this specific modules but
also for others.
The above points are a call for refactoring of the code, not
necessarily reorganizing it into different packages in different
source trees.
IMHO, this will always be an issue because of the structure of GRASS:
Someone develops a new module. Code is specific to this module. Then
someone else develops a second module and reproduces part of the code
of the former, because it does what they want and so they just copy it.
At one point several modules share the same code and it becomes clear
that there is a need for this code at library level.
Unless we work with a much more centralized development system, where
a limited number of developers review each proposed module and then
check whether parts of the code should go into the library instead of
the module, I don't really see a different way of doing things as the
way that has grown organically throughout the development history of
GRASS...
We could decide to organize concerted code review moments aiming at
identifying relevant parts of code that should go into libraries, but
I have the feeling that current ad-hoc management is more efficient:
whenever the need is felt, we do it.
BTW, I don't see how separating the source trees solves this issue.
Many people will still continue to code things in modules first and
only after the same code is used in several modules will it become
apparent that it should go into the libs.
mmh, ok, so let's add a more layer: grass-lib, grass-py, and then the
others..
So grass-lib will contain only C (C++?) code and will be not available
at the PyPi website.
grass-py add the python wrap to grass-lib and add API and go to PyPi.
This is the same approach of GDAL[0], PROJ4[1], mapnik[2] that are all
available as python packages.
I do think that add grass to PyPi can only open new prospective and
use cases reaching a broader group of users and developers.
I have the feeling that the question about possibly extracting the
python libs into an installable PyPi package is a different issue. But
then again, what use would these libraries be without the C-libraries
and actually even without the modules ? Yes Pygrass allows you to code
directly with low-level routines, but for me one of the big strength
of the GRASS modular structure and our Python APIs is that they allow
to work with the modules as "functions", so without forcing to use any
low-level access.
+1 this is indeed a great strength. It makes it relatively easy to
string together a number of functions or combining original code with
existing functions. It makes it possible for people like me with very
limited programming experience, to write my own scripts/add-ons which
would cost considerable more effort on other platforms. I would guess
that this modular structure is also one of the factors that make GRASS
GIS so incredible stable (but no clue how that would change or not under
the proposed changes).
- grass-modules: provides all the GRASS core modules (this could be
also a pure python interface calling functions in the C/Python
libraries), and could be split in other sub categories (e.g. imagery,
temporal, terrain, etc).
What is the difference between these modules and the existing ones ?
Except
for your idea to make all module Python modules.
Because things are complicated and the current status quo is not
flexible and/or very limited.
Basically you can create python packages that act as both library and
grass-addons.
That is again another issue: how to handle addons that consist of more
than just one single file script ? I'm not sure that I would agree
that everytime someone codes a more complex addon, the lib part should
immediately go into the core GRASS libs. So, I'd rather see an easier
way to handle such libs in addons. Currently, it is not easy for
people (see [1] for a recent example), but even though I'm talking
without enough knowledge, here, I don't think it should be too
difficult for addon modules to contain files that get installed in
.grass7/addons/lib instead of the current mix of different solutions.
So for instance the r.green modules require scipy and numexpr to run,
and the only way available in 2016 is that user have to care about it
installing the missing libraries and even worst if some of our modules
depend on another addons there is no way to handle this.
I agree that dependency management between addons would be nice, but I
don't see it as that much of an issue. You can always include in your
module a check for the existence of another and stop with a fatal
error encouraging the user to install that module.
+1 would be nice to have some good examples how to do that on the wiki.
AFAIU, there is also the option to use toolboxes, but personally, I
haven't looked at this in detail, yet.
Other things that are not working with the current set up is that
there is not way to said for each version of grass the addon is
available.
So for instance r.green was developed and tested with grass7 stable,
Sören has improved pygrass vector API in trunk (thank you Sören), but
now in the grass-addons repository I have to choose if I want to
support grass-stable or grass-trunk, I have to specify this somewhere
in the manual and hope user read it. Instead I could create a python
package and installing r.green (v0.3) will use grass-lib(v7.0.x) or
installing r.green(v0.4) will use grass-lib(v7.1.x).
Another option might be to distinguish in the addons repository
between grass_stable and grass_trunk, possibly with some easy option
to create a sort of "symbolic" link between the two for modules than
run with both.
Moreover I could
also have the module documentation using sphinx, instead of writing
html code.
That's a totally different issue again.
Perhaps in the future I would like to add a dedicated GUI to this set
of modules or a web interface and then I will have more dependencies,
and make almost impossible to use this new features for an average
user.
I think that if you go that far away from the KISS principle in GRASS
module elaboration, then you are probably better off packaging the
whole thing...
So basically we can remove g.extension and rely on pip or we have to
reinvent the wheel to get these functionalities in g.extension.
I'm not familiar enough with pip to judge what this would entail for
GRASS.
Sorry If I was not clear, hopefully now it is a bit clearer what kind
of advantages we (as developers) could have.
Well, I mainly find that you mix many different issues, and I'm not
convinced that the proposed solution of breaking up the source tree
really solves all of these.
Maybe it might be better to break up your suggestions into separate
parts to discuss them individually. If at the end we see that they all
point to the same solution, your argument will be ever more convincing
:-)
Just my 2¢,
Moritz
[1] https://lists.osgeo.org/pipermail/grass-dev/2016-March/079481.html
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
