[GRASS-dev] compila grass on fedora31: configure: error: *** Unable to locate GDAL library.

[Pietro]

Dear devs,

I have an issue when* I* try to configure ad compile grass.
At the moment I'm on Fedora 31. So I installed the default gdal (2.3.2)
with:

`sudo dnf install gdal gdal-devel gdal-libs`

I executed locally gdalinfo on a tif and it provide the information of the
raster and then I checked the installation paths with:

```
$ /usr/bin/gdal-config --version
2.3.2
$ /usr/bin/gdal-config --libs
-L/usr/lib64 -lgdal
$ /usr/bin/gdal-config --cflags
-I/usr/include/gdal
$ /usr/bin/gdal-config --datadir
/usr/share/gdal
```

So it seems to work, then my configure command is:

```
$ CFLAGS="-g -O1 -Wall -Wextra -Wundef -Wfloat-equal -Wpointer-arith
-Wcast-align -Wstrict-prototypes -Wstrict-overflow=5 -Wwrite-strings
 -Wshadow -Werror-implicit-function-declaration -Waggregate-return
-Wcast-qual -Wswitch-default -Wswitch-enum -Wconversion -Wunreachable-code
-Winit-self -Wuninitialized -fno-common -fexceptions -fstack-protector
-fmudflap -m64 -mtune=native" \
CXXFLAGS="-g -O1 -Wall" \
./configure \
--prefix=$HOME/.local \
--with-zstd \
--with-blas --with-blas-includes=/usr/include/atlas-x86_64-base/ \
--with-lapack --with-lapack-includes=/usr/include/atlas-x86_64-base/ \
--with-cxx \
--with-cairo \
--with-freetype \
--with-freetype-includes=/usr/include/freetype2 \
--with-geos \
--with-gdal=/usr/bin/gdal-config \
--with-gdal-includes=/usr/include/gdal \
--with-gdal-libs=/usr/lib64 \
--with-gdal-share=/usr/share/gdal \
--with-nls \
--with-netcdf \
--with-odbc \
--with-openmp \
--with-postgres \
--with-python=/usr/bin/python3-config \
--with-readline \
--with-proj \
--with-pthread \
--with-sqlite \
--with-wxwidgets=/usr/bin/wx-config
```

But I get the following error:
```
checking host system type... x86_64-pc-linux-gnu
checking for gcc... gcc
[...]
checking whether to use GDAL... yes
checking for gdal-config... /usr/bin/gdal-config
configure: error: *** Unable to locate GDAL library.
```

Do you understand why I still receive this error even if I specify all the
paths of the gdal iclude/libs/share directories?
Any ideas on how to solve/fix this issue?

Thank you for your support.

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] GRASS GIS on an Alpine Linux docker

[Pietro]

Dear all,

Recently I've started looking for a docker image with GRASS, I'm looking
for a minimal image that is as small as possible.
>From my test the other images available where always above 1GB of size, the
current version once built have a size of ~220 MB.
Another difference respect to others dockers available that I found is that
it is possible to select which version use when building the image:

```
sudo docker build --build-arg GRASS_VERSION=7.7 \
  --build-arg PYTHON_VERSION=3  \
  --build-arg PROJ_VERSION=5.2.0 \
  --build-arg PROJ_DATUMGRID_VERSION=1.8 \
  --volume \
  zarch/alpine-grass-gis
```

You can find the repository here:

https://github.com/zarch/docker-alpine-grass-gis

Do you have any feedback?
Any hint to improve the `Dockerfile` or the `README.md` are welcome. :-)

Thank you for the support.
Best regards

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] pygrass v.import_() strange error

[Pietro]

Thank you Yann and Vaclav, I was not aware of this issue:

On Thu, Aug 2, 2018 at 5:06 AM, Vaclav Petras  wrote:

> >>> a.import()
>   File "", line 1
> a.import()
>^
> SyntaxError: invalid syntax
>

I've fixed the the code to support the use of underscore for python
reserved words and add two doctests in  r73029.

Best regards

pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] pygrass v.import_() strange error

[Pietro]

Hi Yann,

On Wed, Jul 25, 2018 at 9:34 AM, Yann Chemin 
wrote:

> v.import_(input=adminfile,output="admin",quiet=True,superquiet=True)
>

Why are you interested to execute "v.import_(" instead of "v.import(", what
is it the purpose of this extra underscore in this case?
It is quite easy to fix this issue but actually I'm not conviced should be
fixed.

Best regards

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] d.linegraph: secondary_width parameter defined as integer but default value is a float

[Pietro]

Dear dev,

I've noticed that the module d.linegraph define the secondary_width
parameter as follow:


```
/* theoretically for other things than points */
secondary_width_opt = G_define_option();
secondary_width_opt->key = "secondary_width";
secondary_width_opt->description = _("Width of point symbol lines");
secondary_width_opt->type = TYPE_INTEGER;
secondary_width_opt->required = NO;
secondary_width_opt->multiple = YES;
secondary_width_opt->answer = "0.1";
```

So the parameter type is supposed to be an integer, but the default value
it is a float?
Is it intentional and in case why?

All the best

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] New dev team on github for initial tests

[Pietro]

Dear Markus,

On Sat, Mar 17, 2018 at 6:27 PM, Markus Neteler <nete...@osgeo.org> wrote:

> Hi,
>
> as per
> https://grasswiki.osgeo.org/wiki/GRASS_GIS_Community_
> Sprint_Bonn_2018#Move_to_Git
>
> I have created a "GRASS GIS dev" team here:
> https://github.com/orgs/OSGeo/teams/grass-gis/members
>
> ... who wants to join, please speak up (and send your account name).
>

Please add me as: zarch

Best regards

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] pygrass: how to use GridModule with a module that has a 'width' parameter

[Pietro]

Dear Moritz,

On Thu, Mar 8, 2018 at 3:22 PM, Moritz Lennert <mlenn...@club.worldonline.be
> wrote:

> On 08/03/18 15:18, Vaclav Petras wrote:
>
> Try `width_` instead of `width` for the module parameter. This is how
>> grass.script works (mostly to avoid conflicts with Python keywords namely
>> lambda), but without checking the code, I don't know if PyGRASS has this
>> implemented.
>>
>
> It doesn't. I tried width_ for both the i.zc parameter and the GridModule
> parameter. Neither were accepted as valid parameters.


I've modifiredt the Module class to support this in r72367 [0].

Let me know if fixed your problem,

all the best

Pietro

[0] https://trac.osgeo.org/grass/changeset/72367
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] New CLI GSoC Idea: Comments, Mentors, Students Needed

[Pietro]

Dear Vaclav,

On Sun, Mar 11, 2018 at 4:33 AM, Vaclav Petras <wenzesl...@gmail.com> wrote:

> I just compiled a another idea for this year GSoC:
>
> https://trac.osgeo.org/grass/wiki/GSoC/2018#Neweasy-to-
> useCLIandAPIforGRASSGIS
>

I like the idea! It is still not clear to me if you think to have a
persistent storage for the "temporary" location/mapset or if they are
generated every time.

Mentors: I'm seeking an additional mentor for this idea. I put myself as
> first, but you can be first or second mentor as you wish.
>

I'm available to co-mentoring this GSoC.

Best regards

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] t.rast.mapcalc: The temporal map type of the sample dataset must be interval

[Pietro]

Hi Martin,

Sorry to resume such old thread, but I have the same issue.

On Wed, Nov 30, 2016 at 11:08 PM, Martin Landa <landa.mar...@gmail.com>
wrote:
> 2016-11-30 23:05 GMT+01:00 Veronica Andreo <veroand...@gmail.com>:
>> Maybe the method=equal, which is the default method for t.rast.mapcalc,
is
>> not defined for time instance data (?).
>>
>> Have you tried changing it? For example, setting method=start? Perhaps in
>> that way, only start time is evaluated to perform the operation between
time
>> series.
>
> thanks for tip, anyway changing method to 'start' doesn't seem to have
> any effect, I am getting the same error. Ma

Did you find a way to go on?

I have two temporal rasters:

$ *t.rast.list Rbeam_2000m@Rbeam_2000m | head*
name|mapset|start_time|end_time
Rbeam_2000m_20120101_00|Rbeam_2000m|2012-01-01 00:00:00|None
Rbeam_2000m_20120101_01|Rbeam_2000m|2012-01-01 01:00:00|None
Rbeam_2000m_20120101_02|Rbeam_2000m|2012-01-01 02:00:00|None
...

and

$ *t.rast.list Rbeam_250m@Rbeam_250m | head *
name|mapset|start_time|end_time
Rbeam_250m_20120101_00|Rbeam_250m|2012-01-01 00:00:00|None
Rbeam_250m_20120101_01|Rbeam_250m|2012-01-01 01:00:00|None
Rbeam_250m_20120101_02|Rbeam_250m|2012-01-01 02:00:00|None
...

But when I try to execute the t.rast.mapcalc I got:

$ *t.rast.mapcalc input=Rbeam_2000m@Rbeam_2000m,Rbeam_250m@Rbeam_250m
expression='(Rbeam_250m - Rbeam_2000m)/Rbeam_250m' output=Rbeam_norm_diff
basename=Rbeam_norm_diff nprocs=4 method=start --o*
Starting temporal sampling...
ERROR: The temporal map type of the sample dataset must be interval
No samples found for map calculation

I have to delete and register the maps with `t.register -i`? Now I'm going
to try this!
Any hints?

Best regards

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] pip install grass-session - No matching distribution found for grass-session

[Pietro]

Hi Markus,

On Sun, Dec 17, 2017 at 5:11 PM, Markus Neteler <nete...@osgeo.org> wrote:

> [...]
> > No matching distribution found for grass-session
>
> Ah, just discovered the trick:
>
> https://grasswiki.osgeo.org/w/index.php?title=Working_with_
> GRASS_without_starting_it_explicitly=0=24906
>
> Now it works fine.


Sorry if I didn't replied to you, but I thought that the "best" reply was
fix the missing information on the wikipage.
Thank you to pointing out the missing information, it is still under
testing on my side.
I'm going to test and use for an analyses in the next weeks, after this I
would like to publish it on pypi, and therefore should be installable with
"pip install grass-session".
Hopefully for the beginning of the next year.

Today, I found out: https://github.com/perrygeo/mower that it seems
similar, my approach should be more flexible for multiprocessing (this is
my impression).

If it works well for my use case, I will publish it on pypi, and I will
write to grass-user.

Thank you for your interest.

Have a nice night.

All the best.

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] NumPy: timeline for dropping Python 2 support

[Pietro Zambelli]

Dear devs,

I've just find out this news on reddit [0], and  I think it is relevant
also for GRASS.
Numy is planning to drop the python2 support [1]:

The current plan is as follows.

  * Until *December 31, 2018*, all NumPy releases will fully support
both Python2 and Python3.
  * Starting on *January 1, 2019*, any new feature releases will support
only Python3.
  * The last Python2 supporting release will be designated as a long
term support (LTS) release, [...], it will be supported by the
community until *December 31, 2019*.
  *  On *January 1, 2020* we will raise a toast to Python2, and
community support for the last Python2 supporting release will come
to an end.


[0]
https://www.reddit.com/r/Python/comments/7d128x/numpy_announces_timeline_for_dropping_python_2/
[1]
https://github.com/numpy/numpy/blob/master/doc/neps/dropping-python2.7-proposal.rst

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [release planning] GRASS GIS 7.2.2

[Pietro]

On Sat, Aug 26, 2017 at 8:36 PM, Markus Neteler <nete...@osgeo.org> wrote:

> On Sat, Aug 26, 2017 at 3:55 PM, Moritz Lennert
> <mlenn...@club.worldonline.be> wrote:
> ...
> > #3402 seems fixed (backport to release72 : r71440)
> >
> > Pietro has also cleaned up the unnecessary imports of builtins in trunk
> > r71438, but this still needs backporting. Maybe we can wait with RC2
> until
> > this is done ?
>
> Yes, sure.


backported: r71435 <https://trac.osgeo.org/grass/changeset/71435>, r71438
<https://trac.osgeo.org/grass/changeset/71438> in r71442
<https://trac.osgeo.org/grass/changeset/71442>[0] to remove importing
builtins in python2

[0] https://trac.osgeo.org/grass/changeset/71442
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] python script utils encoding/decoding overriding unicode with python-future

[Pietro]

On Fri, Aug 25, 2017 at 5:39 PM, Moritz Lennert
 wrote:
>> https://trac.osgeo.org/grass/browser/grass/tags/release_20170810_grass_7_2_2RC1/lib/python/temporal/core.py#L38
>
>
> So this probably needs to be taken care of before the imminent release.

Done in r71438.
it seems ok now.
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] python script utils encoding/decoding overriding unicode with python-future

[Pietro]

On Fri, Aug 25, 2017 at 5:14 PM, Moritz Lennert
<mlenn...@club.worldonline.be> wrote:
> On 25/08/17 17:06, Pietro wrote:
>>
>> Unfortunately builtins is used also in other files, and I’m afraid
>> that I should also fix these:

Actually most of them are working properly, I was thinking that the
module builtins was present in both python2 and python3 and only
contains different functions.

Instead builtins exists only in python3, so most of them works because
in python2 try to import some function from builtins and it raise an
exception not because the function does not exists but because the
module does not exists!

Do you think that should we fix them in any case?

> Is this in grass72 as well, or only trunk ?

Changes in scripts.utils is only in trunk:

https://trac.osgeo.org/grass/browser/grass/tags/release_20170810_grass_7_2_2RC1/lib/python/script/utils.py

 the other are already in the code.

https://trac.osgeo.org/grass/browser/grass/tags/release_20170810_grass_7_2_2RC1/lib/python/temporal/core.py#L38

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] python script utils encoding/decoding overriding unicode with python-future

[Pietro]

Unfortunately builtins is used also in other files, and I’m afraid
that I should also fix these:

$ rg "builtins" . -B 1 -A 5
script/task.py
24-try:
25:from builtins import unicode
26-bytes = str
27-except ImportError:
28-# python3
29-unicode = str
30-

temporal/core.py
38-try:
39:from builtins import long
40-except ImportError:
41-# python3
42-long = int
43-
44-from .c_libraries_interface import *

script/vector.py
27-# python3
28:import builtins as __builtin__
29-unicode = str
30-
31-from .utils import parse_key_val
32-from .core import *
33-from grass.exceptions import CalledModuleError

script/temporal_script_utils_encoding.diff
10-+try:
11:+from builtins import unicode
12-+except ImportError:
13-+# python3
14-+unicode = str
15-+
16-+

script/raster.py
31-try:
32:from builtins import unicode
33-bytes = str
34-except ImportError:
35-# python3
36-unicode = str
37-

gunittest/utils.py
59-except ImportError:
60:import builtins as __builtin__
61-__builtin__._ = new_translator
62-
63-
64-_MAX_LENGTH = 80
65-

pygrass/vector/table.py
13-try:
14:from builtins import long, unicode
15-except ImportError:
16-# python3
17-long = int
18-unicode = str
19-

script/core.py
43-# python3
44:import builtins as __builtin__
45-from os import environb as environ
46-unicode = str
47-__builtin__.__dict__['_'] = __builtin__.__dict__['_'].__self__.lgettext
48-
49-

ctypes/OBJ.x86_64-pc-linux-gnu/temporal.py
13-try:
14:from builtins import long
15-except ImportError:
16-long = int
17-
18-import os
19-import sys

ctypes/OBJ.x86_64-pc-linux-gnu/cluster.py
13-try:
14:from builtins import long
15-except ImportError:
16-long = int
17-
18-import os
19-import sys

ctypes/OBJ.x86_64-pc-linux-gnu/ogsf.py
13-try:
14:from builtins import long
15-except ImportError:
16-long = int
17-
18-import os
19-import sys

ctypes/OBJ.x86_64-pc-linux-gnu/proj.py
13-try:
14:from builtins import long
15-except ImportError:
16-long = int
17-
18-import os
19-import sys

ctypes/OBJ.x86_64-pc-linux-gnu/vector.py
13-try:
14:from builtins import long
15-except ImportError:
16-long = int
17-
18-import os
19-import sys

ctypes/OBJ.x86_64-pc-linux-gnu/stats.py
13-try:
14:from builtins import long
15-except ImportError:
16-long = int
17-
18-import os
19-import sys

ctypes/OBJ.x86_64-pc-linux-gnu/arraystats.py
13-try:
14:from builtins import long
15-except ImportError:
16-long = int
17-
18-import os
19-import sys

ctypes/OBJ.x86_64-pc-linux-gnu/vedit.py
13-try:
14:from builtins import long
15-except ImportError:
16-long = int
17-
18-import os
19-import sys

ctypes/OBJ.x86_64-pc-linux-gnu/rowio.py
13-try:
14:from builtins import long
15-except ImportError:
16-long = int
17-
18-import os
19-import sys

ctypes/OBJ.x86_64-pc-linux-gnu/display.py
13-try:
14:from builtins import long
15-except ImportError:
16-long = int
17-
18-import os
19-import sys

ctypes/OBJ.x86_64-pc-linux-gnu/date.py
13-try:
14:from builtins import long
15-except ImportError:
16-long = int
17-
18-import os
19-import sys

ctypes/OBJ.x86_64-pc-linux-gnu/raster3d.py
13-try:
14:from builtins import long
15-except ImportError:
16-long = int
17-
18-import os
19-import sys

ctypes/OBJ.x86_64-pc-linux-gnu/segment.py
13-try:
14:from builtins import long
15-except ImportError:
16-long = int
17-
18-import os
19-import sys

ctypes/OBJ.x86_64-pc-linux-gnu/gis.py
13-try:
14:from builtins import long
15-except ImportError:
16-long = int
17-
18-import os
19-import sys

ctypes/OBJ.x86_64-pc-linux-gnu/nviz.py
13-try:
14:from builtins import long
15-except ImportError:
16-long = int
17-
18-import os
19-import sys

ctypes/OBJ.x86_64-pc-linux-gnu/gmath.py
13-try:
14:from builtins import long
15-except ImportError:
16-long = int
17-
18-import os
19-import sys

ctypes/OBJ.x86_64-pc-linux-gnu/rtree.py
13-try:
14:from builtins import long
15-except ImportError:
16-long = int
17-
18-import os
19-import sys

ctypes/OBJ.x86_64-pc-linux-gnu/raster.py
13-try:
14:from builtins import long
15-except ImportError:
16-long = int
17-
18-import os
19-import sys

ctypes/OBJ.x86_64-pc-linux-gnu/dbmi.py
13-try:
14:from builtins import long
15-except ImportError:
16-long = int
17-
18-import os
19-import sys

ctypes/OBJ.x86_64-pc-linux-gnu/imagery.py
13-try:
14:from builtins import long
15-except ImportError:
16-long = int
17-
18-import os
19-import sys

ctypes_pregit/ctypesgencore/processor/operations.py
97-def fix_conflicting_names(data, opts):
98:"""If any descriptions from the C code would overwrite Python builtins or
99-other important names, fix_conflicting_names() adds underscores
to resolve
100-the name conflict."""
101-
102-# This is the order of priority for names
103-descriptions = data.functions + data.variables + data.structs + \
--

Re: [GRASS-dev] python script utils encoding/decoding overriding unicode with python-future

[Pietro]

Hi Moritz,

On Thu, Aug 24, 2017 at 4:58 PM, Moritz Lennert
 wrote:
> Is the override of the definition of unicode really necessary ?

No it is not.
it is sufficient replace the try/except with an if.
Done in r71435 [0]

http://trac.osgeo.org/grass/changeset/71435/grass/trunk/lib/python/script/utils.py
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] g.gui.animation freezes when loading strds

[Pietro]

Hi Markus,

On Thu, Aug 10, 2017 at 8:34 PM, Markus Metz
<markus.metz.gisw...@gmail.com> wrote:
> the OS limit of the number of open files is given in
> /proc/sys/fs/file-max
> but that is irrelevant, what matters is the maximum allowed number of open 
> files for the current user
>
> [cut]
>
> ulimit -n reports the maximum allowed number of open files for the current 
> user, this is the relevant limit

Thank you for the clarification. In my system the user limit is 1024.

I can confirm that raise the current limit with:

```bash
$ ulimit -n 4096
```

fixed the problem.

> A solution to the error "Too many open files" would be to open the files/maps 
> one at a time and closing them again, similar to the -z flag of r.series.

I was trying to understand where these file are opened, and it seems
to me that the python code is always opening and closing all the
files, It is not clear to me which ctypes function is just opening the
files without closing them.

Best regards

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] g.gui.animation freezes when loading strds

[Pietro]

Hi Veronica,

On Tue, Aug 8, 2017 at 3:46 PM, Veronica Andreo <veroand...@gmail.com>
wrote:

I've also experienced the freezing issue (dunno if it the same thing or
> not). Here, the rendering freezes if strds has more than 500 maps. I do not
> even get to the export part. Cancel does not close it, so I have to kill it
> (trunk r71319).
>
Same problem here, when I reach 502 maps raise the following exception:

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/wx-3.0-gtk2/wx/_core.py",
line 16762, in 
  File "/home/pietro/.local/grass-7.3.svn/gui/wxpython/animation/frame.py",
line 194, in SetAnimations
  File "/home/pietro/.local/grass-7.3.svn/gui/wxpython/animation/controller.py",
line 305, in SetAnimations
  File "/home/pietro/.local/grass-7.3.svn/gui/wxpython/animation/controller.py",
line 328, in _setAnimations
  File "/home/pietro/.local/grass-7.3.svn/gui/wxpython/animation/controller.py",
line 407, in _updateBitmapData
  File "/home/pietro/.local/grass-7.3.svn/gui/wxpython/animation/provider.py",
line 234, in Load
  File "/home/pietro/.local/grass-7.3.svn/gui/wxpython/animation/provider.py",
line 366, in Render
  File "/usr/lib64/python2.7/multiprocessing/__init__.py", line 218, in Queue
  File "/usr/lib64/python2.7/multiprocessing/queues.py", line 63, in __init__
  File "/usr/lib64/python2.7/multiprocessing/synchronize.py", line
147, in __init__
  File "/usr/lib64/python2.7/multiprocessing/synchronize.py", line 75,
in __init__
OSError: [Errno 24] Too many open files

But checking the number of opened file on my system, it seems two order
lower than the maximum:

$ cat /proc/sys/fs/file-nr23422   0   1628292
$ cat /proc/sys/fs/file-max 1628292

I’ve attached also the complete log of the command in DEBUG mode, but I was
not able to understand where the problem could be. I’ve also rebooted just
to be sure that the system was fresh, but again after 502 maps it breaks
with the same error.

Any ideas?

Pietro
​


log_gui_animation.7z
Description: application/7z-compressed
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] move everything from /lib/init/grass.py to /lib/python/init

[Pietro]

Hi Vaclav,

sorry for the delay but in the last day I was off-line.

On Mon, Jul 17, 2017 at 5:36 PM, Vaclav Petras <wenzesl...@gmail.com> wrote:

>
> On Mon, Jul 17, 2017 at 12:36 AM, Pietro <peter.z...@gmail.com> wrote:
>
>>
>> On Fri, Jul 14, 2017 at 6:00 PM, Vaclav Petras <wenzesl...@gmail.com>
>> wrote:
>>
>>> This is exactly what I had in my mind when doing the last major changes
>>> in the grass.py file.
>>>
>> I generally like the layout you suggested. It seems to me that choosing a
>>> good name for the whole module will be a bit tricky.
>>>
>>
>> This is intended as a proof of concept to see the feasibility.
>> I've try to found a better name but didn't come up to my mind...
>> Perhaps: session instead of init?
>>
>> My final objective is to be able to do something like:
>>
>
> That makes sense. In fact, that's very similar to a file I drafted some
> time ago splitting the data initialization and runtime in
> grass.script.setup and adding Session (see the attached file). Another
> example, for a different case, is here:
>
> https://github.com/wenzeslaus/g.remote/blob/master/grasssession.py
>

Nice module, I was not aware of it!
However I think that the purpose is slightly different. The grassession
aims is to generate the session remotely, here I would like to setup a
local session. It is true that I should be able to just connect through ssh
to the localhost... but it seems to me not the right way. So I've sketch a
possible implementation just to see how it could work, see:

https://git.osgeo.org/gogs/zarch/grass/commit/b9cb69a1d7381924b0c2229ba43f21b1b7473c72



>
>
> *# Perhaps in GRASS8 we will be able to skip this! ;-)*
>> sys.append(os.environ.get('GISBASE', '/home/pietro/my/gisbase'))
>>
>
> Added to the list, but how to do it remains unclear (many different
> discussions in Trac and ML):
>
> https://trac.osgeo.org/grass/wiki/Grass8Planning
>

Thank you


from grass.init import Session
>>
>> # with statement
>> with Session('mygisdbase/location/mapset') as session:
>> # do my stuff here
>> ```
>>
>>
>
> Unfortunately, here is where the trouble begins. The above leads to the
> following:
>
> with Session as session:
> session.run_command(...)
>
> which fits with API which already exists for Ruby:
>
> https://github.com/jgoizueta/grassgis/
>
> GrassGis.session configuration do+
> r.info 'slope'
> g.region '-p'
> end
> The trouble is that session (at least in Python) needs to depend on the
> rest of the library because it is the interface for the rest (on demand
> imports may help here).
>

Sorry I'm not sure that I get your point here, what do you mean?
The following code is running at the moment on my machine:

```python
import os
import sys

MAPSET = '/home/pietro/docdat/gis/nc_basic_spm_grass7/user1/'
GISBASE = '/home/pietro/docdat/src/gis/ggrass/dist.x86_64-pc-linux-gnu/'

# set the path
sys.path.append(os.path.join(os.environ.get('GISBASE', GISBASE),
 'etc', 'python'))

# import the python GRASS libraries
from grass.script.core import run_command
from session import Session


with Session(MAPSET) as session:
run_command('r.slope.aspect', elevation='elevation',
slope='slope', aspect='aspect',
overwrite=True)
```


So perhaps having grass.init or grass.setup with the low level functions
> and then a separate grass.session with a nice interface which may depend on
> all other modules may be a better way. (Although having each function from
> the library as a method of Session calls for more thinking about
> grass.session.Session.
>

I was thinking to add this Session class definition inside init/session.py
to then try to refactor the main function in parser.py:

https://git.osgeo.org/gogs/zarch/grass/src/grass_session/lib/python/init/parser.py#L189

to start a session and then execute all the rest of the functions to
start/use the grass shell/gui.


Just to be clear: I definitively think this should be done. I'm just not
> sure what is the right way.
>

I'm not sure too! This is why I'm trying to sketch some code drafting to
understand what is feasible and how should this organized.
Thank you for taking time to review the code/changes and to give me
feedback.

Cheers

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] remove todo comment from grass.script.utils set_path

[Pietro]

Dear all,

I would like to remove the following comment:

# TODO: why dirname is checked first - the logic should be revised

in the set_pat function:

https://trac.osgeo.org/grass/browser/grass/trunk/lib/python/script/utils.py#L372

the reason is that if we invert the logic as suggested by the comment the
function could find the libraries in the GRASS library path before the
local directories.
Start looking for the module from the local directory it seems more obvious
to me, as first option, than looking around in the system path.
Furthermore this choice is particularly useful for development because as
developer we don't have to build every time, but we can simply run and test
the module from the local shell with:

python modulename.py key=value --overwrite

What do you think?
Is it clearer now?

Could I remove the comment?

Best regards

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] move everything from /lib/init/grass.py to /lib/python/init

[Pietro]

Hi Vaclav,

On Fri, Jul 14, 2017 at 6:00 PM, Vaclav Petras <wenzesl...@gmail.com> wrote:

> This is exactly what I had in my mind when doing the last major changes in
> the grass.py file.
>
I generally like the layout you suggested. It seems to me that choosing a
> good name for the whole module will be a bit tricky.
>

This is intended as a proof of concept to see the feasibility.
I've try to found a better name but didn't come up to my mind...
Perhaps: session instead of init?

My final objective is to be able to do something like:

```python
import os
import sys

*# Perhaps in GRASS8 we will be able to skip this! ;-)*
sys.append(os.environ.get('GISBASE', '/home/pietro/my/gisbase'))

from grass.init import Session

# open - close mode
session = Session('mygisdbase/location/mapset')
session.open()
# do my stuff here...
session.close()

# with statement
with Session('mygisdbase/location/mapset') as session:
# do my stuff here
```


Also I think one reason for having them there was that grass.py works
> without a the G Python lib found.
>

ok, but I don't see any advantage to have grass.py that works without
loading the grass python libraries. and with the current status we have
code duplication, that mean more code to read, test and maintain.
If you are afraid that the grass python libraries might be broken and
therefore the user can have troubles, they will have troubles in any case
(no gui, several important modules missing). So I will consider stopping
the user earlier.as a feature.

Let me know.

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] move everything from /lib/init/grass.py to /lib/python/init

[Pietro]

Dear devs,

What do you think if we move all the function that at the moment are
contained in `/lib/init/grass.py` into a new subfolder under `/lib/python`?

The main advantages that I see are:

- start a python script in GRASS just setting the the python path and then
I can use the same functions that are defined for the normal GRASS start
up, without duplicate code;
- We can add unittest for the start up functions
- We can remove code duplication between grass.init and grass.script.core


What do you think?

I did the changes and at least on my computer GRASS is working, all the
changes are available at this link:

https://git.osgeo.org/gogs/zarch/grass/commit/27c8351423da645d938fb6c2e54781ee24e6f074


I've split the functions that were contained in the grass.py in the
following files, any comments?


```
$ rg -e "^def\s[a-z_]+\(|^class\s[A-Z][a-z]*|^[A-Za-z_]+\s*="

*gettext.py*
11:_ = gettext.gettext

*message.py*
7:_DEBUG = None
10:def warning(text):
14:def fatal(msg):
19:def message(msg):
24:def is_debug():
41:def debug(msg):

*utils.py*
13:def grep(pattern, lines):
23:def print_params():
62:def get_username():
85:def make_fontcap():
93:def ensure_db_connected(mapset):
102:def get_shell():

*gui.py*
20:def read_gui(gisrc, default_gui):
53:def check_gui(expected_gui):
94:def save_gui(gisrc, grass_gui):
102:def gui_startup(grass_gui):
135:def start_gui(grass_gui):
148:def close_gui():
165:def clear_screen():
176:def show_banner():
188:def say_hello():
203:def show_info(shellname, grass_gui, default_gui):
229:def csh_startup(location, location_name, mapset, grass_env_file):
280:def bash_startup(location, location_name, grass_env_file):
313:PROMPT_COMMAND=grass_prompt\n""" % (_("2D and 3D raster MASKs present"),
338:def default_startup(location, location_name):
353:def done_message():

*subprocess.py*
10:def call(cmd, **kwargs):

*info.py*
5:BUILD_GISBASE = "@GISBASE@"
6:BUILD_PROJSHARE = "@CONFIG_PROJSHARE@"
7:CMD_NAME = "@START_UP@"
8:GRASS_VERSION = "@GRASS_VERSION_NUMBER@"
9:LD_LIBRARY_PATH = '@LD_LIBRARY_PATH_VAR@'
12:GISBASE = os.path.normpath(os.environ.get("GISBASE", BUILD_GISBASE))
13:GRASS_PROJSHARE = os.environ.get("GRASS_PROJSHARE", BUILD_PROJSHARE)

*data.py*
13:def create_location(gisdbase, location, geostring):
47:def is_mapset_valid(full_mapset):
56:def is_location_valid(gisdbase, location):
72:def get_mapset_invalid_reason(gisdbase, location, mapset):
106:def set_mapset(gisrc, arg, geofile=None, create_new=False):
191:def set_mapset_interactive(grass_gui):
218:def lock_mapset(mapset_path, force_gislock_removal, user, grass_gui):
270:class MapsetSettings(object):
301:def load_gisrc(gisrc, gisrcrc):

*clean.py*
8:def try_remove(path):
15:def try_rmdir(path):
22:def cleanup_dir(path):
33:class Cleaner(object):  # pylint: disable=R0903

*env.py*
22:def path_prepend(directory, var):
31:def path_append(directory, var):
40:def set_paths(grass_config_dir):
97:def set_defaults():
126:def set_display_defaults():
134:def set_browser():
173:def ensure_home():
180:def clean_env(gisrc):
192:def load_env(grass_env_file):
218:def set_language(grass_config_dir):

*compatibility.py*
5:ENCODING = locale.getdefaultlocale()[1]
11:def to_text_string(obj, encoding=ENCODING):

*path.py*
11:_WXPYTHON_BASE = None
14:def readfile(path):
21:def writefile(path, s):
27:def gpath(*args):
35:def wxpath(*args):
50:def get_grass_config_dir():
78:def create_tmp(user, gis_lock):
125:def clean_temp():
132:def get_gisrc_from_config_dir(grass_config_dir, batch_job):
143:def get_grass_env_file(sh, grass_config_dir):
160:def find_exe(pgm):

*system.py*
4:windows = sys.platform == 'win32'
5:cygwin = "cygwin" in sys.platform
6:macosx = "darwin" in sys.platform

*gisrc.py*
13:def create_gisrc(tmpdir, gisrcrc):
33:def read_gisrc(filename):
54:def read_env_file(path):
64:def write_gisrc(kv, filename):
71:def create_initial_gisrc(filename):

*batch.py*
9:def get_batch_job_from_env_variable():
32:def run_batch_job(batch_job):

*parser.py*
29:help_text = r"""GRASS GIS $VERSION_NUMBER
114:def help_message(default_gui):
121:class Parameters(object):
135:def parse_cmdline(argv, default_gui):
183:def main(argv):
```

I wish you all a nice week-end.

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Implement a REST API for GRASS

[Pietro]

Dear Māris,

On Wed, May 24, 2017 at 8:52 PM, Maris Nartiss <maris@gmail.com> wrote:

> GRASS GIS code has never been developed with security in mind. I would
> not suggest to run it in a non-trustable environment.
>

Do you think that expose some GRASS modules through a REST API can be more
dangerous than exposing the same modules through a WPS service? Why?

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Implement a REST API for GRASS

[Pietro]

Hi Moritz,

On Wed, May 24, 2017 at 3:31 PM, Moritz Lennert <
mlenn...@club.worldonline.be> wrote:

> - first of all: is build a REST API for GRASS a stupid idea? should we
>> use/promote just the WPS?
>>
>
>
> Well, what is your argument agains WPS ? Why do you think a REST API is
> necessary / useful ?
>

The main REST advantage, that I see, is that it is easier to explore and
understand. If I need to perform just a simple operation I can navigate
over the API and understand how to perform a request in few minutes, with
no need to understand the protocol and the link between different
elements...

The REST is not bonded to one single format (xml). JSON is easier/faster to
read usually than xml and it is generally a better fit for data. REST
allows better support for browser clients due to it’s support for JSON and
generate a browser client for it should be quite straight forward. It seems
like REST has better performance and scalability.

Nowadays REST services seems to be the industry standard (is internally
used by Google, Amazon, Yahoo, etc)...

You may found further analyses here:
http://docs.opengeospatial.org/per/16-035.html#_advantages_and_disadvantages_of_using_rest

Respect to the WPS protocols a GRASS REST API has the advantage that don't
have to be a general standard for every possible processing service but can
be specifically designed/optimized for GRASS.

So in my mind a GRASS REST API is just a simple and convenient way to
expose some selected GRASS modules capabilities through a browser.

Best regards

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] Implement a REST API for GRASS

[Pietro]

Dear devs,

In the last days I've started thinking to a REST API
<https://en.wikipedia.org/wiki/Representational_state_transfer> for GRASS.
I think would be nice to offer some of the GRASS functionalities through a
web service.

I would like to detailed the whole REST API using the YAML
<http://www.yaml.org/> format following the Swagger <http://swagger.io>
specifications/services, but before start the real work I would like to
receive feedback on the idea...

- first of all: is build a REST API for GRASS a stupid idea? should we
use/promote just the WPS?
- I've started a first conceptual draft of a possible API available at this
repository:
https://github.com/HotMaps/GRASS-rest/blob/master/swagger/grass_gis_concept.yaml,
the file is just a sketch of what the API could looks like/do (and is not
yet compliant/usable with Swagger).

I've tried to synthesize the concept in README
<https://github.com/HotMaps/GRASS-rest/blob/master/README.md> let me know
if you have comments and ideas, please feel free to open new issues or pull
requests.

I see a major issue that is that: GRASS is using unix permissions to define
the rules on who can access on what.
To preserve this part, the REST service must be executed as super-user in
order to be able to execute a module as the owner of a specific mapset and
all the user of the platform must be also user of the system.
The other option that I see is to execute the service as a normal user of
the system who is the owner of all the gisbase, locations and mapsets, and
the permission are checked before the execution of each module. The
permission check must be implemented in a new layer on top of GRASS. With
this solution the user of the platform are different from the user and
group of the system.

Any ideas that you want to share?

Best regards

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] pyGRASS: ParallelModuleQueue

[Pietro]

Hi Martin,

On Wed, Nov 30, 2016 at 12:25 PM, Martin Landa <landa.mar...@gmail.com> wrote:
> is it possible to use progress info when using ParallelModuleQueue?
> Something similar to grass.percent(). Eg.

Yes, I think so, if you pass the stderr_=subprocess.PIPE and
stdout_=subprocess.PIPE parameters to the module, then you should be
able to write the current percentage using grass.percent() or
msgr.percent().

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Towards SVN - git integration

[Pietro]

Hi Markus,

On Mon, Aug 29, 2016 at 1:12 PM, Markus Neteler <nete...@osgeo.org> wrote:
> (back to this thread as it comes with a better subject title)
> there was already a GRASS git repo, see
> https://lists.osgeo.org/pipermail/grass-dev/2016-June/080687.html
> --> https://git.osgeo.org/gogs/grass/grassgis
>
> it seems to be outdated but I'll ask Sandro about this.
> Or Pietro Z knows?

Actually I just synced manually (git svn fetch), perhaps we can simply
configure GRASS server that for each commit just sync the git
repository.
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] GSoC 2016 - PyQt

[Pietro]

Hi Ondrej,

On Tue, May 24, 2016 at 12:48 PM, Ondřej Pešek <pesej.ond...@gmail.com>
wrote:
>
> the period of GSoC starts. I just want to present myself. My name is
> Ondrej Pesek and I'm from Czech Republic. I study geodesy, cartography and
> geomatics at Czech technical university in Prague. My project for GSoC 2016
> is reimplementation of GUI generated from xml, wxpython -> PyQt. You can
> see everything at [1].
>
> I'm open to all your requests, comments and hates. Don't hesitate to write
> me, I would appreciate it.
>

Perhaps, it could be useful to have a look at pygrass `Module` class [0],
that already parse the generated xml and instantiate `Parameter` class [1]
that validate the input from the user in the `_check_value` function [2].
So should be possible to dynamically generate a `Qt Widget` from a
`Parameter` instance and then compose everything in complete final
`QtWidget`/`QtDialog`. If you think that  in the `Module` / `Parameter`
class definition something is missing/wrong or bad implemented, after
discussion, we can change it. They were not developed with this use case in
mind and perhaps some changes are needed however I do think they are a
starting point...

You can see a good implementation of this logic in the fromlayout
project[3] that it is well written, support both PyQt and PySide and
Python2 and Python3.

Together with Francesco Geri we started writing a plugin for QGis (a proof
of concept (it works, but a lot of things are missing), just few hours of
work of our spare time..) that generate dynamically the form for the GRASS
modules, the code is not good and it is using qgis API... but I think that
can explain what I have in mind.

I think that we should support both: PyQt and PySide, and I think you
should use as primary choice for development Python3, support for Python2
it is much easier to add later if needed.

I hope these information/ideas can be useful for your GSoC.
Have fun!

all the best

Pietro


[0]
https://trac.osgeo.org/grass/browser/grass/trunk/lib/python/pygrass/modules/interface/module.py#L266
[1]
https://trac.osgeo.org/grass/browser/grass/trunk/lib/python/pygrass/modules/interface/parameter.py#L97
[2]
https://trac.osgeo.org/grass/browser/grass/trunk/lib/python/pygrass/modules/interface/parameter.py#L15
[3] https://github.com/PierreRaybaut/formlayout
[4] https://bitbucket.org/fragit/rgreengui/src
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Tests failing with "requires string, provided"

[Pietro]

Hi,

On Fri, May 13, 2016 at 5:48 PM, Vaclav Petras <wenzesl...@gmail.com> wrote:
> A lot of tests are failing:
>
> http://fatra.cnr.ncsu.edu/grassgistests/reports_for_date-2016-05-02-07-00/report_for_nc_basic_spm_grass7_nc/testfiles.html
> http://fatra.cnr.ncsu.edu/grassgistests/reports_for_date-2016-05-03-07-00/report_for_nc_basic_spm_grass7_nc/testfiles.html
>
> Seems like PyGRASS related marameter checking:
>
> The Parameter  require a string,  instead is provided

the commit: r68465, fix the tests:

http://fatra.cnr.ncsu.edu/grassgistests/reports_for_date-2016-05-20-07-00/report_for_nc_basic_spm_grass7_nc/testfiles.html

All the best

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] start supporting python3 for GRASS

[Pietro]

Dear devs,

I committed some changes to start supporting python3 (r68348 - r68367)
the commits are a bit messy... but I had some problem with git-svn
.
I've used autopep8 to clean the code from old-style syntax (e.g.
dict.has_key, etc) and then I compared file generated with 2to3 and
manually selected each change that it was needed.

I execute our tests suite, I got a regression in:

{{{
$ python ./temporal/t.connect/testsuite/test_distr_tgis_db_raster3d.py
E
==
ERROR: setUpClass (__main__.testRaster3dExtraction)
--
Traceback (most recent call last):
  File "./temporal/t.connect/testsuite/test_distr_tgis_db_raster3d.py",
line 30, in setUpClass
cls.runModule("g.mapset", flags="c", mapset=mapset_name)
  File 
"/home/pietro/docdat/src/gis/grass_git/dist.x86_64-pc-linux-gnu/etc/python/grass/gunittest/case.py",
line 1095, in runModule
errors=errors)
CalledModuleError: Module run 1 g.mapset ended with error
Process ended with non-zero return code g.mapset(mapset='test3d3',
flags=u'c'). See the following errors:
ERROR: There appears to be an active GRASS session in selected mapset
   


--
Ran 0 tests in 6.246s
FAILED (errors=1)
}}}

All the other tests seem to work on my pc.
If you find a new bugs that has been introduced from these changes,
please consider to write a test that highlight the problem before fix
it.

Best regards

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] creating temporary mapsets in a parallelized python script

[Pietro]

> The main issue with use_temp_region() is that the clean-up function
> uses the current value of the environment variable to determine which
> region to delete.
>
> This could be fixed by passing a lambda to atexit.register(), e.g.
>
> def use_temp_region():
> name = "tmp.%s.%d" % (os.path.basename(sys.argv[0]), os.getpid())
> run_command("g.region", save=name, overwrite=True)
> os.environ['WIND_OVERRIDE'] = name
> atexit.register(lambda: run_command("g.remove", flags='f', quiet=True, 
> type='region', name=name))

Perhaps we could decorate the function with contextmanager:

{{{
from contextlib import contextmanager

@contextmanager
def use_temp_region(**reg):
name = "tmp.%s.%d" % (os.path.basename(sys.argv[0]), os.getpid())
original = region()
try:
reg = parse_command("g.region", save=name, overwrite=True, **reg)
os.environ['WIND_OVERRIDE'] = name
yield reg
finally:
# clean created variable and region
print('WIND_OVERRIDE' in os.environ)
os.environ.pop('WIND_OVERRIDE')
run_command("g.remove", flags='f', quiet=True, type='region', name=name)
# restore previous region
for key in 'projection,zone,cells'.split(','):
original.pop(key)
run_command("g.region", **original)
}}}

and then use the function with:

{{{
run_command("g.region", flags='p')
print('=' * 30)

with use_temp_region(res=100) as tmp_region:
run_command("g.region", flags='p')
print('=' * 30)

run_command("g.region", flags='p')
print('=' * 30)
}}}

All the best

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] PEP 394 - The "python" Command on Unix-Like Systems

[Pietro]

On Tue, Mar 29, 2016 at 3:12 PM, Vaclav Petras <wenzesl...@gmail.com> wrote:
>
> On Tue, Mar 29, 2016 at 5:33 AM, Pietro <peter.z...@gmail.com> wrote:
>>
>> I have fixed these errors, but since these changes are not fixing
>> bugs, I've just reverted them. I will commit these change back to
>> trunk as soon as we can start working on grass72.
>
>
> I hope this won't become a common practice. I think trunk is where the
> development happens. It is not a stable branch regardless of any plans.
> Should I also revert all my non-bugfixes from last weeks? If needed, I can
> certainly do that in the 7.2 release branch once thare is one. I just don't
> see need to do this in trunk.

Perhaps I miss-interpreted what Martin said to me in (Re: [GRASS-SVN]
r68140 - in grass/trunk/scripts: d.correlate...):

"personally I would wait with such changes to the moment when we will
create releasebranch_7_2 (two months before planning 7.2. release. For
now I would suggest to revert these changes from trunk. Any opinion?"

no one reply on his question, and therefore I reverted. Perhaps I
should wait longer...

What can I say is that it is quite challenging to maintain such large
changes locally on my pc, with not possibility to commit smaller
changes...

Git+pull request, in this case, seems much more flexible.

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Add a possible decorators to gunittest

[Pietro]

Hi Vaclav,

On Tue, Mar 29, 2016 at 3:02 PM, Vaclav Petras <wenzesl...@gmail.com> wrote:
> Although I can see advantages of decorators, Python unittest package uses
> setUp() and tearDown() functions for that purpose. Introducing another
> mechanism might interfere with the execution of the tests. If nothing else,
> you would need to re-raise the exception to signal that the test failed. I
> suggest to try setUp() and tearDown() as unittest advocates and only when it
> won't work look at decorators (which are used in unittest but it different
> way).

ok, I changed it in r68183, now is using setUp and tearDown.

All the best

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] Add a possible decorators to gunittest

[Pietro]

Dear Vaclav,

I wrote a first draft of a decorator: change_environ [0], that:
1. overwrite the environment variables,
2. execute the test and
3. restore the environment variables.

Probably I should modify the function with something like:

{{{
   try:
   func(*args, **kwargs)
   finally:
   # restore the environment variables
   for k, v in envs.items():
   oval = original_envs[k]
   if oval == NOT_FOUND:
   os.environ.pop(k)
   else:
   os.environ[k] = oval
}}}

Otherwise if the function raise an exception the current
implementation will not restore the environment variables back to
their original value.

But I think this kind of functionalities is general enough to be
somewhere in the gunittest code.

What do you think? Any idea?

All the best.

Pietro

[0] 
https://trac.osgeo.org/grass/browser/grass/trunk/lib/python/script/testsuite/test_utils.py#L10
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] g.copy with pygrass

[Pietro]

Hi Paulo,


On Tue, Mar 29, 2016 at 10:51 AM, Paulo van Breugel
<p.vanbreu...@gmail.com> wrote:
> from grass.pygrass.modules.shortcuts import general as g
>>>> vectinmap = 'MASK'
>>>> vectoutmap = 'TMP'
>>>> g.copy(raster="%s,%s" % (vectinmap,vectoutmap) )

Indeed this is wrong and I fix it.

Please feel free to edit the wiki page if you found other errors.

All the best.

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] correction typo on pygrass sample pygrass scripts manual page

[Pietro]

Hi Paulo,

On Mon, Mar 28, 2016 at 10:48 AM, Paulo van Breugel
<p.vanbreu...@gmail.com> wrote:
> typo on
> https://grasswiki.osgeo.org/wiki/Python/pygrass#Sample_PyGRASS_scripts
>
> Under the header "Interface to listing maps (g.list/g.mlist)", second
> example with "glist" method of the Mapset class, it reads:
>
> m.glist('rast', pattern='elev-*')
>
> This perhaps works for GRASS 6.4, but for GRASS 7.0 the first argument
> should be 'raster'. In the example above, the syntax for both 6.4 and 7.0 is
> given.

Pygrass is not available in grass64, so I fix it, the code was written
before the change in grass70.
Please feel free to fix these erorrs.

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] PEP 394 - The "python" Command on Unix-Like Systems

[Pietro]

Hi Vaclav,

On Fri, Mar 25, 2016 at 8:38 PM, Vaclav Petras <wenzesl...@gmail.com> wrote:
> Nice! But please, check the tests:
>
> http://fatra.cnr.ncsu.edu/grassgistests/reports_for_date-2016-03-25-07-00/report_for_nc_basic_spm_grass7_nc/lib/python/gunittest/test_doctests/index.html
> http://fatra.cnr.ncsu.edu/grassgistests/reports_for_date-2016-03-25-07-00/report_for_nc_basic_spm_grass7_nc/lib/python/gunittest/test_gmodules/index.html
> http://fatra.cnr.ncsu.edu/grassgistests/reports_for_date-2016-03-25-07-00/report_for_nc_basic_spm_grass7_nc/lib/python/temporal/test_doctests/index.html

I have fixed these errors, but since these changes are not fixing
bugs, I've just reverted them. I will commit these change back to
trunk as soon as we can start working on grass72.

I've also fixed locally the temporal modules (to be importable in python3).
Therefore I was able to get the xml description for all the modules
under python3.

Since this morning I add the support to compile with both python2 and
python3 (this is another set of changes that are waiting for grass72).

Best regards

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS-SVN] r68180 - grass/trunk/lib/python/script

[Pietro]

Hi Martin,

On Tue, Mar 29, 2016 at 9:16 AM, Martin Landa <landa.mar...@gmail.com> wrote:
> candidate for backport?

Yes, I think this could be backported with no problems.
Current implementation is not working when for example the LC_ALL=C,

see tests in r68179.

All the best.

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] PEP 394 - The "python" Command on Unix-Like Systems

[Pietro]

On Thu, Mar 24, 2016 at 12:52 AM, Luca Delucchi <lucadel...@gmail.com> wrote:
> On 13 March 2016 at 10:52, Maris Nartiss <maris@gmail.com> wrote:
>> I personally would like to see adoption of PEP8 and moving all python
>> code to be python3 ready. As old python versions are dying out, there
>> is less and less initiative to not do so.
>> Also related to
>> https://lists.osgeo.org/pipermail/grass-dev/2016-February/078828.html
>>
>
> I also have the same feeling, we should work to be python3 ready,
> maybe we could release 7.2 and start to work on it

I commit some changes in pygrass (r68138), temporal library (r68139)
and GRASS scripts (r68140) that were on my trunk from a bit...

At the moment the compile chain is not working on python3, so you have
to compile using python2 but then you can use GRASS with python3.
The issue with ctypes (#2708 [0]) is still valid, I've "locally" fixed
changing / with // only for the line: ('__val', c_ulong * (1024 / (8 *
sizeof(c_ulong

Of course, GRASS with python3 works only in text-mode, no gui.

All the best

Pietro

[0] https://trac.osgeo.org/grass/ticket/2708
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] split GRASS (lib / cli / modules / wx / qt / web / etc.)

[Pietro]

On Fri, Mar 18, 2016 at 2:16 PM, Moritz Lennert
<mlenn...@club.worldonline.be> wrote:
> On 18/03/16 12:58, Pietro wrote:
> In your opinion is this true at the module level, or mostly for the wxGUI ?

No in my opinion things are quite mixed also in C/python modules.


>> Let's start with a simple example: most of the GRASS modules, mix
>> nicely logic and cli, several of them have a single main function with
>> everything inside. I think could be useful to have a more clear
>> distinction between logic/algorithms and their public interface
>> (cli/gui).
>
>
> Why ? I really like the fact that each module works kind of like a
> high-level function with a defined public interface.

Yes, and I like too! Indeed I don't want to change this. What I would
like to change is to better distinguish this high-level
functionalities from the low level parts. So for instance just opening
randomly a GRASS core module: r.resamp.stats

https://trac.osgeo.org/grass/browser/grass/trunk/raster/r.resamp.stats/main.c

here are defined:
- static const struct menu, that probably could be useful also for
other modules so should go to the grass-lib
- static char *build_method_list(void), that could be generalized to
used also from other modules and should go to the grass-lib
- static int find_method(const char *name), could be also generalized
to be used by other modules and should go to the grass-lib
- static void resamp_unweighted(void), this function could be also
changed to be more general and moved to the grass-lib too
- static void resamp_weighted(void), same as before.

For each of the above function we can build tests to improve the
reliability, verify performance regression and so on.

So far you can access to this functions only from the CLI interface,
if we clear separate this two level then we can access to this using
CLI, but also using C/python/etc. So for instance If I need select an
option from a menu list on a new module I have to reinvent the wheel,
write my own buggy code and as GRASS developers we end up having
duplicate buggy code in each module.

The main function could stay or be rewritten in python, this is not
really relevant because it is just defining the CLI interface and
calling the functions and finally adding the history metadata and set
the color table.



>> If we clearly split these two things the GRASS modules
>> became just an interface to some functions inside the GRASS libraries.
>
>
> Many modules (if not most) are already that: a combination of GRASS library
> function calls in order to achieve the specific goal the module is set out
> for.

yes they are calling GRASS library functions, but they are also adding
functionalities that (imho) should be included to the GRASS library.
Because they could be useful not only for this specific modules but
also for others.


>> So how to split GRASS. It  would be nice to open a dedicate repository
>> (git?) for each of this projects:
>>
>> - grass-lib: provides only C and Python API. This component should be
>> a python citizen, I mean that should be available at the PyPI - the
>> Python Package Index [1] and of course install-able as python package
>> through pip;
>> - grass-cli: provides a shell (with no modules!), also available as a
>> pure python package;
>
>
> I'm not sure I like this extremely pythonic approach to GRASS. I love to use
> Python for scripting GRASS, but in my perception GRASS is far from being a
> Python project.

mmh, ok, so let's add a more layer: grass-lib, grass-py, and then the others..

So grass-lib will contain only C (C++?) code and will be not available
at the PyPi website.
grass-py add the python wrap to grass-lib and add API and go to PyPi.

This is the same approach of GDAL[0], PROJ4[1], mapnik[2] that are all
available as python packages.

I do think that add grass to PyPi can only open new prospective and
use cases reaching a broader group of users and developers.


>> - grass-modules: provides all the GRASS core modules (this could be
>> also a pure python interface calling functions in the C/Python
>> libraries), and could be split in other sub categories (e.g. imagery,
>> temporal, terrain, etc).
>
>
> What is the difference between these modules and the existing ones ? Except
> for your idea to make all module Python modules.

Because things are complicated and the current status quo is not
flexible and/or very limited.
Basically you can create python packages that act as both library and
grass-addons.
So for instance the r.green modules require scipy and numexpr to run,
and the only way available in 2016 is that user have to care about it
installing the missing libraries and even worst if some of our modules
depend on another addons there is no way to handle this.

Other things that are not working with the current set up is 

[GRASS-dev] split GRASS (lib / cli / modules / wx / qt / web / etc.)

[Pietro]

Dear devs,

stimulated by the GSOC idea of Ondřej I would like to revive again this topic.
I know that this has been already discussed in the past, I found this
(CLI1=GUI [0]) but it is focused on users and packagers prospective.
Here I would like to face this point from a developer point of view.
As point out in previous threads GRASS it is already modular, but
(imho) there is duplicate code/functionalities and often things and
levels are mixed.

Let's start with a simple example: most of the GRASS modules, mix
nicely logic and cli, several of them have a single main function with
everything inside. I think could be useful to have a more clear
distinction between logic/algorithms and their public interface
(cli/gui). If we clearly split these two things the GRASS modules
became just an interface to some functions inside the GRASS libraries.

Another example is GRASS GUI that have internally a lot of
functionalities that (imho) should be moved|integrated to a
dedicated|existing python library, because their are independent by
the library (wx|qt|javascript+html5) used to render the final GUI, and
again to me it seems that a lot of things are mixed.

Split these main functionalities in different repository can help
developers, because they can focus/work on a smaller base of code.

So how to split GRASS. It  would be nice to open a dedicate repository
(git?) for each of this projects:

- grass-lib: provides only C and Python API. This component should be
a python citizen, I mean that should be available at the PyPI - the
Python Package Index [1] and of course install-able as python package
through pip;
- grass-cli: provides a shell (with no modules!), also available as a
pure python package;
- grass-modules: provides all the GRASS core modules (this could be
also a pure python interface calling functions in the C/Python
libraries), and could be split in other sub categories (e.g. imagery,
temporal, terrain, etc).
- grass-wx: provides a WxPython/Phoenix interface for GRASS
- (grass-qt: provides a PyQt/PySide interface for GRASS)
- (grass-jupyther: provides a Jupyther interface to GRASS)
- (grass-rest: provides a RESTful API for GRASS)
- (add your idea here... :-D)
- etc.

Each point is characterize by a different use-case and this things are
generally developed by different person with different backgrounds and
needs and to me it make sense to split them.

We could have a greater granularity and a clear focus for each
repository and could help to acquire new developers because it open
new GRASS' development possibilities.
Enlarging the use-case of GRASS. Separate things in dedicated
repository force developers to respect the distinction, and force them
to think where the code should be put/published.
Such subdivision could help has to reduce the total amount of code
making things more general and abstract. It should also help making
independent and well isolated tests.


It should also help the development cycle since we can release things
in a independently way, it requires only to specify in the
requirements.txt file a working tested combinations of python packages
versions.

{{{
numpy>=1.10
grass-lib>=8
grass-cli==8.1
grass-modules>=8
grass-wx=8.1.3
}}}

I think this idea could help mainly developers making things clear and
well organized in different sub-projects.
Opening the possibility to integrate GRASS functionalities to other
open-source projects.

This solution could help also in making things easier also for
packager and users, for example users could install GRASS on all the
system (win/Mac/*nix) running a single command:

{{{
$ pip install --user grass-lib grass-cli grass-modules grass-wx
}}}

What do you think?

All the best.

Pietro

[0] https://lists.osgeo.org/pipermail/grass-dev/2010-November/052661.html
[1] https://pypi.python.org/pypi
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Interest in GSoC 2016

[Pietro]

Hi Ondřej,

On Thu, Mar 17, 2016 at 7:45 PM,  <ondra.l...@seznam.cz> wrote:
> My idea was to generalize GUI Code for Qt-based GUI. Nowadays, Qt (PyQt) is
> increasingly used (look at QGis, for example) and I think it would be better
> to have minimally the roots of GUI in Qt. It’s also much easier to maintain
> with new features. Work with design is also much more user-friendly. It
> seems that in the future, it can be nice shortcut to change something in the
> GUI.

I like the idea, and I think could be strategic for the long run,
especially because it is not clear to me if and when Phoenix (the
wxPython for python3) will be ready (and from my superficial
understanding of the GRASS problem on OS X El Captain Wx it seems part
of the issue), instead both pyQt and pySide support python3 (I think
we should build a GUI compatible with both binding. Consider that
GRASS is already working with Python3 (with except of the ctype stuff)
what it is critical (imho) it is the GUI.

I do like the idea of Vaclav to share a core of functionalities of the
GUI (between Wx, Qt, Jupyhter, etc). This shared functionalities
should go in /lib/python/grass/{gui|somethingelse}. This should allow
us to reduce duplication and the number of code that must be
maintained.

I think that we should also consider to split GRASS and its
functionalities, but I open a separate thread on this.

Best regards

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] python manuals pages

[Pietro]

Dear devs,

I've noticed that we have problem (404) with the documentation pages
for the libpython and pygrass

https://grass.osgeo.org/grass71/manuals/

Any idea?

All the best

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] is it time to release GRASS71?

[Pietro]

Dear devs,

I saw the discussion on https://trac.osgeo.org/grass/ticket/2750#comment:48

Perhaps is time to think to release the next stable release of GRASS
before that the stable release and trunk start to diverge too much...

What do you think?

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] List of os.environ["GRASS_*"]

[Pietro]

Hi Yann,

On Wed, Feb 24, 2016 at 7:27 PM, Yann <yann.che...@gmail.com> wrote:
> I am looking for the os.environ["GRASS_..."] that would lead me safely to
> the /etc/proj/ogr_csv
>
> anybody could tell me where I can access to the list of those GRASS_ env
> variables?

I don't know if you already fix this but you can easily get the list with:

{{{
grass-console> python -c "import os, fnmatch, pprint; pprint.pprint([k
for k in sorted(fnmatch.filter(os.environ.keys(), 'GRASS*'))])"
['GRASS_ADDON_BASE',
 'GRASS_GNUPLOT',
 'GRASS_HTML_BROWSER',
 'GRASS_PAGER',
 'GRASS_PROJSHARE',
 'GRASS_PYTHON',
 'GRASS_VERSION']
}}}

All the best

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] GRASS addons and toolboxes

[Pietro]

Dear Anna and Vaclav,

I've recently implemented a function in r.green.install
("add_rgreen_menu" [0]) that modify the main_menu.xml and
toolboxes.xml files in the ~/.grass7/toolboxes directory adding a menu
for the r.green modules.

Just now, I've seen the flag "t" in g.extension but it is not clear to
me how should I prepare the xml files, should I add the r.green
toolboxes in:
https://trac.osgeo.org/grass/browser/grass-addons/grass7/toolboxes.xml ?

If yes there is a way to test it locally without messing the official:
grass-addons/grass7/toolboxes.xml?

Why generate an unique xml file for all the addons?

Perhaps g.extension could detect in the main root of the grass addon a
toolboxes directory and if the flag "-t" is true proceed installing
and merging the toolboxes in the GRASS GUI. For example for the
r.green module I should only define a directory in
grass-addons/grass7/raster/r.green/toolboxes that contains two xml
files (main_menu.xml and toolboxes.xml).

What do you think?

Have a nice day.

Pietro

[0] 
https://trac.osgeo.org/grass/browser/grass-addons/grass7/raster/r.green/r.green.install/r.green.install.py#L376
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Layer Manager notebook

[Pietro]

Il 15 feb 2016 5:56 PM, "Martin Landa" <landa.mar...@gmail.com> ha scritto:
>
> Hi,
>
> 2016-02-15 17:50 GMT+01:00 Vaclav Petras <wenzesl...@gmail.com>:
> > Layers, Console, Modules, Data, Python
> >
> > are the best.
>
> agreed. +1 for this change. Martin

I like it too, so +1

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] python function to get the path to the GRASS configuration directory

[Pietro]

On Fri, Feb 12, 2016 at 4:01 PM, Anna Petrášová <kratocha...@gmail.com> wrote:
> On Fri, Feb 12, 2016 at 4:43 AM, Pietro <peter.z...@gmail.com> wrote:
>> https://grasswiki.osgeo.org/wiki/GRASS_configuration_files_in_home_directory
>>
>> I wonder if already exist somewhere C/Python API a function that
>> return the right path.
>
>
> In gui/wxpython/core/utils:
> https://trac.osgeo.org/grass/browser/grass/trunk/gui/wxpython/core/utils.py#L795
>
> But we should consider moving it into grass/script/utils.py or similar.

Indeed I think would be useful to clearly distinguish the GUI from
other functionalities.
And would also fix things like:

{{{# gui/wxpython/core/toolboxes.py:53
def GetSettingsPath():
# this is for cases during compilation and it is not possible to import wx
# TODO: if the function would be in the grass Python library there would
# no need to do this
try:
from core.utils import GetSettingsPath as actualGetSettingsPath
return actualGetSettingsPath()
except ImportError:
# expecting that there will be no such path
# (these files are always check for existence here)
return ""
}}}

So how to proceed? Should I move this function to grass.script.utils
and then just import the function definition in
gui/wxpython/core/utils.py?

Should I just fix all the following imports in the GRASS source code:

{{{
gui/wxpython/gui_core/gselect.py:69:from core.utilsimport
GetSettingsPath, GetValidLayerName, ListSortLower
gui/wxpython/web_services/dialogs.py:35:from core.utils   import
GetSettingsPath, _
gui/wxpython/core/settings.py:29:from core.utils import
GetSettingsPath, PathJoin, rgb2str, _
}}}

What do you think?

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] python function to get the path to the GRASS configuration directory

[Pietro]

Dear devs,

The GRASS's configuration directory change from grass version and
depending by OS in use as explained here:

https://grasswiki.osgeo.org/wiki/GRASS_configuration_files_in_home_directory

I wonder if already exist somewhere C/Python API a function that
return the right path.

Any idea?
All the best

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Makefile for an addon using Cython

[Pietro]

Dear Laurent,

> I wrote an extension which include a mix of Python and Cyhon files.
> What would be the preferable way to make it easy to install with g.extension?
> I imagine that it could be easier to distribute the generated C file
> along the .pyx file, as it will not require the user to have Cython
> installed.

Indeed I think that the C file should also tracked.

> Which form would take the Makefile in that case? Is there some
> examples I could follow?

As far as I know this is the first GRASS addon written using Cython,
so no examples are available...
But you can see other Makefile for python or C addons.

Best regards

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] r67535: pygrass: change dirname in set_path()

[Pietro]

Hi Martin,

Sorry for the delay.

On Tue, Jan 12, 2016 at 7:11 PM, Martin Landa <landa.mar...@gmail.com> wrote:
> hm, what means `dirname`? I got impression that it's used for
>
> $GRASS_ADDON_BASE/etc/r.mymodule
>
> is find by set_path('r.mymodule')
>
> $GRASS_ADDON_BASE/etc/r.mymodule/lib1
>
> is find by set_path('r.mymodule', 'lib1'). From this logic I don't
> understand why dirname must be set. Martin

dirname must be set to check if the directory is available when using
the code locally, if dirname is available in the local specified path
then add the path to sys.path, otherwise try to find the correct path
within the GRASS standard paths.
I've added some more documentation to clarify this point.

Let me know if you think is still not clear. :-)

all the best

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] r67535: pygrass: change dirname in set_path()

[Pietro]

Hi Martin,

I saw your recent comment in the set_path function:

{{{
# TODO: why dirname is checked first - the logic should be revised
# TODO  probably also 'path' should be also removed - it's used only by
#   compilation process for addons (see r.green for details)
}}}

'path' it is not used for compilation process for addons as you said.
'path' is used to run the code locally without compilation.

In this way it is possible to just unpack the module and call the
sub-modules locally,  compilation is not needed at all... This it is
particularly useful during the development phase but not only. It works
quite well also to overcome eventually g.extension problems.

Of course dirname must be checked before the default path, other wise it is
too late...

Any comments?

Have a nice day

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] error in pygrass doc

[Pietro]

Hi Luca,


On Fri, Jan 8, 2016 at 10:18 AM, Luca Delucchi <lucadel...@gmail.com> wrote:
>>>> for irow in xrange(elev.rows):
> ...new[irow] = elev[irow] < 144
> ...
> Traceback (most recent call last):
>   File "", line 1, in 
> AttributeError: 'RasterSegment' object has no attribute 'rows'

Thanks to discover this bad documentation page.

instead of elev.rows it is possible to use len(elev) to get the number
of rows, and insetad usig a variable to get the row number use just:

{{{
for irow, row in enumerate(elev):
new[irow] = row < 144
}}}

> maybe there is a better solution, otherwise I can replace the wrong
> code in the doc with this solution.

please replace the example with enumerate.

All the best

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] problem with GridModule

[Pietro]

Hi Anna,

soory for the delay...

On Mon, Nov 16, 2015 at 3:23 PM, Anna Petrášová <kratocha...@gmail.com> wrote:
> On Mon, Nov 16, 2015 at 8:28 AM, Markus Neteler <nete...@osgeo.org> wrote:
>> On Tue, Nov 10, 2015 at 7:58 PM, Anna Petrášová <kratocha...@gmail.com>
>> wrote:
>> > Maybe some bug in the pygrass raster library?
>>
>> Looks like a race condition?
>
>
> I don't understand why would that happen, the patching done with pygrass
> raster library is not running in parallel.

Yes I have to investigate more on this to understand what it is not working...


> A workaround I currently use is to patch the rasters using r.patch, which is
> in the case of viewsheds OK (the overlaps are the same), however in other
> cases such as interpolation, the overlaps are different and r.patch won't
> patch the rasters as desired, that's why the GridModule is using pygrass
> raster library I assume.

I've modified r.tile to split and merge using an optional parameter
for the overlap (perhaps I should create just a new module r.untile
instead of adding a new flag in r.tile...).
The approach it is much faster than the actual implementation of
r.patch because minimize reading disk operations since we are not
merging general raster maps with undefined bboxes but  well known
tiles...
Unfortunately the implementation still experimental and has some
problems. I will not able to work on this before next year... maybe I
can put the modified version of r.tile in the sandbox repo.

Let me know.

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] testsuite: some broken tests in pygrass and python script lib

[Pietro]

Hi Markus,


On Tue, Nov 3, 2015 at 6:18 PM, Markus Neteler <nete...@osgeo.org> wrote:
> Hi devs:
>
> any suggestions how to fix those here properly?
>
> http://fatra.cnr.ncsu.edu/grassgistests/reports_for_date-2015-11-02-08-00/report_for_piemonte_utm32_wgs84_grass7_stdmaps/lib/python/script/test_start_command_functions_nc/index.html
>   File "lib/python/script/testsuite/test_start_command_functions_nc.py",
> line 32, in test_prefixed_underscore
> msg="Underscore as prefix was not accepted")
> AssertionError: Underscore as prefix was not accepted

This should be fix with the new dataset that contains the raster map:
"elevation"



> http://fatra.cnr.ncsu.edu/grassgistests/reports_for_date-2015-11-02-08-00/report_for_piemonte_utm32_wgs84_grass7_stdmaps/lib/python/pygrass/test_doctests/index.html
> File "etc/python/grass/pygrass/utils.py", line 230, in
> grass.pygrass.utils.get_raster_for_points
>AttributeError: 'VectorTopo' object has no attribute 'table'

Looking at the complete stderr:
http://fatra.cnr.ncsu.edu/grassgistests/reports_for_date-2015-11-02-08-00/report_for_piemonte_utm32_wgs84_grass7_stdmaps/lib/python/pygrass/test_doctests/stderr.html

it seems like the map test_vect_2 is not found:

{{{
Failed example:
fire.open(mode='r')
Exception raised:
Traceback (most recent call last):
  File "/usr/lib/python2.7/doctest.py", line 1315, in __run
compileflags, 1) in test.globs
  File "",
line 1, in 
fire.open(mode='r')
  File "etc/python/grass/pygrass/vector/abstract.py", line 328, in open
raise OpenError("Map <%s> not found." % self._name)
OpenError: Map  not found.
}}}



> http://fatra.cnr.ncsu.edu/grassgistests/reports_for_date-2015-11-02-08-00/report_for_piemonte_utm32_wgs84_grass7_stdmaps/lib/python/pygrass/modules/grid/test_doctests/index.html
> AssertionError: Failed doctest test for
> grass.pygrass.modules.shortcuts.MetaModule
> File "etc/python/grass/pygrass/modules/shortcuts.py", line 28, in
> grass.pygrass.modules.shortcuts.MetaModule
> Failed example:
> g_list.outputs.stdout # doctest: +ELLIPSIS
> Expected:
> '...basin...soils...'
> Got:
> 'gmted2010\ngmted2010.shade\nlandsat.blue\nlandsat.green\nlandsat.red\nlst_2010_01_average_piemonte\nlst_2010_02_average_piemonte\nlst_2010_03_average_piemonte\nlst_2010_04_average_piemonte\nlst_2010_05_average_piemonte\nlst_2010_06_average_piemonte\nlst_2010_07_average_piemonte\nlst_2010_08_average_piemonte\nlst_2010_09_average_piemonte\nlst_2010_10_average_piemonte\nlst_2010_11_average_piemonte\nlst_2010_12_average_piemonte\norbassano_orto.blue\norbassano_orto.green\norbassano_orto.red\norbassano_orto.rgb\nsrtm\nsrtm_grezzo\nuso_suolo2007\n'
>

Here we can change the test to found some other raster maps instead of
basins and soils, since both maps seem not available in the Piedmont
dataset.

Best regards

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Python equivalent of piping results from one function to another

[Pietro]

Dear Paulo,


On Sun, Oct 18, 2015 at 8:03 PM, Paulo van Breugel
<p.vanbreu...@gmail.com> wrote:
> What would be the python equivalent of:
>
> r.category mymap | r.category mymap rules=-

Let's split the problem in two step.

First capture the output of a module (the python equivalent of bash pipe):

{{{
>>> from subprocess import PIPE  # from the standard library import PIPE costant
>>> from grass.pygrass.modules import Module  # import the Module class from 
>>> pygrass
>>> rcat = Module('r.category', 'landuse', stdout_=PIPE)  # use the spacial 
>>> parameter stdout_
>>> rcat.outputs.stdout   # the stdout is an attribute of the instanced class
'1\tdeveloped\n2\tagriculture\n3\therbaceous\n4\tshrubland\n5\tforest\n6\twater\n7\tsediment\n'
>>> print(rcat.outputs.stdout)
1   developed
2   agriculture
3   herbaceous
4   shrubland
5   forest
6   water
7   sediment
}}}

Second provide a string as stdinput of a module:

{{{
>>> Module('r.category', 'mymap', rules='-', 
>>> stdin_='1\tdeveloped\n2\tagriculture\n3\therbaceous\n4\tshrubland\n5\tforest\n6\twater\n7\tsediment\n')
Module('r.category')
}}}

or to do both things in one line:

{{{
Module('r.category', 'mymap', rules='-', stdin_=Module('r.category',
'landuse', stdout_=PIPE).outputs.stdout)
}}}

You can use a shourtcut to have a cleaner syntax close to bash:

{{{
>>> from grass.pygrass.modules.shortcuts import raster as r
>>> r.category('mymap', rules='-', stdin_=r.category('landuse', 
>>> stdout_=PIPE).outputs.stdout)
Module('r.category')
}}}

There is an example in the documentation using r.colors but it is not
rendered corectly [0]

Have fun!

Pietro

[0] 
https://grass.osgeo.org/grass71/manuals/libpython/pygrass.modules.interface.html#pygrass.modules.interface.module.Module
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Test report: different message breaks the test

[Pietro]

Il 16 ott 2015 5:38 PM, "Vaclav Petras" <wenzesl...@gmail.com> ha scritto:
>
> Hi,
>
> it seems that some change in error message broke the test. I'm not sure
what should the the general approach in these cases. The test should be
changed with the message that's clear but the question is how precise we
want the message test to be. Perhaps we want to test there is some message
or that there is a string such as "SQL" or name of the column in it. Any
opinions?

Yes, I can remove the test on the error message, probably I was just too
pendant...
The important thing is that an exception is raised!

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] pygrass vector changes and backport planning (7.0.x? or 7.1.x?)

[Pietro]

Dear Markus and Sören, dear devs,

Thanks to Sören's work the pygrass vector library has been improved a
lot in trunk during the last weeks. So far they are not signed to be
backported [0].
I would like to understand when do you think these changes can be
ported to the stable branch and in which stable (7.0.X or 7.1.X)
relaease?
Which one will be more appropriate?
When do you think the next stable will be? :-)

I would avoid to let the two branches differ too much, and I think we
should try to make trunk as much close to the release candidate as
possible if the changes seem to work.

I'm thinking when should I start updating all the r.green modules.

If you think that these changes should be ported to 7.1.X then probaly
I need a grass71 directory in grass_addons...


Let me know.

Pietro

[0] https://trac.osgeo.org/grass/wiki/Grass7Planning#a7.0.2tobebackported
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] wx.metadata installation issues & pygrass set_path() question

[Pietro]

Hi Martin,

On Sun, Oct 4, 2015 at 2:30 PM, Martin Landa <landa.mar...@gmail.com> wrote:
> Hi all,
>
> when trying to fix all compilation and installation issues I replaced
> get_lib_path() by set_path() [1]. These functions are not used in
> GRASS core codebase. Their usage is not clear to me.

The idea is to be able to execute the same code in both scenarios:
compiled/installed code, and executing the python code locally (e.g. python
my.grass.module.py parm0=option0 param1=option1 -e --o).
Since GRASS is changing the paths before and after compilations...

Surelly these functions can be implemented in a different way and/or
improved.

> Fn get_lib_path() says "Return the path of the libname contained in
> the module". So I would expect that it checkd that `modname` is a
> directory and contains `libname` Python module. But 'libname' is just
> used to check if it's a directory [2].

Yes, I think you are right, the function should check if libname is a
directory or a python file.

> Only one function calls get_lib_path() - set_path() [3].
>
> Can anybody here to clarify their usage?

They are split for code-granularity reasons and potential re-usability,
Every time that you need to set/modify the python path you need the
get_lib_path and modify the python path, but i think that sometime could be
also useful to just check the path to the library without changing the
python path. Therefore the idea here is:

get_lib_path => Return a string with the path to the library, if found else
return None
set_path() => Set the python path to make the library importable or raise
an exception

What do you think?

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Write array to (point) attribute table

[Pietro]

On Thu, Oct 1, 2015 at 10:01 AM, Michel Wortmann
 wrote:
> Hi Paulo,
> I had the same problem some time ago. I am using pandas for a lot of things
> and discovered the quickest way to append a column to a vector table is
> using pandas' .to_sql dataframe method. Here is my full function for it:

To add a ne column on an existing vector map in pygrass: write:

{{{
if 'column_name' in vect.table.columns:
vect.table.columns.drop('column_name')
vect.table.columns.add('column_name', 'column_type')
}}}
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Write array to (point) attribute table

[Pietro]

On Wed, Sep 30, 2015 at 9:51 AM, Paulo van Breugel
<p.vanbreu...@gmail.com> wrote:
>
>
> On Wed, Sep 30, 2015 at 2:02 AM, Anna Petrášová <kratocha...@gmail.com>
> wrote:
>>
>>
>>
>> On Tue, Sep 29, 2015 at 6:09 PM, Paulo van Breugel
>> <p.vanbreu...@gmail.com> wrote:
>>>
>>> This must be a very basic question, but I can't find an easy/direct way
>>> to do this. In python, if I have an array with values with a length equal to
>>> the number of rows in an attribute table of a (point) vector layer, how can
>>> I write those values to a new column in that attribute table. I can of
>>> course first create the column, but than how to update that column with the
>>> values in the array?
>>
>>
>> it should be pretty easy to do with pygrass, unfortunately there is no
>> example on assigning attributes in the official documentation [1],  but it
>> should be pretty easy, something like that (not tested):
>>
>> with VectorTopo('myvector', mode='w') as vectormap:
>> for feature in vectormap:
>> feature.attrs['mycolumn'] = value
>>
>>
> Thanks, but that seems to write the vector back without attribute table

You have to save the changes in the database out from your cycle, with:

vectormap.table.conn.commit()

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] Vect_cidx_find_all unexpected result and boundary right/left

[Pietro]

Dear devs,

I'm playing with vector features and categories. I'm creating a vector
map for testing, almost a copy and paste from Soeren's function, the
main differenceis that I added different categories for each geometry
feature.

Then If I'm looking for feature with the category number == 8, and I ask for:
- a libvect.GV_AREA, the Vect_cidx_find_all function return an Ilist
that contains 3, but 3 is a point with cat == 3 and not 8!
- a libvect.GV_CENTROID, everything it is fine

Why if I'm using libvect.GV_AREA Vect_cidx_find_all return 3 and not
an empty ilist?
What is it the meaning of the returned value?

Then If I play with the Boundary instance both sides (left and right)
are 0, Instead I'm expecting the right side shouldn't be empty.

Do you have any ideas?

Pietro

Bellow is reported a sample of code to reproduce it.


{{{
import grass.lib.vector as libvect

from grass.pygrass.vector import VectorTopo
from grass.pygrass.vector.geometry import Point, Line, Centroid, Boundary
from grass.pygrass.vector.basic import Ilist


def create_test_vector_map2(map_name="test_vector2", overwrite=False):
"""This functions creates a vector map layer with points, lines, boundaries,
   centroids, areas, isles and attributes for testing purposes

   This should be used in doc and unit tests to create location/mapset
   independent vector map layer. This map includes 3 points, 3 lines,
   11 boundaries and 4 centroids. The attribute table contains cat, name
   and value columns.

param map_name: The vector map name that should be used



  P1 P2 P3
   6   *  *  *
   5
   4___ ___ ___   L1 L2 L3
Y  3   |A1___ *|  *|  *|   |  |  |
   2   | |A2*| |   |   |   |  |  |
   1   | |___| |A3 |A4 |   |  |  |
   0   |___|___|___|   |  |  |
  -1
-1 0 1 2 3 4 5 6 7 8 9 10 12 14
   X
"""
cols = [(u'cat', 'INTEGER PRIMARY KEY'),
(u'name','varchar(50)'),
(u'value', 'double precision')]
with VectorTopo(map_name, mode='w', tab_name=map_name,
tab_cols=cols, overwrite=overwrite) as vect:

# Write 3 points
vect.write(Point(10, 6), cat=1, attrs=("point", 1))
vect.write(Point(12, 6), cat=2)
vect.write(Point(14, 6), cat=3)
# Write 3 lines
vect.write(Line([(10, 4), (10, 2), (10,0)]), cat=4, attrs=("line", 2))
vect.write(Line([(12, 4), (12, 2), (12,0)]), cat=5)
vect.write(Line([(14, 4), (14, 2), (14,0)]), cat=6)
# boundaries 1 - 4
vect.write(Boundary(points=[(0, 0), (0,4)]))
vect.write(Boundary(points=[(0, 4), (4,4)]))
vect.write(Boundary(points=[(4, 4), (4,0)]))
vect.write(Boundary(points=[(4, 0), (0,0)]))
# 5. boundary (Isle)
vect.write(Boundary(points=[(1, 1), (1,3), (3, 3), (3,1), (1,1)]))
# boundaries 6 - 8
vect.write(Boundary(points=[(4, 4), (6,4)]))
vect.write(Boundary(points=[(6, 4), (6,0)]))
vect.write(Boundary(points=[(6, 0), (4,0)]))
# boundaries 9 - 11
vect.write(Boundary(points=[(6, 4), (8,4)]))
vect.write(Boundary(points=[(8, 4), (8,0)]))
vect.write(Boundary(points=[(8, 0), (6,0)]))
# Centroids, all have the same cat and attribute
vect.write(Centroid(x=3.5, y=3.5), cat=7, attrs=("centroid7", 7))
vect.write(Centroid(x=2.5, y=2.5), cat=8, attrs=("centroid8", 8))
vect.write(Centroid(x=5.5, y=3.5), cat=9, attrs=("centroid9", 9))
vect.write(Centroid(x=7.5, y=3.5), cat=10, attrs=("centroid10", 10))

vect.table.conn.commit()

vect.organization = "Thuenen Institut"
vect.person = "Soeren Gebbert"
vect.title = "Test dataset"
vect.comment = "This is a comment"
vect.close()


TMPVECT = 'test_vector2'
CAT = 8

create_test_vector_map2(TMPVECT, overwrite=True)

with VectorTopo(TMPVECT, mode='r') as tv:
features = [f for f in tv]
print(features[-3].cat == CAT, features[-3], features[-3].id)

ilist = Ilist()
libvect.Vect_cidx_find_all(tv.c_mapinfo, 1, libvect.GV_AREA, CAT,
ilist.c_ilist)
print('area', libvect.GV_AREA, [(i, tv[i].cat) for i in ilist])

libvect.Vect_cidx_find_all(tv.c_mapinfo, 1, libvect.GV_CENTROID,
CAT, ilist.c_ilist)
print('centroid', libvect.GV_CENTROID, [(i, tv[i].cat) for i in ilist])

b0 = features[6]
print(b0, b0.left_area_id, b0.right_area_id)
print()
}}}
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #2708: Run GRASS with Python3

[Pietro]

Hi Massimo,

On Wed, Sep 23, 2015 at 4:28 AM, massimo di stefano
<massimodisa...@gmail.com> wrote:
> what’s the state of port, is there any temporary fork of the project or 
> should I apply the several patches on this ticket ?

So far all the grass/script library it seems to work (and it is
already in trunk), therefore you don't need to apply any patch.
The ctypes are still not working, but I have not the deep
understanding of this code, therefore it is difficult for me to port
them. Therefore all the libraries and modules that are refering to the
ctypes are not working on python3 (pygrass, temporal, etc).

Many python modules are not working out of the box for python 3, In
August I've changed them and I'm using them for testing since for most
of them the testsuite is missing.
May be it is time to sync my local changes and trunk.

The GUI will not start on python3, the wxpython became Phoenix:

- http://wiki.wxpython.org/ProjectPhoenix
- http://wxpython.org/Phoenix/docs/html/main.html

Therefore if you want the GUI I don't think that python3 could be an
option (in short time at least).

> At the moment my build fails on most of core modules  my default python is V3.

As I said, I've started fixing most of them... if interested I could
sync my local changes to trunk next week.
However you can compile with python2 and execute with python3.


> Is there a way to specify in the configure which python has to be used for to 
> build grass?

I personally use a virtualenv.
If I remeber correctly Glynn suggest to create a link in the grass
source root of your python binary.

IMHO use GRASS with python3, only for testing and contributing to the
port... not for your daily work.

All the best.

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] grass-addons and nested module libraries

[Pietro]

Dear Matej,

On Fri, Aug 21, 2015 at 11:31 AM, Matej Krejci matejkre...@gmail.com wrote:
 Dear Pietro, devs,

 [...]

 I have exactly the same problem(described bellow) with installation
 wx.metadata from addons. Have you figured out this issue?

No, not yet... I've also noticed that the link to other grass-addons
it is working properly in the GUI, but not on the manual web page, see
for example:

https://grass.osgeo.org/grass70/manuals/addons/r.modis.html

in SEE ALSO, you can find two links at: r.modis.import and
r.modis.dowload both the links are broken and should be change:

from: https://grass.osgeo.org/grass70/manuals/r.modis.*.html
to: https://grass.osgeo.org/grass70/manuals/addons/r.modis.*.html

But if we change the line:

https://trac.osgeo.org/grass/browser/grass-addons/grass7/raster/r.modis/r.modis.html#L113

from: a href=r.modis.import.htmlr.modis.import/a,
to: a href=addons/r.modis.import.htmlr.modis.import/a,

I suspect it will break the manual page in the GUI.

Unfortunately my knowledge on the GRASS make chain is very poor and I
don't know from where should I start to fix these problems.

Any hints?

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Add support for Python3

[Pietro]

Hi Glynn,

On Thu, Jul 30, 2015 at 4:34 PM, Glynn Clements wrote:
 Pietro wrote:
 ok, I've now synchronized my local changes in script/core to trunk:

 In parser() in lib/python/script/core.py:

 s = p.communicate()[0].decode()

 the .decode() shouldn't be there; the output should be kept as bytes.
 This may require changing _parse_opts().

 Likewise for parse_key_val() in utils.py.

 More generally, nothing in grass.script should be using unicode
 values. Arguments which happen to be unicode values can be encode()d
 to bytes, but arguments which are bytes must be used as-is (no
 bytes-unicode-bytes round trips) and data obtained from files, pipes,
 environment variables, OS functions, etc should be returned as bytes.

ok, I try to avoid to use unicode in script with the following changes:

https://trac.osgeo.org/grass/changeset/65804

I've stopped to go on with the syncronization of my local changes and
trunk since I noticed that from the 27th of July the tests are not
running anymore:

http://fatra.cnr.ncsu.edu/grassgistests/summary_report/nc/index.html

I don't understand why, but I have the suspect that the recent changes
done in lib/init/grass.py and gunittest:

https://trac.osgeo.org/grass/changeset/65780
https://trac.osgeo.org/grass/changeset/65782

have broken something.

I will not go further on the development of python2/python3 until this
issue will be fix.

All the best.

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Add support for Python3

[Pietro]

Hi all,

On Mon, Jul 27, 2015 at 8:47 AM, Luca Delucchi lucadel...@gmail.com wrote:
 On 24 July 2015 at 12:42, Pietro peter.z...@gmail.com wrote:
 Are you ok with this schedule?

 yes, let us know when you commit the changes to test them

ok, I've now synchronized my local changes in script/core to trunk:
- https://trac.osgeo.org/grass/changeset/65786
- https://trac.osgeo.org/grass/changeset/65787
- https://trac.osgeo.org/grass/changeset/65788
- https://trac.osgeo.org/grass/changeset/65789
- https://trac.osgeo.org/grass/changeset/65790
- https://trac.osgeo.org/grass/changeset/65791
- https://trac.osgeo.org/grass/changeset/65792
- https://trac.osgeo.org/grass/changeset/65793
- https://trac.osgeo.org/grass/changeset/65794
- https://trac.osgeo.org/grass/changeset/65795
- https://trac.osgeo.org/grass/changeset/65796
- https://trac.osgeo.org/grass/changeset/65797

I tested using the current testsuite, please tell me if you find bugs
or better write a test that highlights the problem and I will try to
fix it as soon as I can.

All the best

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Add support for Python3

[Pietro]

Hi Luca,

On Mon, Jul 27, 2015 at 8:47 AM, Luca Delucchi lucadel...@gmail.com wrote:
 On 24 July 2015 at 12:42, Pietro peter.z...@gmail.com wrote:
 Are you ok with this schedule?

 yes, let us know when you commit the changes to test them

ok, I've commited the changes in lib/init/grass.py and in the
lib/python/gunittest/*

Let me know if you have troubles.

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Add support for Python3

[Pietro]

Hi Markus,

On Mon, Jul 27, 2015 at 4:24 PM, Markus Neteler nete...@osgeo.org wrote:
 On Mon, Jul 27, 2015 at 1:59 PM, Pietro peter.z...@gmail.com wrote:
 ...
 ok, I've commited the changes in lib/init/grass.py and in the
 lib/python/gunittest/*

 Please post to the list how to switch to Python3 in case that multiple
 versions are available.
 Like this more testers may join.

You are righ, here some instruction to test and find bugs...

You must have python3 installed on your machine, from a terminal

{{{
python3 /usr/bin/grass71 -getxt ~/docdat/gis/nc_spm_08_grass7/py34 -c
}}}

then start a python interpreter and execute some code
[Note1: this example should not work on your machine since I didn't
commit this part so far;
 Note2: GRASS with python3 it is working only in text mode no GUI!]:

{{{
$ python3
 from grass.script import core as gcore
 gcore.gisenv()
{'GISDBASE': '/home/pietro/docdat/gis', 'GUI': 'gtext', 'MAPSET':
'py34', 'LOCATION_NAME': 'nc_spm_08_grass7'}
}}}

To check if the result in python2 is the same then open a new terminal
and execute

{{{
$ python2 /usr/bin/grass71 -getxt ~/docdat/gis/nc_spm_08_grass7/py27 -c
}}}

{{{
$ python2
 from grass.script import core as gcore
 gcore.gisenv()
{u'LOCATION_NAME': unc_spm_08_grass7', u'GISDBASE':
u'/home/pietro/docdat/gis', u'MAPSET': u'py27', u'GUI': u'gtext'}
}}}

You can also start executing Vaclav's unittest library, go to a directory:

{{{
$ cd lib/python/script
$ python3 -m grass.gunittest.main --location nc_spm_08_grass7 --location-type nc
test_start_command_functions_nc from . failed
test_start_command_functions from . failed
test_doctests from . failed

Executed 3 test files in 0:00:09.962775.
From them 0 files (0%) were successful and 3 files (100%) failed.

$ python2 -m grass.gunittest.main --location nc_spm_08_grass7 --location-type nc

Executed 3 test files in 0:00:10.358305.
From them 3 files (100%) were successful and 0 files (0%) failed.
}}}

or execute a specific test file with:

{{{
$ python3 testsuite/test_start_command_functions_nc.py
}}}

So far the test in the script directory are still failing I'm working
to fix them... :-)

All the best

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] Add support for Python3

[Pietro]

Dear devs,

Next week I would like to start the merge between my local changes and
trunk to support Python3.
As reported in the ticket: https://trac.osgeo.org/grass/ticket/2708

I've tested the changes with python2.6, python2.7 and python3.4.
So far I've not found regressions in python2.X.

The status quo for python3 is:
- the binding of the C GRASS functions is not working;
- several python modules (~90) are not readable/usable, I've already
started to port them (~20)

My idea for the porting is to gradually introduce changes per steps:
- Monday 27, change only the grass-init and the gunittest;
- If not new problems connected to previous changes raise, Wendesday
29 I commit the grass/script stuff
- If not new problems connected to previous changes raise, I will
start updating the GRASS modules.

In the meanwhile I try to get the ctypes working on PY3.

Are you ok with this schedule?
Other better options?

Let me know

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Travis CI

[Pietro]

Dear Ivan,

On Tue, Jul 21, 2015 at 11:52 AM, Ivan Minčík ivan.min...@gmail.com wrote:
 Hi all,
 we have just integrated Travis Continuous Integration system [1] to the
 GRASS source code. Every commit is now build in Travis [2] using gcc and
 clang. In case the build fails, error message should go to GRASS-dev list.

Do you think that could be possible also to execute the python tests using:
- python 2.6
- python 2.7
- python 3.4 or 3.5-dev

I'm looking here:

http://docs.travis-ci.com/user/languages/python/

But I'm not sure how can we integrate the C stuff with the python part?
Any ideas?

All the best!

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] table with the parser standard options

[Pietro]

Dear Luca and Moritz,

On Wed, Jul 1, 2015 at 11:58 AM, Moritz Lennert
mlenn...@club.worldonline.be wrote:
 May I just suggest the following small modification

ok, added.

 I believe we should add the generated table somewhere in the manual docs.
 Should I add this python script to grass_addons/tools?


 Wouldn't it be better in tools/ directly in the source code ?

ok, committed in r65534.

Some examples:

{{{
$ python2 tools/parser_standard_options.py -t
lib/gis/parser_standard_options.c -f csv -o
parser_standard_options.csv

$ tail parser_standard_options.csv
G_OPT_STR3DS_INPUT;;Name of the input space time raster3d
dataset;;old,str3ds,str3ds;input;nameYES;TYPE_STRING
G_OPT_STR3DS_INPUTS;;Name of the input space time raster3d
datasets;;old,str3ds,str3ds;inputs;name;;YES;;YES;TYPE_STRING
G_OPT_STR3DS_OUTPUT;;Name of the output space time raster3d
dataset;;new,str3ds,str3ds;output;nameYES;TYPE_STRING
G_OPT_STDS_TYPE;strds;Type of the input space time
dataset;;;type;name;;;strds,stvds,str3ds;NO;TYPE_STRING
G_OPT_MAP_INPUT;;Name of the input map;;old,map,map;map;nameYES;TYPE_STRING
G_OPT_MAP_INPUTS;;Name of the input
maps;;old,map,map;maps;name;;YES;;YES;TYPE_STRING
G_OPT_MAP_TYPE;raster;Type of the input
map;;;type;name;;;raster,vector,raster_3d;NO;TYPE_STRING
G_OPT_T_TYPE;absolute;The temporal type of the space time
dataset;;;temporaltype;name;;;absolute,relative;NO;TYPE_STRING
G_OPT_T_WHERE;;Example: start_time  '2001-01-01
12:30:00';;;where;sql_query;WHERE conditions of SQL statement without
'where' keyword used in the temporal GIS framework;;;NO;TYPE_STRING
G_OPT_T_SAMPLE;start;The method to be used for sampling the input
dataset;;;sampling;name;;YES;start,during,overlap,contain,equal,follows,precedes;NO;TYPE_STRING

$ python2 tools/parser_standard_options.py -t
lib/gis/parser_standard_options.c -f html -o
parser_standard_options.html

$ head parser_standard_options.html
table border=1
  tr
tdoption/td
tdanswer/td
tddescription/td
tddescriptions/td
tdgisprompt/td
tdkey/td
tdkey_desc/td
tdlabel/td

$ python2 tools/parser_standard_options.py -t
lib/gis/parser_standard_options.c -f csv -o
parser_standard_options.csv -s Flg

$  tail parser_standard_options.csv
description;key
G_FLG_V_TABLE;Do not create attribute table;'t'
G_FLG_V_TOPO;Do not build topology;'b'
}}}

Best regards

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] Question on C API and GRASS raster/region

[Pietro]

Dear devs,

I'm trying to clarify the behavior of the C API of grass concerning
the interaction between raster and region, since I'm more confortable
with Python, I've just open an ipython  interpreter inside a GRASS
session, the mapset is the North Carolina.

I've imported some libraries and define two function to make the process easier:

{{{
 %paste
from __future__ import print_function
import ctypes
import fnmatch
from pprint import pprint

import grass.lib.gis as libgis
import grass.lib.raster as libraster

def attributes(obj):
return sorted(fnmatch.filter(dir(obj), '[!_]*'))

def struct2items(struct):
struct = struct.contents if hasattr(struct, 'contents') else struct
return [(attr, getattr(struct, attr)) for attr in attributes(struct)]
## -- End pasted text --
}}}

Let's start I create an empty region structure with:

{{{
 c_region = ctypes.pointer(libgis.Cell_head())
 pprint(struct2items(c_region))
[('bottom', 0.0),
 ('cols', 0),
 ('cols3', 0),
 ('compressed', 0),
 ('depths', 0),
 ('east', 0.0),
 ('ew_res', 0.0),
 ('ew_res3', 0.0),
 ('format', 0),
 ('north', 0.0),
 ('ns_res', 0.0),
 ('ns_res3', 0.0),
 ('proj', 0),
 ('rows', 0),
 ('rows3', 0),
 ('south', 0.0),
 ('tb_res', 0.0),
 ('top', 0.0),
 ('west', 0.0),
 ('zone', 0)]
}}}

get the current region:

{{{
 libgis.G_get_set_window(c_region)
 pprint(struct2items(c_region))
[('bottom', 0.0),
 ('cols', 1500),
 ('cols3', 1500),
 ('compressed', -1),
 ('depths', 1),
 ('east', 645000.0),
 ('ew_res', 10.0),
 ('ew_res3', 10.0),
 ('format', 0),
 ('north', 228500.0),
 ('ns_res', 10.0),
 ('ns_res3', 10.0),
 ('proj', 99),
 ('rows', 1350),
 ('rows3', 1350),
 ('south', 215000.0),
 ('tb_res', 1.0),
 ('top', 1.0),
 ('west', 63.0),
 ('zone', 0)]
 print('rows:', libraster.Rast_window_rows(),
...   'cols:', libraster.Rast_window_cols())
rows: 1350 cols: 1500
}}}

Now I change some region attributes and I adjust/update the struct values with

{{{
 print('Change region extent and set as current')
Change region extent and set as current
 # change region attributes
 c_region.contents.west = 644530.0
 c_region.contents.south = 228070.0
 # adjust accordingly the number of rows and columns
 libgis.G_adjust_Cell_head(c_region, False, False)
 libgis.G_set_window(c_region)
 pprint(struct2items(c_region))
[('bottom', 0.0),
 ('cols', 47),
 ('cols3', 1500),
 ('compressed', -1),
 ('depths', 1),
 ('east', 645000.0),
 ('ew_res', 10.0),
 ('ew_res3', 10.0),
 ('format', 0),
 ('north', 228500.0),
 ('ns_res', 10.0),
 ('ns_res3', 10.0),
 ('proj', 99),
 ('rows', 43),
 ('rows3', 1350),
 ('south', 228070.0),
 ('tb_res', 1.0),
 ('top', 1.0),
 ('west', 644530.0),
 ('zone', 0)]
 print('rows:', libraster.Rast_window_rows(),
...   'cols:', libraster.Rast_window_cols())
rows: 1350 cols: 1500
}}}

As you can see the region struct has now 43 rows and 47 columns, but
still the Rast_window_rows and the Rast_window_cols return still the
'old' one.
The Rast_window_[rows|cols] functions are described in the
documentation () as Number of [rows|cols] in active window. with
active here we mean current, aren't we?
http://grass.osgeo.org/programming7/raster_2window_8c.html#ac2ce884f9f92cf60e28850c705ba8175

Therefore why they are not returning 43 and 47?

Creating a new struct and asking for the current region provide me the
expected result.

{{{
 c_region_new = ctypes.pointer(libgis.Cell_head())
 libgis.G_get_set_window(c_region_new)
 pprint(struct2items(c_region_new))
[('bottom', 0.0),
 ('cols', 47),
 ('cols3', 1500),
 ('compressed', -1),
 ('depths', 1),
 ('east', 645000.0),
 ('ew_res', 10.0),
 ('ew_res3', 10.0),
 ('format', 0),
 ('north', 228500.0),
 ('ns_res', 10.0),
 ('ns_res3', 10.0),
 ('proj', 99),
 ('rows', 43),
 ('rows3', 1350),
 ('south', 228070.0),
 ('tb_res', 1.0),
 ('top', 1.0),
 ('west', 644530.0),
 ('zone', 0)]
}}}

Probably I misunderstood something... Someone can help to understand
this point better?
Thank you very much

Pietro


test_region.py
Description: Binary data
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] table with the parser standard options

[Pietro]

Dear devs,

I was not able to find a table in the GRASS manual pages with a
summary of all the default options as define in
parser_standard_options.c (lib/gis/parser_standard_options.c).
Therefore I wrote a python command line tool to extract them from the
source and print/write to a file in CSV/HTML format.
Probably I should use regexp but I'm not good with them...

The command line tool is attached.

Some usage example are:

$ python2 options.py -t lib/gis/parser_standard_options.c -f csv -o
stadard_options.csv

The command line should be able also to download the source file
directly from a link but it works only partially.

I believe we should add the generated table somewhere in the manual docs.
Should I add this python script to grass_addons/tools?

What do you think?

Pietro


options.py
Description: Binary data
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Documentation server build log full of syntax errors in pygrass

[Pietro]

On Thu, Jun 11, 2015 at 1:34 PM, Markus Neteler nete...@osgeo.org wrote:
 Unless there are other implications, a backport of r65438 would be good.

Right, the change should be save therefore backported in r65439.

Have a nice day.

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Documentation server build log full of syntax errors in pygrass

[Pietro]

Hi Vaclav,

Il 11/giu/2015 04:33 AM, Vaclav Petras wenzesl...@gmail.com ha scritto:
 It seems to me that dictionary comprehension is not supported in Python
2.6 but I was not able to confirm this in the Python documentation.

Yes, python2.6 does not support the list comprehension syntax, I will fix
this today...

Can we drop the support for python 2.6?

Best regards

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Backport gunittest and tests to 7.0?

[Pietro]

Hi Vaclav,

What do you think, is it time to move the gunittest suite from trunk to 70?

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] replace G_debug C function with a python one only for the Module class (pygrass)

[Pietro]

Dear all,

I would like to remove the line 680:

{{{
get_msgr().debug(1, self.get_bash())
}}}

from the run method of the Module class.

I would like to remove it because the get_msgr is using ctypes (grass.lib.gis),
this it means that all the LD_LIBRARY_PATH must be set correctly.

Set the LD_LIBRARY_PATH could be a problem, for example if you try to use
the class from QGIS, because it require to launch QGIS with something like;

{{{
#!/usr/bin/sh
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$GISBASE/lib \
PATH=$PATH:$GISBASE/bin:$GISBASE/scripts:$HOME/.grass7/addons/bin:$HOME/.grass7/addons/scripts
\
PYTHONPATH=$PYTHONPATH:$GISBASE/etc/python:$GISBASE/gui/wxpython \
GRASS_ADDON_BASE=$HOME/.grass7/addons qgis
}}}

On the other hand, I understand that we need the debug. therefore I
was thinking to provide the same functionalities but avoiding to call
directly the G_debug functions, G_debug is define as follow:

{{{
int G_debug(int level, const char *msg, ...)
{
char *filen;
va_list ap;
FILE *fd;

G_init_debug();

if (grass_debug_level = level) {
va_start(ap, msg);

filen = getenv(GRASS_DEBUG_FILE);
if (filen != NULL) {
   fd = fopen(filen, a);
   if (!fd) {
G_warning(_(Cannot open debug file '%s'), filen);
return 0;
   }
}
else {
   fd = stderr;
}

fprintf(fd, D%d/%d: , level, grass_debug_level);
vfprintf(fd, msg, ap);
fprintf(fd, \n);
fflush(fd);

if (filen != NULL)
   fclose(fd);

va_end(ap);
}

return 1;
}
}}}

and I'm defining a function (probably is still too simple) such as:

{{{
from __future__ import print_function
import os
import sys


def get_env():
gisrc = os.environ.get('GISRC')
if gisrc is None:
raise RunError('You are not in a GRASS session')
with open(gisrc, mode='r') as grc:
env = {k.strip(): v.strip() for k, v in [row.split(':') for
row in grc if row]}
return env


def get_debug_level():
debug = get_env().get('DEBUG')
return int(debug) if debug else 0


def G_debug(level, *msg):
debug_level = get_debug_level()
if debug_level = level:
dfile = os.environ.get(GRASS_DEBUG_FILE)
fd = sys.stderr if dfile is None else open(dfile, mode='a')
print(D%d/%d:  % (level, debug_level), *msg, end='\n', file=fd)
}}}

Concerning the performance of course the python version is slower but
only 6 times:

{{{
%timeit G_debug_python(5, 'provami...')
The slowest run took 9.75 times longer than the fastest. This could
mean that an intermediate result is being cached
10 loops, best of 3: 13.8 µs per loop

In [25]: %timeit G_debug_grass(5, 'provami...')
The slowest run took 32.00 times longer than the fastest. This could
mean that an intermediate result is being cached
10 loops, best of 3: 2.35 µs per loop
}}}

But in this way the ctypes are not required by the pygrass.modules
classes and functions.

What do you think?
Other ideas and/or solutions?

Best regards

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] inconsistency between G_OPT_V_INPUT and G_OPT_R_INPUT

[Pietro]

Hi Martin,

On Wed, May 13, 2015 at 11:08 AM, Martin Landa landa.mar...@gmail.com wrote:
 Hi,

 2015-05-13 8:03 GMT+02:00 Pietro peter.z...@gmail.com:
 #% description: Name of vector map with interested segments of rivers

 - label

Therefore I should use label instead of description, something like:

{{{
#!/usr/bin/env python
# -- coding: utf-8 --
#%module
#% description: Test
#% keywords: test
#%end
#%option G_OPT_R_INPUT
#% key: discharge
#% label: Name of river discharge [m3/s]
#% required: yes
#%end
#%option G_OPT_V_INPUT
#% key: river
#% label: Name of vector map with interested segments of rivers
#% required: yes
#%end
from __future__ import print_function
import sys

# import grass libraries
from grass.script import core as gcore


def main(opts, flgs):
print(opts, flgs)

if __name__ == __main__:
options, flags = gcore.parser()
sys.exit(main(options, flags))
}}}

The GUI is now fine and the command output looks:

{{{$ r.example --h

Description:
 Test

Keywords:
 test

Usage:
 r.example discharge=name river=name [--help] [--verbose] [--quiet]
   [--ui]

Flags:
 --h   Print usage summary
 --v   Verbose module output
 --q   Quiet module output
 --ui  Force launching GUI dialog

Parameters:
  discharge   Name of river discharge [m3/s]
   Name of input raster map
  river   Name of vector map with interested segments of rivers
   Or data source for direct OGR access
}}}


 As you can see the description of the parameter add the default
 description set in G_OPT_V_INPUT and in a new line the description
 added manually in the definition of the module parameters. Instead the
 G_OPT_R_INPUT shows only the description set manually.

 right, G_OPT_V_INPUT has defined both label and description [1]. This
 the source of inconsistency. There are more standard options which
 defines both label and description.

Do you think this inconsistency should be fix, or just reported
somewhere on a Wiki page?

All the best.

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] inconsistency between G_OPT_V_INPUT and G_OPT_R_INPUT

[Pietro]

Dear all,

I found an inconsistency of behaviors between G_OPT_R_INPUT and
G_OPT_V_INPUT, here the test code:

{{{
#!/usr/bin/env python
# -- coding: utf-8 --
#%module
#% description: Test
#% keywords: test
#%end
#%option G_OPT_R_INPUT
#% key: discharge
#% description: Name of river discharge [m3/s]
#% required: yes
#%end
#%option G_OPT_V_INPUT
#% key: river
#% key_desc: name
#% description: Name of vector map with interested segments of rivers
#% required: yes
#%end
from __future__ import print_function
import sys

# import grass libraries
from grass.script import core as gcore


def main(opts, flgs):
print(opts, flgs)

if __name__ == __main__:
options, flags = gcore.parser()
sys.exit(main(options, flags))
}}}

here is the output of the manual page:

{{{
python r.example.py --help

Description:
 Test

Keywords:
 test

Usage:
 r.example.py discharge=name river=name [--help] [--verbose] [--quiet]
   [--ui]

Flags:
 --h   Print usage summary
 --v   Verbose module output
 --q   Quiet module output
 --ui  Force launching GUI dialog

Parameters:
  discharge   Name of river discharge [m3/s]
  river   Name of input vector map
   Name of vector map with interested segments of rivers
}}}

As you can see the description of the parameter add the default
description set in G_OPT_V_INPUT and in a new line the description
added manually in the definition of the module parameters. Instead the
G_OPT_R_INPUT shows only the description set manually.

The behavior concerning the GUI is worst, because it shows only the
default description string, see attached figure.

What do you think?

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] GRASS7 GUI error on Fedora 22 (beta)

[Pietro]

Dear All,

I'm trying to make a virtual machine with GRASS7 on Fedora 22 (only
grass64 is available so far), I didn't have any particular problem
configuring with:

{{{
./configure \
  --with-cxx \
  --with-gdal=/usr/bin/gdal-config \
  --with-proj --with-proj-share=/usr/share/proj \
  --with-sqlite \
  --with-nls \
  --with-wxwidgets=/usr/bin/wx-config \
  --with-fftw \
  --with-python=/usr/bin/python-config \
  --with-freetype --with-freetype-includes=/usr/include/freetype2 \
  --enable-largefile \
  --without-odbc
}}}

and then compiling...

but when I start GRASS I got this Error:

{{{
$ grass71
Fatal Error: Mismatch between the program and library build versions detected.
The library used 3.0 (wchar_t,compiler with C++ ABI 1008,wx
containers,compatible with 2.8),
and wxPython used 3.0 (wchar_t,compiler with C++ ABI 1002,wx
containers,compatible with 2.8).
ERROR: Invalid return code from GUI startup script.
Please advise GRASS developers of this error.
Exiting...
}}}

Any ideas?

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] GRASS7 GUI error on Fedora 22 (beta)

[Pietro]

On Mon, May 4, 2015 at 11:04 PM, Vaclav Petras wenzesl...@gmail.com wrote:
 Nothing in particular, but you can try to `import wx` in Python command line
 inside and outside of GRASS session to see what happens.

ok, I've tried with:

{{{
$ python -c import wx
Fatal Error: Mismatch between the program and library build versions detected.
The library used 3.0 (wchar_t,compiler with C++ ABI 1008,wx
containers,compatible with 2.8),
and wxPython used 3.0 (wchar_t,compiler with C++ ABI 1002,wx
containers,compatible with 2.8).
Cancelled (core dump created)
}}}

Therefore it seems not a GRASS problem... but more a library/distribution one.

Do you know where this problem should be point out?

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] pygrass: ipdb dependency

[Pietro]

Hi Martin,

On Sun, Apr 19, 2015 at 10:22 AM, Martin Landa landa.mar...@gmail.com wrote:
 lib/python/pygrass/vector/geometry.py:import ipdb; 
 ipdb.set_trace()

 on other places such line is commented out. Is such dependency really needed?

no, it is just for debugging.
In particular the one that is present on geometry.py is acalled only
when an exception from the SQL is raised:

{{{
try:
cur = self.table.execute(sql.SELECT_WHERE.format(cols=key,
 tname=self.table.name,
 condition=self.cond))
except:
import ipdb; ipdb.set_trace()
}}}

Because I was trying to catch a bug, therefore the ipdb is imported
only if the sql execution fails, I can easily remove it.
I don't think it is a big issue.

Best regards

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] pygrass: region do not align to raster

[Pietro]

Hi Martin,

On Sun, Apr 19, 2015 at 3:32 PM, Martin Landa landa.mar...@gmail.com wrote:
 it's seems to me that Region.align() is not working in pygrass as expected.

 region = Region()
 print region.rows- is 977
 region.align(rast)
 print region.rows- should be 11682, but prints 977

You are right, all the methods: align and zoom are wrong, because
their are calling the g.region module, that it is execute in an
external process and therefore does not affect the current one,
therefore they should be:
- removed or
- re-implemented to provide the same functionality remaining in the
same process or
- move the align and zoom functions from the g.region module to the
library link them with ctypes and call them from the method.

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] pygrass - stdout2dict

[Pietro]

On Tue, Apr 7, 2015 at 1:05 PM, Martin Landa landa.mar...@gmail.com wrote:
 hm, is there any way how to parse command output using pygrass without
 need to define your own function?

Why not use: parse_key_val(s, sep='=', dflt=None, val_type=None, vsep=None)

define in grass.script.utils?
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] pygrass - stdout2dict

[Pietro]

Hi Martin,



On Tue, Apr 7, 2015 at 11:10 AM, Martin Landa landa.mar...@gmail.com wrote:
 pygrass manual uses stdout2dict() [1] which doesn't seems to be
 available.

 [1] http://grass.osgeo.org/grass70/manuals/libpython/pygrass_modules.html

You are right the function has not been defined... probably I can
simply update the docstring adding the function, something like:

{{{
def stdout2dict(stdout):
return dict([kv.split('=') for kv in stdout.strip().split('\n')])
}}}

or with lambda

{{{
stdout2dict = lambda x: dict([i.split('=') for i in x.strip().split('\n')])
}}}

Best regards

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] pygrass - stdout2dict

[Pietro]

On Tue, Apr 7, 2015 at 1:54 PM, Martin Landa landa.mar...@gmail.com wrote:
 you are right, are you planning to update pygrass manual? Martin

done in r65015

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] g.parser and link a relation between parameters

[Pietro]

Hi devs,

may be I'm missing something stupid, but if I want to write a GRASS
modules that takes two different vectors and allows for each vetor to
select a column, something like:

{{{
#%module
#% description: Test
#% keyword: vector
#%end
#%option G_OPT_V_INPUT
#% key: vect0
#%end
#%option G_OPT_V_FIELD
#% key: vect0_layer
#% required: no
#%end
#%option G_OPT_DB_COLUMN
#% key: vect0_column_cost
#%end
#%option G_OPT_DB_COLUMN
#% key: vect0_column_maintenance
#%end
#%option G_OPT_V_INPUT
#% key: vect1
#%end
#%option G_OPT_V_FIELD
#% key: vect1_layer
#% required: no
#%end
#%option G_OPT_DB_COLUMN
#% key: vect0_column_other
#%end
#%option G_OPT_V_OUTPUT
#%end
from pprint import pprint
import grass.script.core as gcore


def main(options, flags):
pprint(options)
pprint(flags)

if __name__ == __main__:
main(*gcore.parser())
}}}

How can I say to WxGui that vect0_column_cost should display a
combobox with the available columns of the table linked with a layer
specify in vect0_layer of the vector map with name specify in vect0?

For the GRASS addons and GUI that we are developing we decide to use
the same root word: vect0 and vect1 to link the different
parameters to the vector, there are any better way to solve this? May
be something that it is working natively in GRASS?

Thank you for your help.

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] QGIS GRASS Plugin Upgrade Crowdfunding

[Pietro]

Hi Vaclav,

On Tue, Mar 24, 2015 at 3:54 PM, Vaclav Petras wenzesl...@gmail.com wrote:
 On Tue, Mar 24, 2015 at 2:00 AM, Pietro peter.z...@gmail.com wrote:
 together with Francesco Geri, last week we started to write a first
 draft that generate the GUI for the GRASS module dynamically (based on
 the xml generated with: --interface-description), so far it is based
 on grass7 but should work also on grass6 (not tested, should be enough
 to backport the pygrass.modules.Module class).

 Hi Pietro and Francesco,

 this sounds good. Are you speaking about QGIS GUI (in Python) generated from
 --interface-description? Do you think it would be possible to write it in a
 way which would be GUI framework independent so that it is usable in both
 GRASS and QGIS? The GUIs for modules in GRASS are quite good now but the
 code should be rewritten and this would be a great way how to get this new
 implementation.

yes, actually what I wrote it is completely independent from QGIS, it
is compatible with pyqt/pyside and with python2/python3,
It is just a proof of concept. Now Francesco it is working to better
integrate this draft with QGIS stuff.

The idea was heavily based on formlayout code:

https://code.google.com/p/formlayout/

Basically each parameter is a GUI field that it is add to the main
window, therefore based on the parameter type the field could be:

if raster/vector = a combo box with the rasters/vectors available
if integer = a spinbox
if string/double = a text field
if file = a file dialog

based on the guisection parameter all the options are organized in tabs.

The code is under 500 lines of code.

I guess should be possible to follow the same approach using the wxpython.
I think that with a bit more effort should be possible to make it
general enough to make it independent from the GUI framework (Qt/Wx)
that we want to use.

I have not link to share at this stage because we are still working on it.
But I guess we can share the code as soon as we have something that it
is able to cover the basic needs.


Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] QGIS GRASS Plugin Upgrade Crowdfunding

[Pietro]

Hi Radim,

together with Francesco Geri, last week we started to write a first
draft that generate the GUI for the GRASS module dynamically (based on
the xml generated with: --interface-description), so far it is based
on grass7 but should work also on grass6 (not tested, should be enough
to backport the pygrass.modules.Module class).

Best regards

Pietro

On Mon, Mar 23, 2015 at 7:56 PM, Radim Blazek radim.bla...@gmail.com wrote:
 Hi all,

 I have finally launched the crowdfunding campaign to support the GRASS
 plugin upgrade. Briefly, it covers upgrade to GRASS 7, browser
 integration, drag-and-drop import and new vector editing. All the
 details are available here:

 http://www.gissula.eu/qgis-grass-plugin-crowdfunding/

 Please propagate this info to all relevant channels, national mailing lists 
 etc.

 Radim
 ___
 grass-dev mailing list
 grass-dev@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] QGIS GRASS Plugin Upgrade Crowdfunding

[Pietro]

Hi Radim,

On Tue, Mar 24, 2015 at 5:11 PM, Radim Blazek radim.bla...@gmail.com wrote:
 On Tue, Mar 24, 2015 at 7:00 AM, Pietro peter.z...@gmail.com wrote:
 together with Francesco Geri, last week we started to write a first
 draft that generate the GUI for the GRASS module dynamically (based on
 the xml generated with: --interface-description), so far it is based
 on grass7 but should work also on grass6 (not tested, should be enough
 to backport the pygrass.modules.Module class).

 The GUI for GRASS module in the plugin is also generated based on
 --interface-description, qgm files are only used to define a subset of
 module's options or to override option attributes (like label) if
 necessary. So it is already implemented, but always combined with t,
 skipping qgm, it maybe used to generate generic UI for any module.
 Using qgm gives more power, for example table field combo options
 based on current layer selected as input, setting input type from
 input layer, selecting feature categories in map canvas and so on.

 You may be interested in
 https://github.com/qgis/QGIS/blob/master/src/plugins/grass/qgsgrassmodule.cpp

 Do you see a possibility how we could join the effort?

Yes I know that that there was this tool already available, but we
want to build something that use only the  --interface-description, to
avoid to synchronize the qgm file for each change that we made on the
grass-addon.

I would love to join the effort, the main problem here is that I don't
know cpp, therefore I can not play actively on this side.

However I think that some extra fields in the xml description are
missing and should be add to easily go on the direction of the
dynamically generated GUI.

But I will open a new thread.

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Compute mahalanobis distance using Scipy

[Pietro]

Hi Paulo,

On Mon, Mar 16, 2015 at 9:23 PM, Paulo van Breugel
p.vanbreu...@gmail.com wrote:
 Paulo van Breugel wrote:
 Alternatively, it might indicate an issue in the PyGrass library; that
 code is relatively new.


 OK, hopefully the PyGrass gurus can have a look at it.

Ofcourse I will look into it, but since I need two/three weeks, may be
could a good idea to open a ticket on track, just to not forget this
problem.
Thank you for reporting this.

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] vector group by based on attribute table

[Pietro]

On Wed, Mar 4, 2015 at 4:29 PM, Moritz Lennert
mlenn...@club.worldonline.be wrote:
 On 04/03/15 15:14, Pietro wrote:

 Dear all,

 I'm looking for a GRASS module that unify the geometries and the
 values, If I have a configuration like:

 geo features  |  cat   |   id | A  | B
 geofeature1 = cat1, id1, a1, b1
 geofeature2 = cat2, id1, a2, b2
 geofeature3 = cat3, id2, a3, b3
 geofeature4 = cat4, id2, a4, b4
 geofeature5 = cat5, id2, a5, b5

 I would like to have something like:

 SELECT id, mean(A) as meanA, mean(B) as meanB FROM table GROUP BY id;

 geo features   |  cat   | meanA  | meanB
 geo1, geo2   = id1, mean(a1, a2), mean(b1, b2)
 geo3, geo4, geo5 = id2, mean(a3, a4, a5), mean(b3, b4, b5)

 There is already something to do this simple operation?
 Do you have any suggestions?


 Does this mean you want to fusion the geofeatures (thus imlying that they
 are adjacent) or do you want to group them in a sort of typology ?

No, I don't want the fusion of the geometry features, I like the idea
of different/distinct geometry features with the same category.

 If the second, I could imagine a combination of v.reclass / v.category to
 create a new/second layer and then your SQL query to populate the table of
 that layer.

Ok, I will look into it, thanks for the suggestion.

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] vector group by based on attribute table

[Pietro]

Dear all,

I'm looking for a GRASS module that unify the geometries and the
values, If I have a configuration like:

geo features  |  cat   |   id | A  | B
geofeature1 = cat1, id1, a1, b1
geofeature2 = cat2, id1, a2, b2
geofeature3 = cat3, id2, a3, b3
geofeature4 = cat4, id2, a4, b4
geofeature5 = cat5, id2, a5, b5

I would like to have something like:

SELECT id, mean(A) as meanA, mean(B) as meanB FROM table GROUP BY id;

geo features   |  cat   | meanA  | meanB
geo1, geo2   = id1, mean(a1, a2), mean(b1, b2)
geo3, geo4, geo5 = id2, mean(a3, a4, a5), mean(b3, b4, b5)

There is already something to do this simple operation?
Do you have any suggestions?

Thank you

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Non-linear (multiple) regression in GRASS?

[Pietro]

On Tue, Feb 24, 2015 at 5:30 PM, Paulo van Breugel
p.vanbreu...@gmail.com wrote:
 I wonder if it is possible to utilize the scikit-learn library for rasters
 as well.. The library offers a broad range of regression models, as well as
 clustering and classification functions.

yes, I've done it.

I've applied severl regression models for more than 300 rasters (as
features) and it was fast but the region was quite small.

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Vertical tabs for module's wxGUI interface?

[Pietro]

Hi Nikos,

On Sun, Feb 22, 2015 at 3:55 PM, Nikos Alexandris
n...@nikosalexandris.net wrote:
 where is this screenshot
 http://ncsu-osgeorel.github.io/grass-temporal-workshop/pictures/tregister.png
 coming from?  The vertical arrangement of tabs is awesome!

You can set the vertical tabs in preference, it is my favorite
arrangement, I thin should be the GRASS default option (imho).

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Non-linear (multiple) regression in GRASS?

[Pietro]

Hi Nikos,

On Sat, Feb 21, 2015 at 2:08 PM, Nikos Alexandris
n...@nikosalexandris.net wrote:
 and advanced users.  Has anyone perforned, or has plans to implement,
 non-linear (multi-) regression in GRASS  (not through R, of course)?

I did a pixel by pixel regression using most of the code of the
v.class.ml module, but the approach still need to act manually.
So it is not ready...

During next spring I would like to make a bit of order on the
v.class.ml module, maybe change also the name, to allow supervised,
unsupervised and regression tasks through scikit-learn library. If you
have some ideas on how would you like to organized these
module/modules, like: which inputs are required, the outputs, the
optional options and steps, please share them and we can start to
discuss on how implement what.

Best regards.

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] osgeo4w package for GRASS 7

[Pietro]

On Sat, Feb 21, 2015 at 2:22 PM, Helmut Kudrnovsky hel...@web.de wrote:
 Martin Landa wrote
 Personally I would vote for b). Any opinion? It's strategic decision,
 so please feel free to give me feedback ASAP...

 I tend to b), as we may retire the grass6x-lines with grass7.0.1. thus only
 grass, grass-dev and grass-trunk-dev will remain.

I personally prefer option b too.

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Compute mahalanobis distance using Scipy

[Pietro]

Hi,

On Tue, Feb 17, 2015 at 10:44 AM, Paulo van Breugel
p.vanbreu...@gmail.com wrote:
 Can raster_input be multiple rasters?

yes, I've define a new split_rasters function, that should does what
you want, note, the code is not tested, just to provide a rough idea.

{{{
import numpy as np

from grass.pygrass.gis.region import Region
from grass.pygrass.raster import RasterRow
from grass.pygrass.raster.buffer import Buffer
from grass.pygrass.utils import split_in_chunk


def mahalanobis_distances(X):
m = np.mean(X, axis=0)
L = np.cov(m)
cX = X - m[np.newaxis, :]
tmp = solve_triangular(L, cX.T, lower=True).T
tmp **= 2

def split_rasters(rasts, nrows=100):
generators = [split_in_chunk(rst, nrows) for rst in rasts]
for chunk in zip(*generators):
yield chunk


def tiled_function(raster_inputs, raster_output, func,
   out_mtype='DCELL', overwrite=False, nrows=100,
   **kwargs):
reg = Region()
rasts = [RasterRow(rst) for rst in raster_inputs]
# open input raster maps
for rst in rasts:
rst.open('r')
# open the output map
with RasterRow(raster_output, mode='w', mtype=out_mtype) as rout:
buf = Buffer((reg.cols, ), mtype=out_mtype)  # take a row buffer
for chunk in split_rasters(rasts, nrows):
ichunk = np.array(chunk)
for row in func(ichunk, **kwargs):
buf[:] = row[:]
rout.put_row(buf)


tiled_function('elevation', 'distance', mahalanobis_distances,
   overwrite=True)
}}}
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS-SVN] r64649 - in grass-addons/grass7/raster/r.green: . r.green.biomassfor/r.green.biomassfor.economic r.green.biomassfor/r.green.biomassfor.impact r.green.biomassfor/r.green.bio

[Pietro]

Hi Martin,

On Mon, Feb 16, 2015 at 3:20 PM, Martin Landa landa.mar...@gmail.com wrote:
 Hi,

 2015-02-16 14:36 GMT+01:00  svn_gr...@osgeo.org:
 Author: zarch
 Date: 2015-02-16 05:36:51 -0800 (Mon, 16 Feb 2015)
 New Revision: 64649

 Modified:

 grass-addons/grass7/raster/r.green/r.green.biomassfor/r.green.biomassfor.economic/Makefile

 why so much nested? The right naming should be
 grass-addons/grass7/raster/r.green/r.green.biomassfor/Makefile
 ...

because thee are several modules going on (but not yet published, they
are under testing) for other different sources (r.green.hydro,
r.green.wind, r.green.solar), that are following the same structures
and have shared libraries between them.

I'm open to suggestions, here it is a complete list of modules:

{{{
r.gree/
├── AUTHORS.txt
├── biblio
│   └── 1-s2.0-S01676105137X-main.pdf
├── libgreen
│   ├── checkparameter.py
│   ├── checkparameter.pyc
│   ├── __init__.py
│   ├── __init__.pyc
│   ├── Makefile
│   ├── utils.py
│   └── utils.pyc
├── LICENSE.txt
├── Makefile
├── README.txt
├── r.green.biomassfor
│   ├── Makefile
│   ├── r.green.biomassfor.economic
│   ├── r.green.biomassfor.html
│   ├── r.green.biomassfor.impact
│   ├── r.green.biomassfor.legal
│   ├── r.green.biomassfor.recommended
│   ├── r.green.biomassfor.technical
│   └── r.green.biomassfor.theoretical
├── r.green.html
├── r.green.hydro
│   ├── libhydro
│   ├── Makefile
│   ├── r.green.hydro.closest
│   ├── r.green.hydro.delplants
│   ├── r.green.hydro.discharge
│   ├── r.green.hydro.economic
│   ├── r.green.hydro.html
│   ├── r.green.hydro.legal
│   ├── r.green.hydro.optimal
│   ├── r.green.hydro.recommended
│   ├── r.green.hydro.structure
│   ├── r.green.hydro.technical
│   └── r.green.hydro.theoretical
├── r.green.solar
│   ├── Makefile
│   ├── r.green.solar.economic
│   ├── r.green.solar.html
│   ├── r.green.solar.theoretical
│   ├── r.green.solar.recommended
│   └── r.green.solar.technical
├── r.green.TEV
│   └── r.green.TEV.py
└── r.green.wind
├── Makefile
├── r.green.wind.economic
├── r.green.wind.html
├── r.green.wind.theoretical
├── r.green.wind.recommended
└── r.green.wind.technical

34 directories, 22 files
}}}

Any better ideas?
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Compute mahalanobis distance using Scipy

[Pietro]

On Mon, Feb 16, 2015 at 10:50 PM, Paulo van Breugel
p.vanbreu...@gmail.com wrote:
 On Sat, Feb 14, 2015 at 7:29 PM, Vaclav Petras wenzesl...@gmail.com wrote:
 On Sat, Feb 14, 2015 at 11:47 AM, Paulo van Breugel
 p.vanbreu...@gmail.com wrote:
 For a quick solution, what about using r.tile to split the input data in
 tiles and compute the mahalanobis distance per tile.

 See PyGRASS GridModule class which will do tiling (and a lot of other
 things) for you.

 http://grass.osgeo.org/grass70/manuals/libpython/pygrass.modules.grid.html

 The Grid Module does indeed what I want, except that I want to run a python
 function (the mahalanobis distance computation) on the tiles, while the
 GridModule requires a raster GRASS r.* command. Any way around that? Writing
 an intermediate addon/function that I can then call with the GridModule?

mhh the GridModule it is heavily using the Module interface, therefore
I think you should provide a module and then use the GridModule.

Therefore I wrote a function that do exactly what you are looking for:

{{{
import numpy as np

from grass.pygrass.gis.region import Region
from grass.pygrass.raster import RasterRow
from grass.pygrass.raster.buffer import Buffer
from grass.pygrass.utils import split_in_chunk


def mahalanobis_distance(array, param):
# compute mahalanobis distance, here just a dummy computation
dist = array / param
return dist


def tiled_function(raster_input, raster_output, func,
   out_mtype='DCELL', overwrite=False, nrows=100,
   **kwargs):
reg = Region()
with RasterRow(raster_input, mode='r') as rinp:
with RasterRow(raster_output, mode='w', mtype=out_mtype) as rout:
buf = Buffer((reg.cols, ), mtype=out_mtype)  # take a row buffer
for chunk in split_in_chunk(rinp, nrows):
ichunk = np.array(chunk)
for row in func(ichunk, **kwargs):
buf[:] = row[:]
rout.put_row(buf)


tiled_function('elevation', 'distance', mahalanobis_distance,
   overwrite=True, param=2.)
}}}

I'm not sure where should I put this function, maybe in pygrass.raster
or in pygrass.utils (but the last one it is a bit generic...).

All the best

Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev