Re: [GRASS-dev] r.mapcalc - expression line to long
Glynn Clements wrote Helmut Kudrnovsky wrote: by running the python script, a r.mapcalc-error pops up that the expression line is to long. Can you provide the exact message? There shouldn't be any practial limitation on the length of an expression in r60662 and later. [Previously, there may have been OS-imposed limitations on the length of a command line, but r60662 changed grass.script.mapcalc() to pass the expression via stdin rather than the command line.] -- Glynn Clements Step 3: Calculation of elevation percentile PCTL3 ... ... Die Befehlszeile ist zu lang. ERROR: An error occurred while running r.mapcalc translation: The command line is too long. it's this script http://trac.osgeo.org/grass/browser/grass-addons/grass7/raster/r.valley.bottom/r.valley.bottom.py - best regards Helmut -- View this message in context: http://osgeo-org.1560.x6.nabble.com/r-mapcalc-expression-line-to-long-tp5149281p5149370.html Sent from the Grass - Dev mailing list archive at Nabble.com. ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] r.mapcalc - expression line to long
Helmut Kudrnovsky wrote Glynn Clements wrote Helmut Kudrnovsky wrote: There shouldn't be any practial limitation on the length of an expression in r60662 and later. [Previously, there may have been OS-imposed limitations on the length of a command line, but r60662 changed grass.script.mapcalc() to pass the expression via stdin rather than the command line.] -- Glynn Clements now tested with GRASS version: 7.0.0svn GRASS SVN Revision: 61115 Build Date: 2014-07-02 Build Platform: i686-pc-mingw32 GDAL/OGR: 1.11.0 PROJ.4: 4.8.0 GEOS: 3.4.2 SQLite: 3.7.17 Python: 2.7.4 wxPython: 2.8.12.1 Platform: Windows-Vista-6.0.6002-SP2 (OSGeo4W) could it be that r60662 is only in trunk and not in 7.0.0svn? - best regards Helmut -- View this message in context: http://osgeo-org.1560.x6.nabble.com/r-mapcalc-expression-line-to-long-tp5149281p5149371.html Sent from the Grass - Dev mailing list archive at Nabble.com. ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] r.mapcalc - expression line to long
There shouldn't be any practial limitation on the length of an expression in r60662 and later. there are similar steps in the script [1] http://trac.osgeo.org/grass/browser/grass-addons/grass7/raster/r.valley.bottom/r.valley.bottom.py#L205 [2] http://trac.osgeo.org/grass/browser/grass-addons/grass7/raster/r.valley.bottom/r.valley.bottom.py#L350 [1] works, [2] fails with the mentioned error. thanks. - best regards Helmut -- View this message in context: http://osgeo-org.1560.x6.nabble.com/r-mapcalc-expression-line-to-long-tp5149281p5149375.html Sent from the Grass - Dev mailing list archive at Nabble.com. ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] using rand(x,y) in r.mapcalc (grass7)
On 03-07-14 03:43, Vaclav Petras wrote: On Wed, Jul 2, 2014 at 8:15 PM, Glynn Clements gl...@gclements.plus.com mailto:gl...@gclements.plus.com wrote: Shouldn't the seed not be generated on e.g, OS time, which would ensure that each run would give a different result? No. The reason is to provide reproducibility. Anyone running the same command with the same data should obtain the same result. It is certainly be good to be able to reproduce commands. However, I think in most (statistical) software the default / expected behaviour is to have a new automatically generated seed at each run. In R for example, if you have to explicitly specify the seed using the function set.seed(). I would think therefore what most users will expect a similar behaviour in GRASS. It would certainly be my personal preference to have the option to set the seed explicitly if you want reproducibility, but have it generated automatically otherwise. But that is just a personal preference. Does the reproducibility go behind one operating system, compiler or library? I don't think that the first random number is specified by the C language standard. If the results would be really reproducible it would be good for testing framework but I'm afraid that they are not (with my limited knowledge about the topic). If you want a different result each time, set GRASS_RND_SEED to a different value each time, e.g. GRASS_RND_SEED=`date +%N` r.mapcalc a = rand(0,100) [%N is the nanoseconds portion of the current time; this is a GNU extension.] Perhaps this can be explained like this in the manual page? A far better option would be to provide this as a normal parameter so it can be set from the gui interface or command line like any other variable. I've heard that this is not enough on powerful computers/clusters, that you have to use also PID because nanoseconds might be the same (I think I rememberer that it was nanoseconds not seconds). On a related note, it would be nice to be able to set the seed (I think there has been such a request before, but not sure about the answer at that time). GRASS_RND_SEED was the answer. I think there should be some possibility of randomization (auto-setting of seed) build-in the modules providing random(ized) results. Perhaps a flag which would turn it on. It can be also an option which would behave like GRASS_RND_SEED but would have one special value for auto-generating the seed. (GRASS_RND_SEED if present would override this option.) With the default value of the option we should ask a question what is actually the expected behavior of the module giving random results. Yes, that would be great. As for the default value, see my earlier argument. This would provide a nicer interface in Python, standard interface in command line, and possibility to set it in the GUI (which means possibility to set it for users which don't use command line.) Moreover, it would provide all users with the way of setting the random seen in the manner which we consider the best according to our knowledge. Agree. The way to set the seed now may not be understood by everybody and with all the work going into streamlining the GUI, this kind of fairly important options should also be available through the GUI Vaclav ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] using rand(x,y) in r.mapcalc (grass7)
Just a quick additional question, how to set this GRASS_RND_SEED from within a python script (I want to add the option to set the seed with a seed parameter in my script, as suggested in the previous email). On Thu, Jul 3, 2014 at 8:55 AM, Paulo van Breugel p.vanbreu...@gmail.com wrote: On 03-07-14 03:43, Vaclav Petras wrote: On Wed, Jul 2, 2014 at 8:15 PM, Glynn Clements gl...@gclements.plus.com wrote: Shouldn't the seed not be generated on e.g, OS time, which would ensure that each run would give a different result? No. The reason is to provide reproducibility. Anyone running the same command with the same data should obtain the same result. It is certainly be good to be able to reproduce commands. However, I think in most (statistical) software the default / expected behaviour is to have a new automatically generated seed at each run. In R for example, if you have to explicitly specify the seed using the function set.seed(). I would think therefore what most users will expect a similar behaviour in GRASS. It would certainly be my personal preference to have the option to set the seed explicitly if you want reproducibility, but have it generated automatically otherwise. But that is just a personal preference. Does the reproducibility go behind one operating system, compiler or library? I don't think that the first random number is specified by the C language standard. If the results would be really reproducible it would be good for testing framework but I'm afraid that they are not (with my limited knowledge about the topic). If you want a different result each time, set GRASS_RND_SEED to a different value each time, e.g. GRASS_RND_SEED=`date +%N` r.mapcalc a = rand(0,100) [%N is the nanoseconds portion of the current time; this is a GNU extension.] Perhaps this can be explained like this in the manual page? A far better option would be to provide this as a normal parameter so it can be set from the gui interface or command line like any other variable. I've heard that this is not enough on powerful computers/clusters, that you have to use also PID because nanoseconds might be the same (I think I rememberer that it was nanoseconds not seconds). On a related note, it would be nice to be able to set the seed (I think there has been such a request before, but not sure about the answer at that time). GRASS_RND_SEED was the answer. I think there should be some possibility of randomization (auto-setting of seed) build-in the modules providing random(ized) results. Perhaps a flag which would turn it on. It can be also an option which would behave like GRASS_RND_SEED but would have one special value for auto-generating the seed. (GRASS_RND_SEED if present would override this option.) With the default value of the option we should ask a question what is actually the expected behavior of the module giving random results. Yes, that would be great. As for the default value, see my earlier argument. This would provide a nicer interface in Python, standard interface in command line, and possibility to set it in the GUI (which means possibility to set it for users which don't use command line.) Moreover, it would provide all users with the way of setting the random seen in the manner which we consider the best according to our knowledge. Agree. The way to set the seed now may not be understood by everybody and with all the work going into streamlining the GUI, this kind of fairly important options should also be available through the GUI Vaclav ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] [SoC] gsoc2014 grass metadata week 6
Hi Matej, Thanks for your reply. Metadata is a current topic I my organization at the moment, so your work comes exactly at the right time for me. Here in NINA people are using many different kinds of GIS software and we are considering introducing GeoNetwork as a central metadata storage solution, which makes metadata management to some extend independent from the GIS software used. Among public and environmental organisations, GeoNetwork is quite popular for metadata management (amongst others Mapping authorities from Netherlands, Switzerland, and the Nordic countries are using it, see also: http://geonetwork-opensource.org/gallery/gallery.html). I have no real personal experience with GeoNetwor (yet), but from what I read in the documentation I could imagine two ways how GRASS metadata could make their way into such a central metadata catalogue: 1) XML import (+ this option is in place when GRASS metadata is written to XML, - import requires manual labor) 2) Metadata harvesting (GeoNetwork can “harvest” metadata from other CSWs, so maybe pycsw already provides automatic an interface in that direction(?). That would be great!) When it comes to consistent metadata, I guess e.g. lists with values for metadata fields will in practice have to be more or less dynamic (as new staff member are being employed, new topics for spatial analysis arise (requiring new keywords), new types of data being developed and so on). Not sure if manipulating the templates is convenient enough for average users (seems to be mainly suitable for users with some degree of programming experience). However, good to know that providing a custom set of values is already possible through the templates. If you start working on a DB for metadata related content, feel free to let me know if I can support you by any means (yet I will be on leave for almost the rest of the year, which means I can spend only a limited amount of time). However, I had a second look on GeoNetwork in this regard, and GeoNetwork is (naturally) using a database for metadata management (one option here is PostgreSQL). If it would be possible to just connect to that same DB (e.g. for fetching and storing key words, data on contact persons…), that would be really, really great, as this ensures that metadata produced in GRASS are 100% compliant to metadata standards used in GeoNetwork (if GRASS users also use GeoNetwork). Anyway, Rome was not built in a day and I am 100% sure that your work will be very, very useful already from what you planned for GSoC. All the best, Stefan ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] [SoC] gsoc2014 grass metadata week 6
Hi Stefan, On Thu, Jul 3, 2014 at 9:46 AM, Blumentrath, Stefan stefan.blumentr...@nina.no wrote: Hi Matej, Thanks for your reply. Metadata is a current topic I my organization at the moment, so your work comes exactly at the right time for me. Here in NINA people are using many different kinds of GIS software and we are considering introducing GeoNetwork as a central metadata storage solution, which makes metadata management to some extend independent from the GIS software used. Among public and environmental organisations, GeoNetwork is quite popular for metadata management (amongst others Mapping authorities from Netherlands, Switzerland, and the Nordic countries are using it, see also: http://geonetwork-opensource.org/gallery/gallery.html). I have no real personal experience with GeoNetwor (yet), but from what I read in the documentation I could imagine two ways how GRASS metadata could make their way into such a central metadata catalogue: 1) XML import (+ this option is in place when GRASS metadata is written to XML, - import requires manual labor) 2) Metadata harvesting (GeoNetwork can “harvest” metadata from other CSWs, so maybe pycsw already provides automatic an interface in that direction(?). That would be great!) pycsw is a CSW server that allows to harvest information from other OGC services, as well as Geonetwork. This latter offers also the editor, while pycsw doesn't, but is used internally by other applications which offer full metadata management (and a GUI), such as CKAN, GeoNode, etc. and the idea is to add GRASS to this list as well :-) If you want to stick with geonetwork, you can harvest the metadata from the pycsw, there's no need of manual work. When it comes to consistent metadata, I guess e.g. lists with values for metadata fields will in practice have to be more or less dynamic (as new staff member are being employed, new topics for spatial analysis arise (requiring new keywords), new types of data being developed and so on). Not sure if manipulating the templates is convenient enough for average users (seems to be mainly suitable for users with some degree of programming experience). However, good to know that providing a custom set of values is already possible through the templates. If you start working on a DB for metadata related content, feel free to let me know if I can support you by any means (yet I will be on leave for almost the rest of the year, which means I can spend only a limited amount of time). The idea is to allow user to generate a custom template using the gui, that can be reused for any metadata generated afterwards. Your idea of a database deserves consideration as well Thanks Madi However, I had a second look on GeoNetwork in this regard, and GeoNetwork is (naturally) using a database for metadata management (one option here is PostgreSQL). If it would be possible to just connect to that same DB (e.g. for fetching and storing key words, data on contact persons…), that would be really, really great, as this ensures that metadata produced in GRASS are 100% compliant to metadata standards used in GeoNetwork (if GRASS users also use GeoNetwork). Anyway, Rome was not built in a day and I am 100% sure that your work will be very, very useful already from what you planned for GSoC. All the best, Stefan ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev -- Best regards, Dr. Margherita DI LEO Scientific / technical project officer European Commission - DG JRC Institute for Environment and Sustainability (IES) Via Fermi, 2749 I-21027 Ispra (VA) - Italy - TP 261 Tel. +39 0332 78 3600 margherita.di-...@jrc.ec.europa.eu Disclaimer: The views expressed are purely those of the writer and may not in any circumstance be regarded as stating an official position of the European Commission. Hi Matej, Thanks for your reply. Metadata is a current topic I my organization at the moment, so your work comes exactly at the right time for me. Here in NINA people are using many different kinds of GIS software and we are considering introducing GeoNetwork as a central metadata storage solution, which makes metadata management to some extend independent from the GIS software used. Among public and environmental organisations, GeoNetwork is quite popular for metadata management (amongst others Mapping authorities from Netherlands, Switzerland, and the Nordic countries are using it, see also: http://geonetwork-opensource.org/gallery/gallery.html). I have no real personal experience with GeoNetwor (yet), but from what I read in the documentation I could imagine two ways how GRASS metadata could make their way into such a central metadata catalogue: 1) XML import (+ this option is in place when GRASS metadata is written to XML, - import requires manual labor) 2) Metadata harvesting (GeoNetwork can “harvest”
Re: [GRASS-dev] [GRASS GIS] #1787: Profile Analysis Tool Problems
#1787: Profile Analysis Tool Problems +--- Reporter: stu | Owner: grass-dev@… Type: defect | Status: new Priority: major | Milestone: 7.1.0 Component: wxGUI | Version: svn-trunk Keywords: profile tool r.profile |Platform: Linux Cpu: x86-64 | +--- Changes (by dadudeman): * priority: normal = major * platform: Unspecified = Linux * version: 6.4.3 RCs = svn-trunk * milestone: 6.4.3 = 7.1.0 * keywords: profile tool = profile tool r.profile * cpu: OSX/Intel = x86-64 Comment: Used r.profile interactively in GRASS 7.1 SVN. Same problems described by stu. If I use horizontal and vertical units of meters, the plot is perfect. If I change to horizontal and vertical units of feet, the Y axis of elevations is displayed correctly, but the X axis is squished to stop at the unit value in meters rather than feet. Example: I had a 800 meter transect, but when reprojected to feet and profiled, the graph stops at 800 feet rather than the approximate 2600. The endpoint triangle, however, is placed around the 2600 distance. It's as if it thinks the units are meters horizontal. I get a seemingly related issue when I use the measure tool, as the distances are reported as units meters, but the values are reported correctly as feet. I also tested this with grass 6.4 binary. -- Ticket URL: https://trac.osgeo.org/grass/ticket/1787#comment:7 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] [GRASS-SVN] r60679 - grass/trunk/lib/python/script
Hi, 2014-07-02 20:54 GMT+02:00 Martin Landa landa.mar...@gmail.com: It's because it can't find v.in.gps. I suggest to add to the patch something like this: thanks, I included modified patch to the next build n1011. Martin seems to be set forward, committed as r61135, feel free to improve. Martin -- Martin Landa * http://geo.fsv.cvut.cz/gwiki/Landa ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Submitting rules: Commit messages
2014-07-03 2:31 GMT+02:00 Glynn Clements gl...@gclements.plus.com: Martin Landa wrote: Personally, I don't favour duplicating information which is stored elsewhere (e.g. which files are affected). we know, btw r60590 (G_fatal_longjmp()) lacks _any_ documentation, at least minimal info in doxygen format would be nice... /* You probably shouldn't be using this function */ /* Please direct enquiries to the grass-dev mailing list */ Everything about that function (including its existence) should be considered unstable. Any usage (or even intended usage) should be known to us. hm, probably some code sample would be nice include similarly to other functions. Martin -- Martin Landa * http://geo.fsv.cvut.cz/gwiki/Landa ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] sphinx documentation for lib python
Hi devs, I just commit (r61141) the first step of Python Sphinx documentation. Now it compile with WXGUI Sphinx documentation so just launching 'make sphinxdoc' I removed pygrass/docs and kept the useful pages in the new lib/python/docs/ Please test it and report any problems. Next steps is to update the docstring in the python files. I already started with lib/python/imaging/ -- ciao Luca http://gis.cri.fmach.it/delucchi/ www.lucadelu.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] sphinx documentation for lib python
Hi, 2014-07-03 17:02 GMT+02:00 Luca Delucchi lucadel...@gmail.com: I just commit (r61141) the first step of Python Sphinx documentation. Now it compile with WXGUI Sphinx documentation so just launching 'make sphinxdoc' I removed pygrass/docs and kept the useful pages in the new lib/python/docs/ thanks for this work, I like to see that sphinx is not called from main rule (`make`). Martin -- Martin Landa * http://geo.fsv.cvut.cz/gwiki/Landa ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] sphinx documentation for lib python
Hi, 2014-07-03 17:06 GMT+02:00 Martin Landa landa.mar...@gmail.com: sphinxdoc' please consider renaming this rule to `sphinxdocs` (similarly to `htmldocs` and friends - see include/Make/Docs.make). Martin -- Martin Landa * http://geo.fsv.cvut.cz/gwiki/Landa ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] [GRASS GIS] #2357: r3.info segfaults or reports incorrect values on Windows
#2357: r3.info segfaults or reports incorrect values on Windows
-+--
Reporter: annakrat | Owner: grass-dev@…
Type: defect | Status: new
Priority: normal | Milestone: 7.1.0
Component: Raster3D | Version: svn-releasebranch70
Keywords: r3.info |Platform: MSWindows 8
Cpu: Unspecified |
-+--
When I run r3.info on Windows on a map computed with v.vol.rst, it's
segfaults and stops writing output after:
{{{
|Projection: Lambert Conformal Conic (zone 0)
|N:100S: 0 Res: 1
|E:100W: 0 Res: 1
}}}
When I create a random map:
{{{
g.region w=0 e=100 s=0 n=100 t=100 b=0 res=1 res3=1
r3.mapcalc test = rand(0,100)
}}}
it doesn't segfault but I get nonsense (I guess it's the same problem):
{{{
|Projection: Lambert Conformal Conic (zone 0)
|N:100S: 0 Res: 1
|E:100W: 0 Res: 1
|T: 0B: 0 Res: 0
| Range of data: min = 0 max = 0
}}}
The same map copied to linux has no problems. Perhaps the function
`format_double` could cause it since it is used for reporting top, bottom
and range values.
Further investigation shows that r.info reports wrong number of categories
(0) on a map which has categories. This would again point at
`format_double` function which looks completely the same as in r3.info.
However, v.info seems to work although it uses again `format_double`
function, but slightly modified:
r(3).info:
{{{
void format_double(const double value, char *buf)
{
sprintf(buf, %.8lf, value);
G_trim_decimal(buf);
}
}}}
v.info:
{{{
void format_double(double value, char *buf)
{
sprintf(buf, %.8f, value);
G_trim_decimal(buf);
}
}}}
Could the `l` in the formatting make difference on Windows?
--
Ticket URL: http://trac.osgeo.org/grass/ticket/2357
GRASS GIS http://grass.osgeo.org
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] [GRASS GIS] #1787: Profile Analysis Tool Problems
#1787: Profile Analysis Tool Problems +--- Reporter: stu | Owner: grass-dev@… Type: defect | Status: new Priority: major | Milestone: 7.1.0 Component: wxGUI | Version: svn-trunk Keywords: profile tool r.profile |Platform: Linux Cpu: x86-64 | +--- Comment(by annakrat): Replying to [comment:7 dadudeman]: I get a seemingly related issue when I use the measure tool, as the distances are reported as units meters, but the values are reported correctly as feet. I actually fixed the measurement tool yesterday (r61132). The problem is I guess that the units are FootUSs which is not recognized as feet. So now the original units are displayed as footuss which is still kind of weird but not wrong at least. -- Ticket URL: https://trac.osgeo.org/grass/ticket/1787#comment:8 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Unify python GRASS defined exception in one common file
Dear devs,
What do you think if we unify all our custom exceptions in one common file?
Just to avoid code repetitions and help consistency.
Looking into the code I found the following custom exceptions:
{{{
$ grep --color=auto
--exclude-dir={.svn,.git,.OBJ,locale,dist.x86_64-unknown-linux-gnu}
-IrnE class\s\w+\(Exception\)\:
lib/python/pygrass/messages/__init__.py:21:class FatalError(Exception):
lib/python/pygrass/errors.py:11:class AbstractError(Exception):
lib/python/pygrass/errors.py:19:class ParameterError(Exception):
lib/python/pygrass/errors.py:23:class FlagError(Exception):
lib/python/pydispatch/errors.py:4:class DispatcherError(Exception):
lib/python/temporal/abstract_dataset.py:26:class ImplementationError(Exception):
lib/python/temporal/temporal_algebra.py:655:class FatalError(Exception):
lib/python/ctypes/ctypesgencore/parser/lex.py:46:class LexError(Exception):
lib/python/ctypes/ctypesgencore/parser/yacc.py:87:class
YaccError(Exception): pass
lib/python/script/core.py:50:class ScriptError(Exception):
gui/wxpython/core/gcmd.py:156:class GException(Exception):
gui/wxpython/wxgui.py:86:class Usage(Exception):
}}}
Perhaps we could create a new file in grass.script.errors or
grass.script.exceptions, or outside script so something like
grass.errors or grass.exceptions.
Do you think it is a bad idea?
Best regards
Pietro
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] [GRASS GIS] #2326: Command functions in grass.script.core miss a correct error reporting
#2326: Command functions in grass.script.core miss a correct error reporting +--- Reporter: wenzeslaus | Owner: grass-dev@… Type: enhancement | Status: new Priority: major | Milestone: 7.1.0 Component: Python | Version: svn-trunk Keywords: script, exceptions |Platform: All Cpu: Unspecified | +--- Comment(by wenzeslaus): What do you (all) think about moving the function `call_module()` from testing framework into the `grass.script.core` module? The main point of this function is that it raises exception when return code of the module is non-zero. Additionally, it provides an convenient interface for capturing stdout and stderr and also for (optional) providing stdin (as string, or stream). It uses the safest possible way to achieve this which is `Popen.communicate()` method. By default it captures stdout (stdout=PIPE) and returns it as string and captures stderr (stderr=PIPE) and puts it into the exception when return code was non-zero. This can be disabled or stderr can be merged to stdout (stderr=STDOUT). * source:sandbox/wenzeslaus/gunittest/gmodules.py?rev=61134#L84 The function is universal and it can be used to implement the other alternatives (`run_`, `write_`, and `read_`). We can change the name to `call_command()` to be consistent with other functions or we can use `call_module()` to emphasize that it have different interface. It raises exception derived from `subprocess`'s `CalledCommandError`. But there is not special need for that. This is just for consistency with `subprocess`'s `check_call` function which is not used by `call_command` or in GRASS. * source:sandbox/wenzeslaus/gunittest/gmodules.py?rev=61134#L It has tests to prove that it behaves how it should. * source:sandbox/wenzeslaus/gunittest/testsuite/test_gmodules.py?rev=61134#L26 This of course does not solve the problems of wrong usage of `run_command` and it does not enforce usage of fatal error. However, it allows to write new code for libraries and scripts in a better, Python-friendly, way. The usage for this function is where the module is used more like a function then as a subprocess, in this case we don't care much about module progress or messages unless there was an error. Typical usage are the modules providing some stdout with key-value pairs. -- Ticket URL: http://trac.osgeo.org/grass/ticket/2326#comment:18 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] [SoC] gsoc2014 grass metadata week 6
Hei Madi, Thanks for your clarification. Very interesting! I see I have to take a closer look on pycsw and it`s front-ends (as a possible alternative to GeoNetwork). The pycsw documentation [1] says that also pycsw Metadata repositories are set up with a database backend (SQLite, PostgreSQL, and even PostGIS support). However, integration with data portal solutions like e.g. GeoNode or Open Data Catalogue is unfortunately “read-only” meaning that meta-data can be only queried [2]. Would be nice if metadata from GRASS could somehow go directly into a geodata portal (and that the other way around e.g. people, keywords … could be fetched from there)… (BTW, in the proprietary world such a functionality requires yet another extension: https://www.geocat.net/bridge/) For organization with a geodata portal solution it is maybe possible to just sync pycsw`s DB with a DB from e.g. GeoNode or so, if one wants to have that consistent… If you allow me two more questions in this regard: Will pycsw become a dependency for GRASS (with metadata support)? And will it be necessary, that pycsw runs on the same computer as GRASS (meaning also one pycsw for each GRASS installation) or could that somehow be centralized (in other words that GRASS connects to a central pycsw)? Cheers Stefan [1] http://docs.pycsw.org/en/1.8.3/administration.html#metadata-repository-setup [2] http://docs.pycsw.org/en/1.8.3/geonode.html From: Margherita Di Leo [mailto:dileomargher...@gmail.com] Sent: 3. juli 2014 13:40 To: Blumentrath, Stefan Cc: Matej Krejci; Martin Landa; GRASS-dev Subject: Re: [GRASS-dev] [SoC] gsoc2014 grass metadata week 6 Hi Stefan, On Thu, Jul 3, 2014 at 9:46 AM, Blumentrath, Stefan stefan.blumentr...@nina.nomailto:stefan.blumentr...@nina.no wrote: Hi Matej, Thanks for your reply. Metadata is a current topic I my organization at the moment, so your work comes exactly at the right time for me. Here in NINA people are using many different kinds of GIS software and we are considering introducing GeoNetwork as a central metadata storage solution, which makes metadata management to some extend independent from the GIS software used. Among public and environmental organisations, GeoNetwork is quite popular for metadata management (amongst others Mapping authorities from Netherlands, Switzerland, and the Nordic countries are using it, see also: http://geonetwork-opensource.org/gallery/gallery.html). I have no real personal experience with GeoNetwor (yet), but from what I read in the documentation I could imagine two ways how GRASS metadata could make their way into such a central metadata catalogue: 1) XML import (+ this option is in place when GRASS metadata is written to XML, - import requires manual labor) 2) Metadata harvesting (GeoNetwork can “harvest” metadata from other CSWs, so maybe pycsw already provides automatic an interface in that direction(?). That would be great!) pycsw is a CSW server that allows to harvest information from other OGC services, as well as Geonetwork. This latter offers also the editor, while pycsw doesn't, but is used internally by other applications which offer full metadata management (and a GUI), such as CKAN, GeoNode, etc. and the idea is to add GRASS to this list as well :-) If you want to stick with geonetwork, you can harvest the metadata from the pycsw, there's no need of manual work. When it comes to consistent metadata, I guess e.g. lists with values for metadata fields will in practice have to be more or less dynamic (as new staff member are being employed, new topics for spatial analysis arise (requiring new keywords), new types of data being developed and so on). Not sure if manipulating the templates is convenient enough for average users (seems to be mainly suitable for users with some degree of programming experience). However, good to know that providing a custom set of values is already possible through the templates. If you start working on a DB for metadata related content, feel free to let me know if I can support you by any means (yet I will be on leave for almost the rest of the year, which means I can spend only a limited amount of time). The idea is to allow user to generate a custom template using the gui, that can be reused for any metadata generated afterwards. Your idea of a database deserves consideration as well Thanks Madi However, I had a second look on GeoNetwork in this regard, and GeoNetwork is (naturally) using a database for metadata management (one option here is PostgreSQL). If it would be possible to just connect to that same DB (e.g. for fetching and storing key words, data on contact persons…), that would be really, really great, as this ensures that metadata produced in GRASS are 100% compliant to metadata standards used in GeoNetwork (if GRASS users also use GeoNetwork). Anyway, Rome was not built in a day and I am 100% sure that
Re: [GRASS-dev] [GRASS GIS] #494: break in importing and cleaning very large vector datasets
#494: break in importing and cleaning very large vector datasets -+-- Reporter: gisboa | Owner: grass-dev@… Type: defect | Status: reopened Priority: major | Milestone: 6.4.0 Component: Vector | Version: 6.4.0 RCs Resolution: |Keywords: vector import clean build Platform: All | Cpu: All -+-- Changes (by dido): * status: closed = reopened * resolution: worksforme = Comment: Same issue was observed on a large (282 MB shapefile) containing ~350 000 polygons. A dump from the output window: Layer: BGM_Polygons_L0 Counting polygons for 358823 features... Importing map 358823 features... - ... 483699 primitives registered 17237143 vertices registered Number of nodes: 414562 Number of primitives: 483699 Number of points: 0 Number of lines: 0 Number of boundaries: 483699 Number of centroids: 0 Number of areas: - Number of isles: - - Cleaning polygons, result is not guaranteed! - Snap boundaries (threshold = 1.000e-005): G_realloc: unable to allocate 155760024 bytes at snap.c:155 Finished with error System is Win7x64 with 16G of RAM. RAM usage reported was ~4.3G at max, dropped down to ~2.6G, then the error popped-up. Progress was at ~70%. This was seen first on a QGIS 2.2.0 Valmiera, same behavior was seen on 1.8.0 Lisboa. -- Ticket URL: https://trac.osgeo.org/grass/ticket/494#comment:4 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] [GRASS GIS] #494: break in importing and cleaning very large vector datasets
#494: break in importing and cleaning very large vector datasets --+- Reporter: gisboa | Owner: grass-dev@… Type: defect | Status: reopened Priority: major| Milestone: 6.4.0 Component: Vector | Version: 6.4.2 Resolution: |Keywords: vector import clean build Platform: MSWindows 7 | Cpu: x86-64 --+- Changes (by dido): * platform: All = MSWindows 7 * version: 6.4.0 RCs = 6.4.2 * cpu: All = x86-64 -- Ticket URL: https://trac.osgeo.org/grass/ticket/494#comment:5 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] [GRASS-SVN] r61135 - grass/trunk/lib/python/script - # TODO: what are cmd.exe's parsing rules?
http://lists.osgeo.org/pipermail/grass-commit/2014-July/032050.html may be some hints: http://msdn.microsoft.com/en-us/library/a1y7w461.aspx http://blogs.msdn.com/b/twistylittlepassagesallalike/archive/2011/04/23/everyone-quotes-arguments-the-wrong-way.aspx http://ss64.com/nt/syntax-esc.html http://ss64.com/nt/syntax-args.html - best regards Helmut -- View this message in context: http://osgeo-org.1560.x6.nabble.com/GRASS-SVN-r61135-grass-trunk-lib-python-script-TODO-what-are-cmd-exe-s-parsing-rules-tp5149549.html Sent from the Grass - Dev mailing list archive at Nabble.com. ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Week 7: GRASS GIS 3D flowlines
Hi, I will be traveling tomorrow, so I am sending this a bit earlier. 1. What did you get done this week: I fixed gradient computation from last week and did some profiling. I started with rewriting the module to C to see how familiar I am with C. 2. What do you plan on doing next week? I will continue with the C implementation but I will probably keep getting back to the Python module. 3. Are you blocked on anything? No. Anna [1] http://trac.osgeo.org/grass/wiki/GSoC/2014/ImplementationOf3DRasterFlowLine [2] http://trac.osgeo.org/grass/browser/sandbox/annakrat ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] [GRASS GIS] #1787: Profile Analysis Tool Problems
#1787: Profile Analysis Tool Problems +--- Reporter: stu | Owner: grass-dev@… Type: defect | Status: new Priority: major | Milestone: 7.1.0 Component: wxGUI | Version: svn-trunk Keywords: profile tool r.profile |Platform: All Cpu: x86-64 | +--- Changes (by annakrat): * platform: Linux = All Comment: Replying to [comment:7 dadudeman]: Used r.profile interactively in GRASS 7.1 SVN. Same problems described by stu. If I use horizontal and vertical units of meters, the plot is perfect. If I change to horizontal and vertical units of feet, the Y axis of elevations is displayed correctly, but the X axis is squished to stop at the unit value in meters rather than feet. Example: I had a 800 meter transect, but when reprojected to feet and profiled, the graph stops at 800 feet rather than the approximate 2600. The endpoint triangle, however, is placed around the 2600 distance. It's as if it thinks the units are meters horizontal. Yes, it's meters because meters come from r.profile and the GUI profiling tool just takes the values as they are. I wonder if this should be changed in r.profile rather than in the GUI? -- Ticket URL: https://trac.osgeo.org/grass/ticket/1787#comment:9 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] [GRASS GIS] #2357: r3.info segfaults or reports incorrect values on Windows
#2357: r3.info segfaults or reports incorrect values on Windows -+-- Reporter: annakrat | Owner: grass-dev@… Type: defect | Status: new Priority: normal | Milestone: 7.1.0 Component: Raster3D | Version: svn-releasebranch70 Keywords: r3.info |Platform: MSWindows 8 Cpu: Unspecified | -+-- Comment(by wenzeslaus): I seems that some C standards allow `l` to precede `f` but they also say that `lf` is the same as `f`. From this it seems that on MS Windows, because of MinGW or whatever, `lf` have unpredictable behavior. So, since it is not different form `f` (both mean `double`), I suggest to replace all occurrences of `lf` by `f`. Also, the fact that there is more than `format_double()` function tells me that it should be moved to the library. It might also avoid possibility of bugs like this. The function looks pretty general, or at least, it is used generally. Or do you think that these small functions should not go to the library? -- Ticket URL: http://trac.osgeo.org/grass/ticket/2357#comment:1 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
