[GRASS-dev] Re: [GRASS-SVN] r48376 - in grass/trunk: lib/python lib/temporal temporal temporal/t.create temporal/t.time.abs temporal/t.time.rel
Hallo Martin, Thanks for the hint. The messages will be prepared for translation. lm Still in an experimental State So all messages are subject of Change. l will consider the translation preparation for new messages from now on. Best Regards Sören Am 20.09.2011 16:39 schrieb Martin Landa landa.mar...@gmail.com: ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Create new branch from trunk for temporal GIS integration
Hi, [snip] I had a private branch of trunk for more than a year when developing and testing new vector topology. It was not always easy to keep it working, although I only had to svn up to get in sync with the official trunk. This is not so easy when creating a real branch of trunk in svn. But if you feel more comfortable with a separate branch for experimenting then do it. I wanted to warn, but of course will assist you whatever your decision. After some experimenting and the development of simple prototypes (getting very frustrated with the db library ... ) i decided to use python for all the temporal logic, library and modules. The modification in GRASS core libraries will be minimized. Only a few C library functions are planed in the future, implementing the INSERT/UPDATE/DELETE SQL queries for raster, vector and raster3d maps in the location specific time and metadata database. The consequence is: * Using sqlite3 exclusively, no support for other databases as backend (Unfortunately the implementation of trigger and temporal logic is database specific ...) * Implementation of all the temporal logic in python * Using an OO approach to reflect the theory of generalized fields [Liu and Goodchild 2008] * Using sqlite3 python library directly to have full datatype support and python based sqlite3 user functions * Using a location wide database rather than a mapset specific one, to store the temporal and metadata information for all mapsets, primary key will be name@mapset Because of the minimized footprint of this approach in the GRASS core libraries, i decided to implement the temporal GIS extension in trunk. So no new branch is needed. Thanks for all your suggestions and comments. Cheers Soeren Markus M The branch should only exist for experimenting and rapid development and merged ASAP. Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Renaming G3d into Rast3d
Dear devs, i just noticed that massive renaming in the G3d library is going on. Nothing against to follow name convention, but it would be nice if this kind of changes are discussed in the list first, so i can prepare myself and my G3d dependent software for it. I only noticed the changes because my vtk-grass-bridge is not working anymore ... . Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Renaming G3d into Rast3d
Hello Martin, renaming the G3D - Raster3d functions to follow the grass naming convention is on my TODO list for years now ... . But i never took the time to start, so thanks for making the first step. Unfortunately i have currently no time to join you. :/ Best regards Soeren 2011/8/11 Martin Landa landa.mar...@gmail.com: Hi, 2011/8/11 Soeren Gebbert soerengebb...@googlemail.com: i just noticed that massive renaming in the G3d library is going on. Nothing against to follow name convention, but it would be nice if this kind of changes are discussed in the list first, so i can prepare myself and my G3d dependent software for it. I only noticed the changes because my vtk-grass-bridge is not working anymore ... . sorry for inconvenience, this step should have been done much more earlier, but nobody did it. There are more important issues - fn naming convention in this library completely breaks standard naming convention from other GRASS core libraries. Moreover, some of the fns should be replaced by similar functions for GIS library (see at least alloc.c and error.c). I decided just to break the ice ;-) Anyway I should have to announce it. Are you interested to rename and clean up subroutines in this library towards standard naming convention and no overlap with other (GIS and Raster) libraries? Thanks, Martin -- Martin Landa landa.martin gmail.com * http://geo.fsv.cvut.cz/~landa ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] G7: parser - suppressing label in scripts
Hello Glynn,
thanks for your suggestions. But i think you are two steps ahead from
me, because i am not able to find any strdup() usage in parser*.c.
Are there any steps to do before replacing strdup with your suggested xstrdup?
Thanks and best regards
Soeren
2011/8/10 Glynn Clements gl...@gclements.plus.com:
Markus Neteler wrote:
we have a problem with the introduction of labels:
http://trac.osgeo.org/grass/changeset/39886/grass/trunk/lib/gis/parser_standard_options.c
Actually, the problem only arose with the addition of standard options
in r47421.
It would be good to be able to suppress label with the a reserved word
like None or NULL:
#%option G_OPT_V_INPUTS
#% key: vector
#% label: NULL
#% description: Name of input vector map(s) used by GRASS script
#% required: no
#%end
Any suggestions how to implement this suppressing label in the parser?
In parse.c, I suggest adding e.g.:
static char *xstrdup(const char *arg) {
return (strcmp(arg, {NULL}) == 0)
? NULL
: strdup(arg);
}
and replacing all references to strdup() with xstrdup(). Also,
translate() will need a test for NULL pointers.
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: GRASS Test suite
Hello Anne, 2011/7/4 Anne Ghisla a.ghi...@gmail.com: Le lundi 04 juillet 2011 à 10:52 +0200, Soeren Gebbert a écrit : Hi all, 2011/7/4 Martin Landa landa.mar...@gmail.com: Hi, 2011/7/4 Anne Ghisla a.ghi...@gmail.com: I'm just looking at grass7/raster/r.colors/ and the test files really dominate the module dir. +1 from me on having a testsuite/ or tests/ subdir for each module. also +1 from me. The directory layout has been discussed here: http://osgeo-org.1803224.n2.nabble.com/Test-suite-for-GRASS-proposal-discussion-welcome-td6391218.html I would suggest to either use reference_data/ subdir for each module, so that the test scripts may stay in the module directory, not in a subdirectory, or a testsuite/ subdir including all data? I personally prefer a reference data subdir, to have a clear separation of test case scripts and reference data. Otherwise the reference data will dominate again and finding the test scripts is not that obvious. In the case of r.colors, there are a lot of .ref files and one .sh script, that is hidden among them when the folders contents are ordered alphabetically. I haven't checked if it will be the case of other modules. This is the case for most modules with tests applied: http://trac.osgeo.org/grass/browser/grass/trunk/raster3d/r3.out.ascii http://trac.osgeo.org/grass/browser/grass/trunk/raster3d/r3.cross.rast http://trac.osgeo.org/grass/browser/grass/trunk/raster3d/r3.retile http://trac.osgeo.org/grass/browser/grass/trunk/raster3d/r3.stats http://trac.osgeo.org/grass/browser/grass/trunk/raster3d/r3.to.rast http://trac.osgeo.org/grass/browser/grass/trunk/raster/r.to.rast3elev Personally I prefer to keep all test data and scripts together in the tests/ subfolder. But if the majority find it uncomfortable as the script file is less accessible, let's keep it in module folder and put data in the subfolder (after correcting the paths in script file). Opinions? The paths used in the test scripts should not be affected by the location of the reference files. Only the framework and the test designer need to know that. The user/dev who implements the tests must assure that the reference files are located in the reference directory. The framework than take care that while testing generated files are compared with the reference files located in the reference directory. H, ... i think the test scripts should always write any ascii file output in the current working directory, so the framework can execute the tests in a temporary directory which can be removed for clean up after reference data validation. I will add this to the wiki. Lets keep for practical reason the test scrip in the module directory and add a test_reference_data or test_refs directory to the module directory keeping the reference data. Best regards Sören ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] G3d library code review and raster lib tile approach
Hi Glynn, i hope i don't bother you to much about GDAL-GRASS raster implementation. :) How is the performance of this approach compared to the native raster I/O in GRASS? It depends entirely upon the format and the usage. E.g. compressed formats such as PNG are likely to be inefficient if you try to skip rows. Pleas excuse my little knowledge about GDAL but an important point to change the grass raster row approach was to enable random access for single values using tiles? Will the GDAL approach provide such capabilities? What kind of new function will be available with GDAL? Should such a strong external dependency really be considered? GDAL is already almost mandatory. I know little about GDAl community but would the GDAL community accept that GRASS GIS core functionality will be part of GDAL? So GRASS developer/documenter who need to fix bugs or try to improve GRASS raster library need to have write access to the GDAL repository? Will GRASS depend on GDAL release cycles in case of stable version? Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: GRASS Test suite
Hi all, 2011/7/4 Martin Landa landa.mar...@gmail.com: Hi, 2011/7/4 Anne Ghisla a.ghi...@gmail.com: I'm just looking at grass7/raster/r.colors/ and the test files really dominate the module dir. +1 from me on having a testsuite/ or tests/ subdir for each module. also +1 from me. The directory layout has been discussed here: http://osgeo-org.1803224.n2.nabble.com/Test-suite-for-GRASS-proposal-discussion-welcome-td6391218.html I would suggest to either use reference_data/ subdir for each module, so that the test scripts may stay in the module directory, not in a subdirectory, or a testsuite/ subdir including all data? I personally prefer a reference data subdir, to have a clear separation of test case scripts and reference data. Otherwise the reference data will dominate again and finding the test scripts is not that obvious. Best regards Sören Martin -- Martin Landa landa.martin gmail.com * http://geo.fsv.cvut.cz/~landa ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: GRASS Test suite
Hi Anne, 2011/7/4 Anne Ghisla a.ghi...@gmail.com: Le dimanche 03 juillet 2011 à 15:53 -0700, Hamish a écrit : Hi, can we move the test files into $MODULE/testsuite/ or similar? I'm just looking at grass7/raster/r.colors/ and the test files really dominate the module dir. +1 from me on having a testsuite/ or tests/ subdir for each module. I'm starting to implement the test framework, it will be in trunk/testsuite. Hints on how to modify the Makefile are welcome! Thank you for starting the implementation. If you have any tiny question, don't bother to ask. I hope the whole test suite picture comes much clearer when some basic framework is available to play around a with. Important: in case the test script approach which i have used for the test cases in the g3d modules is not sufficient, i will modify them to fit the new guidelines and insights. Best regards Soeren best, Anne ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: GRASS Test suite
Hi Anne,
i will put the GRASS test suite implementation discussion on the dev
list if you don't mind? :)
A segfault should not happen. All tests run on my systems very well
(latest grass7, openSuse 11.2).
Do you use the latest grass7 trunk? Can you please use gdb to hunt
down the error, i will fix it.
I use GRASS 7 r 46894, compiled this morning, on Fedora 14 64bit.
`r3.colors -e map=volume_double_null rules=example1`
this command segfaults at raster/r.colors/stats.c, line 155, with
invalid pointer:
G_free(dcell);
Markus looked at the problem and suggested this patch, line 150:
if (type == RASTER_TYPE) {
Rast_close(fd);
G_free(dcell);
}
(sorry for patch format but it's such a tiny modification)
Many thanks for the patch, i have committed your solution to svn. Good
to have some tests. :)
I see that in the wiki the part about Python has been removed, which
parts of the system we drafted in Prague are still valid?
Not much. :)
I'm fine with setting up the Python part of the framework. I fear that
as I'm not familiar with makefiles, it's better to ask Glynn and the dev
list for guidance too.
I have updated the wiki with more implementation details for the
Python test framework:
http://grass.osgeo.org/wiki/Test_Suite#Implementation
This is just a very draft concept but it hopefully shows the
implementation approach i have in mind. IMHO the implementation should
not be very complex, except the ability of the framework to act as a
simple shell.
Regarding the Make system: unfortunately i am not familiar with
Makefile handling too, so i am no help on this topic.
Should we move the wiki page to the new trac wiki or can we leave it
in the old? I would suggest to move ... .
Best regards
Soeren
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] G3d library code review and raster lib tile approach
Hi Glynn, thanks for your answer. * Can the G3d library be used as backend for tile storage in the raster library (setting the z dimension to one)? Potentially. Just an implementation example which was in my mind using g3d as raster backend: I think the Rast_put_row() and Rast_get_row() function can be efficiently mapped using the g3d X-direction tile cache approach. The X-direction cache can be automatically set when the mapped Rast_open_*() functions are called. Consider a tile size of 64x64 pixel. An allocated raster buffer will be filled with for col in cols: buffer[col] = G3d_getValue(map, col, row, 0) or its content will be written with for col in cols: G3d_putValue(map, col, row, 0, buff[col]). In case the tile cache size is big enough reading/writing the first raster row will result in reading/creating all tiles in a row in memory caching 64 raster rows. In case of g3d raster maps with 1 columns the cache size must be 157 tiles == 5,2 MB. In case of 10 columns 52 MB. It would be interesting to compare the performance of this approach with the current raster approach. IMHO the g3d approach should not be much slower. Additionally Rast_put_value() and Rast_get_value() can easily be implemented allowing arbitrary single cell value access. The drawback is that CELL values are not supported and must be currently mapped using DCELL values ... well at least the g3d library has the ability to reduce the mantissa to 32Bit to minimize the disk space usage for compressed tiles. IMHO the g3d tile approach should be considered as raster backend and will result in minor to no changes in existing raster modules. Many of the ideas came up before I implemented r.external and r.external.out. That lead to an alternate possibility, that GRASS itself wouldn't actually have /any/ particular raster format but just deal with whatever GDAL supported. Using a pure gdal approach leads me to many questions: What would be the default format for reading and writing in a grass mapset? At which location will the files and the metadata be stored? Are there modules which need to be modified to use this approach? How is the performance of this approach compared to the native raster I/O in GRASS? Should such a strong external dependency really be considered? The would require that GDAL had native GRASS raster I/O code (i.e. not depending upon libgis/libraster). The main issue here is that the code would probably need to be written from scratch due to licensing issues: GRASS is GPL, GDAL is LGPL, and trying to track down everyone who might have a copyright interest in the raster I/O code wouldn't be practical. Will the benefit of this approach justify this huge effort of writing the I/O code from scratch? Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] G3d library code review and raster lib tile approach
Hi all, i have created a wiki page about the G3d library and related modules code review: http://trac.osgeo.org/grass/wiki/Grass7/G3dLib While surfing the wiki i found a section about the new rater library using a tile based approach instead of row based: http://trac.osgeo.org/grass/wiki/Grass7/RasterLib#Corerasterformat I have added some comments to the ideas with my knowledge of the G3d library gained while code reviewing. I would like to reactivate this topic and have some questions: * Its there a time frame when the new raster tile approach will be implemented? * Are there plans to merge the G3d library and the raster library? * Can the G3d library be used as backend for tile storage in the raster library (setting the z dimension to one)? Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS-SVN] r46744 - in grass/trunk: lib/g3d lib/gis lib/raster raster3d raster3d/r3.in.ascii
Hi Markus, thanks you very much. :) I hope the row order of the g3d tile approach is a bit clearer now ... and at least documented. Best regards Soeren 2011/6/27 Markus Neteler nete...@osgeo.org: Hi Soeren, On Mon, Jun 20, 2011 at 12:34 PM, svn_gr...@osgeo.org wrote: Author: huhabla Date: 2011-06-20 03:34:19 -0700 (Mon, 20 Jun 2011) New Revision: 46744 Added: grass/trunk/lib/g3d/g3d_volume_layout.png grass/trunk/lib/g3d/g3d_volume_layout.xcf grass/trunk/lib/g3d/volume_tile_cell_generator.sh grass/trunk/raster3d/r3.in.ascii/g3d_volume_layout.png (http://trac.osgeo.org/grass/browser/grass/trunk/raster3d/r3.in.ascii/g3d_volume_layout.png) ... this is really great! thanks Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] GRASS Make-system question
-linux-gnu/stats.o-lgrass_g3d.7.0.svn -lgrass_raster.7.0.svn -lgrass_gis.7.0.svn -lm GISRC=/home/soeren/src/grass7.0/grass_trunk/dist.x86_64-unknown-linux-gnu/demolocation/.grassrc70 GISBASE=/home/soeren/src/grass7.0/grass_trunk/dist.x86_64-unknown-linux-gnu PATH=/home/soeren/src/grass7.0/grass_trunk/dist.x86_64-unknown-linux-gnu/bin:$PATH PYTHONPATH=/home/soeren/src/grass7.0/grass_trunk/dist.x86_64-unknown-linux-gnu/etc/python:$PYTHONPATH LD_LIBRARY_PATH=/home/soeren/src/grass7.0/grass_trunk/dist.x86_64-unknown-linux-gnu/bin:/home/soeren/src/grass7.0/grass_trunk/dist.x86_64-unknown-linux-gnu/lib:/home/soeren/src/grass7.0/grass_trunk/dist.x86_64-unknown-linux-gnu/lib:/home/soeren/src/grass7.0/grass_trunk/dist.x86_64-unknown-linux-gnu/lib:/usr/local/lib/vtk-5.9:/usr/lib64/jvm/java-1.6.0-sun-1.6.0/jre/lib/amd64/xawt:/usr/local/lib:/home/soeren/src/grass7.0/grass_trunk/dist.x86_64-unknown-linux-gnu/lib:/usr/local/lib64/R/lib LC_ALL=C thumbnails.py --html-description /dev/null | grep -v '/body\|/html' r.colors.tmp.html /bin/sh: thumbnails.py: command not found make: *** [r.colors.tmp.html] Error 1 }}} Can you please give me a hint what i did wrong and how the Makefile should look like? Many thanks Soeren 2011/6/23 Glynn Clements gl...@gclements.plus.com: Soeren Gebbert wrote: Can anybody give me a hint how to solve this. Don't try to use quick hacks; do it properly. Either clone the file as e.g. main3.c or make it support both 2d and 3d at run-time. Well, actually my intention was not a quick hack. I thought it would be meaningful to use macros to reduce the appearance of cloned code for easier code maintainability. I did not know that using macros to decide at compile time the behavior of a program is a hack at all. It's not the use of macros that's a hack (there's plenty of code with conditional sections), but building multiple, distinct object files from the same source file. With one exception[1], the GRASS 7.0 build system relies entirely upon the '%.o: %.c' (etc) pattern rules in Compile.make. This ensures that any changes to those rules affect everything; there's no need to hunt down special cases which need to be changed separately (and, in practice, are likely to be overlooked). [1] macosx/app/Makefile has an explicit rule for main.o: main.m. I changed the code of r/r3.color.out to meet your guideline. I have to rethink the r/r3.colors approach to meet your guideline too, but to also reducing cloned code. I am not a fan of cloned code and modules with mixed behavior (raster and raster3d support) with the need adding scripts to hide there complexity: If you want distinct modules, make each #ifdef'd chunk into a separate function, then move those functions into separate source files, one for 2d and one for 3d. -- Glynn Clements gl...@gclements.plus.com ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] GRASS Make-system question
Hi Glynn, This won't work with Multi.make, as HTMLSRC will be empty and so $ will end up as thumbnails.py. Instead, use: r.colors.tmp.html: $(BIN)/r.colors$(EXE) thumbnails.py and: r3.colors.tmp.html: $(BIN)/r3.colors$(EXE) thumbnails.py This works perfectly, thanks a lot! The new r3.colors implementation is available in svn. Still missing is the -e flag for r3.colors. I think i have to implement something similar to the Cell_stats approach in the raster library based on tiles in the g3d library. Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] GRASS Make-system question
Hi Glynn, Can anybody give me a hint how to solve this. Don't try to use quick hacks; do it properly. Either clone the file as e.g. main3.c or make it support both 2d and 3d at run-time. Well, actually my intention was not a quick hack. I thought it would be meaningful to use macros to reduce the appearance of cloned code for easier code maintainability. I did not know that using macros to decide at compile time the behavior of a program is a hack at all. I changed the code of r/r3.color.out to meet your guideline. I have to rethink the r/r3.colors approach to meet your guideline too, but to also reducing cloned code. I am not a fan of cloned code and modules with mixed behavior (raster and raster3d support) with the need adding scripts to hide there complexity: For example: r.colors.fency does it all - script r.colors to offer only raster capabilities of r.colors.fancy - script r3.colors to offer only raster3d capabilities of r.colors.fancy Best regards Soeren -- Glynn Clements gl...@gclements.plus.com ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] GRASS Make-system question
Hi, i have a problem to set up a correct Makefile to compiling two different modules using the same source file (main.c) but with different EXTRA_CFLAGS. This is how far i got: MODULE_TOPDIR = ../.. LIBES2 = $(RASTERLIB) $(GISLIB) LIBES3 = $(G3DLIB) $(RASTERLIB) $(GISLIB) DEPENDENCIES = $(G3DDEP) $(GISDEP) $(RASTERDEP) PROGRAMS = r.colors.out r3.colors.out r_colors_out_OBJS = main.o r3_colors_out_OBJS = main.o include $(MODULE_TOPDIR)/include/Make/Multi.make default: multi $(BIN)/r.colors.out$(EXE): LIBES = $(LIBES2) $(BIN)/r3.colors.out$(EXE): LIBES = $(LIBES3) I stuck trying to add a preprocess macro -DUSE_RASTER3D for r3.colors.out? Can anybody give me a hint how to solve this. Where to put the preprocess macro in the Makefile so is will be used exclusively by r3.colors.out? Thanks and best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Principle G3D library question
Hi Maris,
2011/6/21 Maris Nartiss maris@gmail.com:
Hello,
It's not an backporting of header it's about GRASS6 being able to
distinguish GRASS7 format g3d files. I have not peaked into code if
there such feature exists. If not, it should be implemented to prevent
GRASS6 failures (or worse - data corruption) on GRASS7 files.
I see, good point!
The grass6 implementation checks for valid header:
static void readHeaderString(FILE * fp, char *valueString, double *value)
{
static char format[100];
/* to avoid buffer overflows we use snprintf */
G_snprintf(format, 100, %s %%lf, valueString);
if (fscanf(fp, format, value) != 1) {
G_debug(0, bad value for [%s], valueString);
fatalError(readHeaderString: header value invalid);
}
while (fgetc(fp) != '\n') ;
}
I think this should prevent the import of new grass7 volume ascii
files in grass6. The grass6 import will break in when the new lines of
the grass7 header are parsed.
Best regards
Soeren
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Principle G3D library question
Hi Maris,
the new rater3d ascii header looks like this now:
version: grass7
order: nsbt
north: 80.00
south: 0.00
east: 120.00
west: 0.00
top: 50.00
bottom: 0.00
rows: 8
cols: 12
levels: 5
The options version and order are introduced with my changes on
the g3d library in grass7. The manual page of the svn version of
r3.out.ascii in grass7 explains a bit more.
Best regards
Soeren
2011/6/21 Maris Nartiss maris@gmail.com:
(Still haven't looked at code) - I hope GRASS7 files will have a
version number in header to be future-proof.
Maris.
2011/6/21 Soeren Gebbert soerengebb...@googlemail.com:
Hi Maris,
2011/6/21 Maris Nartiss maris@gmail.com:
Hello,
It's not an backporting of header it's about GRASS6 being able to
distinguish GRASS7 format g3d files. I have not peaked into code if
there such feature exists. If not, it should be implemented to prevent
GRASS6 failures (or worse - data corruption) on GRASS7 files.
I see, good point!
The grass6 implementation checks for valid header:
static void readHeaderString(FILE * fp, char *valueString, double *value)
{
static char format[100];
/* to avoid buffer overflows we use snprintf */
G_snprintf(format, 100, %s %%lf, valueString);
if (fscanf(fp, format, value) != 1) {
G_debug(0, bad value for [%s], valueString);
fatalError(readHeaderString: header value invalid);
}
while (fgetc(fp) != '\n') ;
}
I think this should prevent the import of new grass7 volume ascii
files in grass6. The grass6 import will break in when the new lines of
the grass7 header are parsed.
Best regards
Soeren
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Principle G3D library question
Hello Hamish, many thanks for code reviewing. 2011/6/18 Hamish hamis...@yahoo.com: [r3.in.ascii] readHeaderString() while (fgetc(fp) != '\n'); probably it should use G_getl2(), which also accepts DOS and MacOS9 newlines? Done in svn. version: order: once formally decided, we should modify the 6.x versions to deal with the extra header instructions, and continue on (with a message?) if it matches grass6's format, otherwise fatal_error out with a reasonable error message. I don't think it is necessary to backport the header changes to 6.x. The grass6 version works fine, maybe a manual update is meaningful? For backward compatibility a flag was added to r3.out.ascii which is IMHO not overkill. :) ASCII is an acronym of a named entity, and should remain uppercased. Fixed in svn. Note that unit tests for r3.in.ascii are implemented in the emtest.r3.out.ascii.sh/em script located in the r3.out.ascii directory. if many/most modules will eventually get test scripts, is it worth mentioning that in the help page? maybe !-- commented out -- so only developers see it? Good test scripts provide valuable information about the module capabilities, data handling and possible bugs. Test scripts should be designed so that grass user can easily read and write them ... at least this is the goal we have. My hope is that user will write test scripts in case they found a bug or in case test scripts are still missing. Therefor in my humble opinion the test scripts should be added automatically to the end of the modules HTML manual pages. In case they are missing in the module directory (r3.in.ascii) the manual should explain why. I think is it enough to describe in the grass7 help page how to make a grass6-compatible G3D output file, probably with an example. A flag for that could be overkill. btw, I believe the AV responsible for parts of the G3D code is Alfonso Vitti. Thanks, i was now able to find the g3d related mails in the grass5 developer mail archive. Very valuable information. Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome
Hello Pawel,
2011/6/17 Pawel Netzel pawel.net...@uni.wroc.pl:
Hi,
By my opinion, sections are not clearly separated in the examples on wiki
page.
This is intention. IMHO the annotation should be part of the
documentation so the test writer need to formulate sentence using
predefined words.
It will be something like good practise to write:
# description of test
# .
#
#@preprocessing
# ... some comments
command1
command2
#
#@test
# some comments
command3
command4
or alternatively
# ... some commants ...
#@test
commandX
This will work of course too. If its generally accepted to use this
style in the test script we can document and suggest it as good
practice.
Are you going to introduce annotation @assert('...message...',condition) or
something like this?
No, it should be as simple as possible but not too simple. IMHO only
the framework has to take car of the error message generation. In case
special conditions are tested it should be noticed in the test
documentation.
Best regards
Soeren
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Principle G3D library question
Dear developer, i have a principal question regarding the coordinate system in the g3d library. I currently review the g3d code and related modules to get a clear picture of the order of the underlying coordinate system. It looks like different approaches have been developed in the modules and the g3d library. This is very confusing. I will try to harmonize this, so here is my question: What kind of coordinate system should be used in the g3d library? From my point of view: the original authors implemented the idea of a cube coordinate system starting in the lower left (SW) corner. This is the common cube coordinate system used in a wide range of software. BUT this coordinate system uses a different row order than the raster coordinate system which start in the upper left corner. Later development of the g3d library tried to establish the upper left corner as origin, but this messed up the code base and result in confusion about the correct coordinate transformation and row ordering. I currently try to reestablish the lower left (SW) corner approach using the common x, y, z cube coordinate system. The code works fine and many tests have been implemented to assure correct behavior. BUT the result is that volume maps created with several modules before these changes have now flipped rows. AND the g3d cube coordinate system is not row compatible with the raster coordinate system. For example: g.region b=0 t=1 s=0 w=0 e=120 n=80 res=10 res3=10 tbres=1 r.mapcalc --o expr=raster_map = row() r3.mapcalc --o expr=volume_map = row() This code will result in raster and volume maps with different row counting. The raster lib counts rows from north to south but the g3d lib counts from south to north. #: r.out.ascii raster_map north: 80 south: 0 east: 120 west: 0 rows: 8 cols: 12 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 5 5 6 6 6 6 6 6 6 6 6 6 6 6 7 7 7 7 7 7 7 7 7 7 7 7 8 8 8 8 8 8 8 8 8 8 8 8 #: r3.out.ascii volume_map dp=0 version: grass7 order: nsbt north: 80.00 south: 0.00 east: 120.00 west: 0.00 top: 1.00 bottom: 0.00 rows: 8 cols: 12 levels: 1 8 8 8 8 8 8 8 8 8 8 8 8 7 7 7 7 7 7 7 7 7 7 7 7 6 6 6 6 6 6 6 6 6 6 6 6 5 5 5 5 5 5 5 5 5 5 5 5 4 4 4 4 4 4 4 4 4 4 4 4 3 3 3 3 3 3 3 3 3 3 3 3 2 2 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 1 1 Booth modules export the data in the raster coordinate system printing the rows from north to south. What i don't know: Is the row ordering irrelevant for g3d internal value storage, tile storage and tile cache coordinates? So only coordinate transformation functions should take care of correct counting? So all g3d modules can use the raster coordinate system using x, y, and z coordinates and interact with raster modules in the same coordinate system: row == y: Top (z) | | North West | _ _ _ _ _ _ East (x) / Bottom / / South (y) Or does the row ordering affect the internal g3d storage and caching too and should always count from south to north??? So any module which interacts with raster rows need to translate the y coordinate into the correct raster row: row = rows - y - 1? I prefer the last solution (which i currently implement) to avoid potential conflicts with the (by myself not fully understood) storage and tile caching in the g3d library and to have a well understood and consistent behavior: Top (z) North (y) | / | / | / West | / _ _ _ _ _ _ East (x) Bottom South Maybe i am to blind to see the obvious? Any hints or suggestions are very welcome. Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Principle G3D library question
Hello Glynn, 2011/6/17 Glynn Clements gl...@gclements.plus.com: Soeren Gebbert wrote: What kind of coordinate system should be used in the g3d library? Make it match the 2d raster library. Row zero should be north; in the library, in the modules, in the file format, and in terms of externally-visible behaviour. E.g. this: Well you just destroyed my dreams to finish the g3d library and related modules next week. :) I will try to implement the addressing to match 2d raster library ... many test have to be rewritten and validated. Don't worry too much about compatibility; for 7.0, sanity comes first. Ok. However: if the existing file format has row zero at the southern edge, then we should to be able to distinguish the new format from the old, and the library should be able to (transparently) read either (it doesn't necessarily need to be able to write the old format). The problem is that i currently not completely understand the internal tile caching algorithm. I will need more time to understand it in more detail. Best regards Soeren -- Glynn Clements gl...@gclements.plus.com ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome
... Ugh; v.random should have a seed= option (or something) to allow repeatability. Or at least an option to disable seeding with the PID. The seed option for v.random is now available in r46730 including tests. Cheers Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome
Hi all, i have summarized the ideas of Anne, Glynn and me in the wiki: http://grass.osgeo.org/wiki/Test_Suite Hopefully most of the ideas a listed there. While reviewing the g3d library code and related modules i implemented prototypes of test cases for several r3.* modules. Some of them are added as test case code examples to the wiki. Any comments, suggestions are welcome. Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome
Hello,
I assume such critical modules are coded in the framework, not in the
test scripts?
I was thinking about a directive (e.g. @critical) in the test script
so that any failure during the test would generate a more prominent
Such kind of annotations/directives are a great idea. I was thinking
about a similar approach. I have had in mind to add such directives
into test cases to identify data preprocessing steps, test calls and
critical modules. I would like to use them as part of the test case
documentation:
r.mapclac_test.sh
{{{
# This test case is designed to test r.mapcalc,
# which is @critical for many other tests.
# We need to perform a @preprocess step
# with g.region to set up a specific LL test region
g.region s=0 w=0 n=90 e = 180 res=1
# The first @test generates a CELL raster map with value 1
r.mapcalc expression=result1 = 1
...
}}}
IMHO each test case should be well documented, so why not using
annotations as part of the documentation? Additionally i would like to
add the tests to the bottom of the HTML manual pages automatically as
examples.
message. If any such errors occured as a result of make test, you
would ignore all the other failures. In the same way that if error.log
has an error for e.g. lib/gis, you wouldn't bother about all of the
subsequent errors and focus on what was wrong with lib/gis.
Is it possible to stop make test in case a library test failed or a
critical module?
But this also means that the test scripts
must be interpreted and executed line by line by the framework to
identify critical modules used for data generation?
Test failures should not occur for critical modules. If they do, you
deal with the critical module, and ignore everything else until that
has been dealt with.
Indeed. I would suggest to put critical modules on top of the
directory makefiles to assure
that they are executed first and recursive testing stops when one of
them fails.
In case any library test or critical module failed, no further module
test should performed.
The test scripts would need to be processed a command at a time for
other reasons (assuming that the framework is going to be doing more
than simply executing the commands).
I had in mind that the return value of each command is checked and
stderr is logged for further analysis. The framework must be able to
identify which command failed/succeeded and for which command data
validation was available and successful. This data should be available
in the detailed test case specific HTML log files.
Example for a synthetic r.series test using r.mapcalc for data
generation. r.mapcalc is marked as critical in the framework:
In case r.mapcalc is marked as critical and the framework finds the
keyword r.mapcalc in the script, appearing as first word outside
of a comment, it checks if the r.mapcalc test(s) already run
correctly and stop the r.series test if they not.
I wouldn't bother with this part. If the user runs make test from
the top level, r.mapcalc's tests will end up getting run. If they
fail, then the user will get an error message informing them that a
critical module failed and that they should ignore everything else
until that has been addressed.
If you're doing repeated tests on a specific module that you're
working on, you don't want to be re-running the r.mapcalc, r.out.ascii
etc tests every time.
I don't know if this can be avoided in an automated test system.
Especially when each time a test case gets executed a temporary mapset
is created.
Except the developer comments the preprocessing steps out and executes
the script manually in the test location.
In case r.mapcalc
tests are valid it starts the r.mapcalc commands and checks there
return values. If the return values are correct, then the rest of the
script is executed. After reaching the end of this script the
framework looks for any generated data in the current mapset (raster,
raster3d, vector, color, regions, ...) and looks for corresponding
validation files in the test directory. In this case it will find the
raster maps input1, input2 and result in the current mapset and
validation.ref in the test directory. It will use r.out.ascii on
result map choosing a low precision (dp=3??) and compares the output
with result.ref which was hopefully generated using the same
precision.
Only result.ref would exist, so there's no need to export and validate
input1 and input2. In general, you don't need to traverse the entire
mapset directory, but only test for specific files for which a
corresponding .ref file exists.
Thats indeed much more efficient.
I'd export as much precision as is likely to be meaningful, erring on
the side of slightly too much precision. The default comparison
tolerance should be just large enough that it won't produce noise for
the majority of modules. Modules which require more tolerance (e.g.
due to numerical instability) should explicitly enlarge the tolerance
and/or set an allowed
Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome
Hi, Thats why I compare the topology information and not the position/coordinates. In any cases 100 points must be generated. Best Soeren Am 09.06.2011 12:21 schrieb Moritz Lennert mlenn...@club.worldonline.be: On 09/06/11 10:01, Glynn Clements wrote: Soeren Gebbert wrote: v.random output=random_points n=100 v.info -t random_points result If the framework understands vector maps, it shouldn't be necessary to use v.info; it should be able to compare the random_points map to random_points.ref. But how to compare two randomly generated maps ? Unless the system is not really random, the maps _should_ be different. Moritz ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome
Hello Glynn,
i was thinking a lot about your and my approach and decided finally to
try your approach first with the hope it will be sufficient for any
kind of test cases. I still have concerns about the comparison of
floating point data regarding precision, i.e: coordinates, region
settings, FCELL and DCELL maps, vector attributes ... . More below:
2011/6/3 Glynn Clements gl...@gclements.plus.com:
Soeren Gebbert wrote:
I was thinking about a similar approach, but the effort to parse the
modules XML interface description to identify the command line
arguments to compare the created data was to much effort for me.
I don't see a need to parse the command; just execute it and see what
files it creates.
Ok, i see.
Besides that, the handling of test description, module dependencies
and the comparison of multiple/timeseries outputs (r.sim.water)
bothered me too. I still have no simple (interface) answers to this
issues (maybe these are no issues??).
Dependencies aren't really an issue. You build all of GRASS first,
then test. Any modules which are used for generating test maps or
analysing data are assumed to be correct (they will have test cases of
their own; the most that's required is that such modules are marked as
critical so that any failure will be presumed to invalidate the
results of all other tests).
I assume such critical modules are coded in the framework, not in the
test scripts? But this also means that the test scripts
must be interpreted and executed line by line by the framework to
identify critical modules used for data generation?
Example for a synthetic r.series test using r.mapcalc for data
generation. r.mapcalc is marked as critical in the framework:
{{{
# r.series synthetic average test with r.mapcalc generated data
# The r.series result is validated using the result.ref file in this
test directory
# Generate the data
r.mapcalc expression=input1 = 1
r.mapcalc expression=input2 = 2
# Test the average method of r.series
r.series input=input1,input2 output=result method=average
}}}
Here the assumed workflow:
The framework will read the test script and analyse it line by line.
In case r.mapcalc is marked as critical and the
framework finds the keyword r.mapcalc in the script, appearing as
first word outside of a comment, it checks if the r.mapcalc test(s)
already run
correctly and stop the r.series test if they not. In case r.mapcalc
tests are valid it starts the r.mapcalc commands and checks there
return values. If the return values are correct, then the rest of the
script is executed. After reaching the end of this script the
framework looks for any generated data in the current mapset (raster,
raster3d, vector, color, regions, ...) and looks for corresponding
validation files in the test directory. In this case it will find the
raster maps input1, input2 and result in the current mapset and
validation.ref in the test directory. It will use r.out.ascii on
result map choosing a low precision (dp=3??) and compares the output
with result.ref which was hopefully generated using the same
precision.
This example should cover many raster and voxel test cases.
I don't normally advocate such approaches, but testing is one of those
areas which (like documentation) is much harder to get people to work
on than e.g. programming, so minimising the effort involved is
important.
Minimising the learning curve is probably even more important. If you
can get people to start writing tests, they're more likely to put in
the effort to learn the less straightforward aspects as it becomes
necessary.
Ok, i will try to summarize this approach:
The test framework will be integrated in the source code of grass and
will use the make system to execute tests.
The make system should be used to:
* run single module or library tests
* run all module (raster|vector|general|db ...) tests
* run all library tests
* run all tests (library than modules)
* in case of an all-modules-test it should run critical module tests
automatically first
Two test locations (LL and UTM?) should be generated and added to the
grass sources. The test locations provide all kind of needed test data
- raster maps of different type (elevation maps, images, maps of
CELL, FCELL and DCELL type, ...), vector maps (point, line, area,
mixed with and with out attribute data), voxel data, regions, raster
maps with different color tables, reclassified maps and so on ... .
The test data is only located in PERMANENT mapset. But the locations
should be small enough to fit in svn without performance issues.
Each module and library has its own test directory. The test
directories contain the test cases, reference text files and data for
import (for *.in.* modules). Validation of data is based on the
reference text files located in the test directories for each
module/library. Files implementing test cases must end with .sh,
reference files must end with .ref . The test cases are based on
simple shell style text files, so
[GRASS-dev] First java module with native grass7 library access
Dear Developers and Users, JFYI i am developing an object oriented wrapper library (vtk-grass-bridge) around the grass7 raster, vector, dbmi and raster3d libraries based on VTK[1], which supports now the implementation of Java based grass modules with native library access. Additionally C++ and Python are supported using the VTK wrapping approach. I have implemented a simple module which samples multiple raster maps based on the points of an input vector map.It shows how the vtk-grass-bridge works with vector, raster and dbmi access. This module is implemented in C++, Python and finally Java. Here the links: The vtk-grass-bridge main page http://code.google.com/p/vtk-grass-bridge/ The Java module v_sample_rast: http://code.google.com/p/vtk-grass-bridge/source/browse/trunk/Modules/Java/v_sample_rast.java The Python module v.sample.rast.py http://code.google.com/p/vtk-grass-bridge/source/browse/trunk/Modules/Python/v.sample.rast.py The C++ module v.sample.rast http://code.google.com/p/vtk-grass-bridge/source/browse/trunk/Modules/Cxx/v.sample.rast.cxx In case someone would like to try out the vtk-grass-bridge, here is a short introduction how to compile the vtk-grass-bridge: http://code.google.com/p/vtk-grass-bridge/wiki/HowToCompile Any feedback is very welcome. Best regards Soeren [1] http://www.vtk.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome
Hi Glynn, 2011/6/3 Glynn Clements gl...@gclements.plus.com: Soeren Gebbert wrote: i have updated the wiki and added a simple Python example of a hypothetical r.series test. Please have a look at: http://grass.osgeo.org/wiki/Test_Suite#Test_framework The sample Python code shows in principle how a test case would look like and what kind of methods should be provided by the framework. This isn't a particularly good example. For most modules, you won't be able to synthesise the reference data with r.mapcalc; you'll need to provide it. Indeed, you are right, bad example. I just tried to show some hypothetical interface methods. There should be of course default region settings and test maps available. But my main concern is that few people will bother to learn a complex test framework. If you want people to write tests, they need to be little more than sample commands. Anything else (e.g. directives which indicate how to compare the output) needs to be capable of being summarised in a dozen lines. E.g.: # Test the average method r.series input=testmap1,testmap2 output=resmap_av method=average # Test the sum method r.series input=testmap1,testmap2 output=resmap_sum method=sum # Test the max method r.series input=testmap1,testmap2 output=resmap_max method=max I was thinking about a similar approach, but the effort to parse the modules XML interface description to identify the command line arguments to compare the created data was to much effort for me. Besides that, the handling of test description, module dependencies and the comparison of multiple/timeseries outputs (r.sim.water) bothered me too. I still have no simple (interface) answers to this issues (maybe these are no issues??). The implementation of the framework which executes test scripts can be as complex as you like, but the interface (i.e. the syntax of test scripts) needs to be trivial. Well, i thought my example is trivial ... but your may be right. My perspective is a developers perspective familiar with different test frameworks. Switching to a software tester perspective is difficult for me, so many thanks for your feedback Glynn! Boilerplate should be minimised; a region should already be set, test maps should be available, parameters should have default values. Don't force the user to specify anything that the framework ought to be able to do itself. Your are completely right, i had this in mind too. I would like to put several small test locations in the grass source tree for this reason. These locations, which differ only in the coordinate reference system, should provide test maps and data for most test cases in the PERMANENT mapset. The module test mapsets are created on the fly and removed for each module test, to avoid a test location mess-up. E.g. in the above example, if a command creates a raster map named resmap_av and a file named resmap_av.ref exists, that should be sufficient for the framework to deduce that it should compare the map to the reference data using default parameters. I see, a simple but powerful approach, i have sometimes the feeling i think much to complicated. Best regards Soeren -- Glynn Clements gl...@gclements.plus.com ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome
Hi, i have updated the wiki and added a simple Python example of a hypothetical r.series test. Please have a look at: http://grass.osgeo.org/wiki/Test_Suite#Test_framework The sample Python code shows in principle how a test case would look like and what kind of methods should be provided by the framework. I have put the old but functional test suite for grass6 into the google code repository: http://code.google.com/p/grass6-test-suite/ Maybe the documentation is of interest for you too: http://grass6-test-suite.googlecode.com/files/GRASS-Testsuite-Presentation.pdf Here a quick start: http://code.google.com/p/grass6-test-suite/wiki/QuickStart Best Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome
Hi, [snip] It saves us writing lots of functions that would compare datatypes; but OTOH it would add dependencies of other GRASS modules. Then, it will be necessary (or at least better) to test first the modules without dependencies. We would need to check if a test includes more than one GRASS module... or just let all tests run and fail, and then understand which modules are actually broken, and causing other (correct) modules' test to fail. I don't think that there's any way around this. Few GRASS modules are useful in isolation. On the positive side, modules which are essential for tests (e.g. g.region, *.out.ascii) etc are unlikely to be changed regularly. Indeed, we will need several modules for data generation, region settings and data output. The testsuite for grass 6.4 uses g.region, r/r3/v.out.ascii, r.mapcalc and several other modules. We chose Python, as Sören already wrote tests as shell scripts and found them too cumbersome and difficult to mantain. But is this because you're trying to perform analysis of the results within the test scripts? In many cases there are situtation where you will need to perform several independent tests for a single module with several kinds of outputs each test: * Module output on stdout * single or multiple raster and voxel maps * single or multiple vector maps * mixed raster and vector output * and so on The framework should be able to handle such output automatically i.e: * parsing the stout and compare it with validation data (text file, MD5 checksum, ...) * Generation of ASCII output from raster, vector and voxel maps to compare with validation data (text file, MD5 checksum, ...) To make it as easy as possible the writer of the test should only provide the command line of the test and data generation and the validation data. I tried to implement this using the shell approach, but because of the limitation of the shell this was much to complex. I am not sure if this kind of data handling can be done using only the make system? The next reason is the generation of HTML representation of the test results for web access. The idea is to run tests on several different configured server after compilation. The result of these tests will be posted in the web. This is an important feature, because the developer will see what modules/libraries failed the tests on which machine configuration. Have a look at the VTK dashboard[1] and the result of the old testsuite [2]. The old testsuite has the possibility to run each module in a valgrind environment to find memory leaks. It is able to validate outputs of different kind against MD5 checksums. It logs stdout and stderr of the module and provides this data as HTML output. [3] It links to the test description [4]. Using python the implementation will be much easier as with shell and the related unix toolset. Best regards Soeren [1] http://www.cdash.org/CDash/index.php?project=VTK [2] http://www-pool.math.tu-berlin.de/~soeren/grass/GRASS_TestSuite/html_grass-6.4/ [3] http://www-pool.math.tu-berlin.de/~soeren/grass/GRASS_TestSuite/html_grass-6.4/log.html [4] http://www-pool.math.tu-berlin.de/~soeren/grass/GRASS_TestSuite/html_grass-6.4//r.gwflow-test.sh.html -- Glynn Clements gl...@gclements.plus.com ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] GRASS GIS 7 and QGIS connected using the Web Processing Service (WPS)
Hi Moritz, the problem may be related to the QGIS WPS plugin version. The plugin is still in an experimental state and under active development, but the latest SVN version should be usable. Which version of the plugin do you use? Best regards Soeren 2011/4/14 Moritz Lennert mlenn...@club.worldonline.be: Soeren, Thanks for the great work on WPS integration ! On 28/01/11 11:44, Soeren Gebbert wrote: How to attach GRASS modules to PyWPS using the wps-grass-bridge is documented here: http://code.google.com/p/wps-grass-bridge/wiki/PyWPS_Integration Well, the code isn't stable yet but it demonstrates the possibilities of this approach. I've tried following the above instructions, and the server does give a correct answer for GetCapabilities. I can also see the list of modules in the QGIS WPS plugin and get the GUI of a module. However, once I run the module I always get the following error, whatever the module I try: wps:ProcessFailed wps:ExceptionReport ows:Exception exceptionCode=NoApplicableCode ows:ExceptionTextargument 1 must be string or buffer, not None/ows:ExceptionText /ows:Exception /wps:ExceptionReport /wps:ProcessFailed Any idea what could be wrong ? I can't find a way to see what URL the QGIS plugin sends to the server. Moritz ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] GRASS GIS 7 and QGIS connected using the Web Processing Service (WPS)
Hi Moritz,
i can reproduce this bug using the SVN revision 63. I have applied a
patch which should solve the problem and which will add new shiny
features (i was to lazy to remove the features from the patch).
Please apply the patch to the latest svn version of the plugin.
Best regards
Soeren
2011/4/15 Moritz Lennert mlenn...@club.worldonline.be:
On 15/04/11 09:20, Soeren Gebbert wrote:
Hi Moritz,
the problem may be related to the QGIS WPS plugin version. The plugin
is still in an experimental state and under active development, but
the latest SVN version should be usable. Which version of the plugin
do you use?
0.5.
Moritz
Best regards
Soeren
2011/4/14 Moritz Lennertmlenn...@club.worldonline.be:
Soeren,
Thanks for the great work on WPS integration !
On 28/01/11 11:44, Soeren Gebbert wrote:
How to attach GRASS modules to PyWPS using the wps-grass-bridge is
documented here:
http://code.google.com/p/wps-grass-bridge/wiki/PyWPS_Integration
Well, the code isn't stable yet but it demonstrates the possibilities
of this approach.
I've tried following the above instructions, and the server does give a
correct answer for GetCapabilities. I can also see the list of modules in
the QGIS WPS plugin and get the GUI of a module. However, once I run the
module I always get the following error, whatever the module I try:
wps:ProcessFailed
wps:ExceptionReport
ows:Exception exceptionCode=NoApplicableCode
ows:ExceptionTextargument 1 must be string or
buffer,
not None/ows:ExceptionText
/ows:Exception
/wps:ExceptionReport
/wps:ProcessFailed
Any idea what could be wrong ? I can't find a way to see what URL the
QGIS
plugin sends to the server.
Moritz
Index: qgswpstools.py
===
--- qgswpstools.py (Revision 63)
+++ qgswpstools.py (Arbeitskopie)
@@ -211,14 +211,16 @@
##
- def decodeBase64(self, infileName, mimeType=image/tiff):
+ def decodeBase64(self, infileName, mimeType=image/tiff, outfileName=None):
# This is till untested
# TODO: test it
try:
# User project dir
-#filename = QFileDialog.getSaveFileName(None, Chose filename for output file, /home/soeren);
-filename = tempfile.mktemp(prefix=base64)
+if outfileName == None:
+filename = tempfile.mktemp(prefix=base64)
+else:
+filename = outfileName
infile = open(infileName)
outfile = open(filename, 'w')
base64.decode(infile,outfile)
@@ -264,9 +266,15 @@
writer.addFeature(feat)
del writer
-
+
+# Check in case a .g was attached to the tempfile
+if os.path.isfile(tmpFile + .g) == True:
+ # The gdal creator attached a .g to the temp file
+ tmpFile = tmpFile + .g
+
myFile = QFile(tmpFile)
if (not myFile.open(QIODevice.ReadOnly | QIODevice.Text)):
+ print Unable to open file + tmpFile
pass
myGML = QTextStream(myFile)
Index: qgswps.py
===
--- qgswps.py (Revision 63)
+++ qgswps.py (Arbeitskopie)
@@ -30,10 +30,12 @@
from httplib import *
from urlparse import urlparse
import os, sys, string, tempfile, urllib2, urllib, mimetypes
+import time
# initialize Qt resources from file resources.py
import resources
+# Set this FLAG true to have Debug output, The request and the response will be saved in /tmp
DEBUG = False
# Our main class for the plugin
@@ -44,6 +46,8 @@
self.iface = iface
self.minimumRevision = 12026
self.localePath =
+# Switch between downloading the data via a reference or to have embedded data in the WPS responce.
+self.asReference = True
#Initialise the translation environment
userPluginPath = QFileInfo(QgsApplication.qgisUserDbFilePath()).path()+/python/plugins/wps
@@ -171,7 +175,8 @@
self.complexInputTextBoxList = [] # complex inpt of type text/plain
self.literalInputComboBoxList = [] # literal value list with selectable answers
self.literalInputLineEditList = [] # literal value list with single text line input
-self.complexOutputComboBoxList = [] # list combo box
+self.complexOutputComboBoxList = [] # list combo box with output layer names
+self.complexOutputNameList = {} # list output names in the file system
self.inputDataTypeList = {}
self.inputsMetaInfo = {} # dictionary for input metainfo, key is the input identifier
self.outputsMetaInfo = {} # dictionary for output metainfo, key is the output identifier
@@ -362,8 +367,10 @@
self.outputsMetaInfo[outputIdentifier] = supportedcomplexOutputFormat
self.outputDataTypeList[outputIdentifier] = complexOutputFormat
-widget, comboBox
Re: [GRASS-dev] GRASS GIS 7 and QGIS connected using the Web Processing Service (WPS)
Hello Moritz,
the problem is the temporal GML file generation in qgswpstools.py.
This piece python code should solve the problem, which is also
included in the patch:
qgswpstools.py at line ~230
# Check in case a .g was attached to the tempfile
if os.path.isfile(tmpFile + .g) == True:
tmpFile = tmpFile + .g
This error should only occur with vector data. Raster data should work.
Best regards
Soeren
2011/4/15 Moritz Lennert mlenn...@club.worldonline.be:
Hi Soeren,
[cc to Horst Düster - I'm not subscribed to the QGIS lists, trying to limit
my flow of mails, but we can take it over there if you prefer, but I want to
make sure that this is really a plugin problem and not my bad usage of your
wps-grass bridge]
On 15/04/11 09:40, Moritz Lennert wrote:
On 15/04/11 09:20, Soeren Gebbert wrote:
Hi Moritz,
the problem may be related to the QGIS WPS plugin version. The plugin
is still in an experimental state and under active development, but
the latest SVN version should be usable. Which version of the plugin
do you use?
0.5.
0.5.0 to be precice.
I've found the DEBUG option in the source code. Now I can see the request
XML and the problem seems to be this (trying to launch a simple v.perturb on
the busstopsall map of the shapefile collection of the nc_spm grass demo
data):
ows:Identifierinput/ows:Identifier
ows:Titleinput/ows:Title
wps:Data
wps:ComplexData mimeType=text/xml
schema=http://schemas.opengis.net/gml/2.1.2/feature.xsd;
enconding=utf-8/wps:ComplexData
/wps:Data
/wps:Input
i.e. the input data info is empty.
The same seems to be true for the output data:
ows:Identifierinput/ows:Identifier
ows:Titleinput/ows:Title
wps:Data
wps:ComplexData mimeType=text/xml
schema=http://schemas.opengis.net/gml/2.1.2/feature.xsd;
enconding=utf-8/wps:ComplexData
/wps:Data
/wps:Input
Whereas the 'parameter' option contains the value I gave:
wps:Input
ows:Identifierparameters/ows:Identifier
ows:Titleparameters/ows:Title
wps:Data
wps:LiteralData20/wps:LiteralData
/wps:Data
/wps:Input
Here some info about my config:
pywps.cfg:
[grass]
path=/home/mlennert/SRC/GRASS/grass_trunk/dist.i686-pc-linux-gnu/bin:/home/mlennert/SRC/GRASS/grass_trunk/dist.i686-pc-linux-gnu/scripts
addonPath=
version=7.0svn
gui=text
gisbase=/home/mlennert/SRC/GRASS/grass_trunk/dist.i686-pc-linux-gnu
ldLibraryPath=/home/mlennert/SRC/GRASS/grass_trunk/dist.i686-pc-linux-gnu/lib
gisdbase=/home/mlennert/WWW/pywps/grassdata
pywps.cgi:
export PYWPS_CFG=/home/mlennert/WWW/pywps/pywps.cfg
export PYWPS_PROCESSES=/home/mlennert/SRC/WPS_PROCESS
and in WPS_PROCESSES I have all the files from your original
wps-grass-bridge svn directory:
GlobalGrassSettings.py
GlobalGrassSettings.pyc
GrassXMLtoPyWPS.py
GrassXMLtoYAML.py
GrassXMLtoZCFG.py
PyWPSGrassModuleStarter.py
the gms and wpsXML directories (I erased the zoo-related) and a long list of
grass modules descriptors, of which just the v.perturb one as example:
# #
# This process was generated using GrassXMLtoPyWPS #
# Author: Soeren Gebbert #
# Mail: soerengebbert at googlemail dot com #
# #
from pywps.Process import WPSProcess
from PyWPSGrassModuleStarter import PyWPSGrassModuleStarter
class v_perturb(WPSProcess):
def __init__(self):
WPSProcess.__init__(self, identifier = 'v.perturb', title = 'Random
location perturbations of vector points.', version = 1, statusSupported =
True, storeSupported = True, metadata = [{'type': 'simple', 'title':
'vector'}, {'type': 'simple', 'title': 'statistics'}], abstract =
'http://grass.osgeo.org/grass70/manuals/html70_user/v.perturb.html')
# Literal and complex inputs
self.addComplexInput(identifier = 'input', title = 'Data source for
direct OGR access', minOccurs = 1, maxOccurs = 1, formats = [{'mimeType':
'text/xml', 'schema': 'http://schemas.opengis.net/gml/2.1.2/feature.xsd',
'encoding': 'UTF-8'}, {'mimeType': 'text/xml', 'schema':
'http://schemas.opengis.net/kml/2.2.0/ogckml22.xsd', 'encoding': 'UTF-8'},
{'mimeType': 'application/dgn', 'schema': 'None', 'encoding': 'None'},
{'mimeType': 'application/shp', 'schema': 'None', 'encoding': 'None'},
{'mimeType': 'application/x-zipped-shp', 'schema': 'None', 'encoding':
'None'}])
self.addLiteralInput(identifier = 'layer', title = 'A single vector map
can be connected to multiple database tables. This number determines which
table to use. Layer name for direct OGR access.', minOccurs = 1, maxOccurs =
1, type = type(string), default = -1)
self.addLiteralInput(identifier = 'distribution', title = 'Distribution
of perturbation', minOccurs = 0, maxOccurs = 1, type = type(string),
default = uniform, allowedValues = ['uniform', 'normal'])
self.addLiteralInput(identifier = 'parameters', title = 'If the
distribution is uniform, only one parameter, the maximum
Re: [GRASS-dev] GRASS GIS 7 and QGIS connected using the Web Processing Service (WPS)
Hi Moritz, please use the latest pywps and wps-grass-bridge svn versions. Can you please verify if the mime type and the schema of the input vector are set in the WPS request generated by qgis? More below: 2011/4/15 Moritz Lennert mlenn...@club.worldonline.be: On 15/04/11 10:59, Soeren Gebbert wrote: Hello Moritz, the problem is the temporal GML file generation in qgswpstools.py. This piece python code should solve the problem, which is also included in the patch: qgswpstools.py at line ~230 # Check in case a .g was attached to the tempfile if os.path.isfile(tmpFile + .g) == True: tmpFile = tmpFile + .g Thanks for the patch ! I hope it works correctly. - Running the grass processes however gave me the same error message. This was using grass7 trunk and qgis trunk. - I've installed the Debian stable 1.4 version of QGIS. - I set up a simple pywps instance with only two of the example processes from the source code: ultimatequestionprocess and buffer. Both run, but the buffer process gives completely wrong buffers (much too large - maybe it is working in degrees instead of meters, even though the QGIS project has units meters and the correct projection defined). Unfortunately i am not familiar with the default processes of PyWPS. - Trying to run v.perturb or v.buffer without or with your patch applied gives me ows:Exception exceptionCode=NoApplicableCode ows:ExceptionTextFailed to execute WPS process [v.perturb]: Unsupported MimeType: text/xml. Unable to create input location from input ./pywpsInput0DORwt/ows:ExceptionText /ows:Exception This error is related to the wps-grass-bridge. It requires the definition of the mime type and the schema in case of XML data to identify GML or KML data. Because QGIS always produce GML this error should not occur in case the mime types are set correctly in the request. In case the request is correct the error is located in the wps-grass-bridge and i can fix it in the latest svn version. I'm currently recompiling a fresh a fresh checkout from qgis trunk in order to test with that. Thanks for your effort and testing. The main problem is that all required software is in experimental state so the identification of the problem is quite hard. Best regards Soeren I'll keep you posted. Moritz ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] GRASS GIS 7 and QGIS connected using the Web Processing Service (WPS)
Hello Moritz, 2011/4/15 Moritz Lennert mlenn...@club.worldonline.be: On 15/04/11 12:56, Soeren Gebbert wrote: Hi Moritz, please use the latest pywps and wps-grass-bridge svn versions. That seems to have solved it. I had used the install script for pywps, but this installed the FOSS4G2010 branch. Now I've installed the latest trunk and everything seems to work. Sorry for the noise ! Thanks for testing this solution, feedback is very important! A few rapid remarks for future development from my first usage: - Flags are all True by default. I think they should be False by default. I see that e.g. for v.buffer, the v_buffer.py process file contains: self.addLiteralInput(identifier = '-s', title = 'Make outside corners straight', minOccurs = 0, maxOccurs = 1, type = type(True), default = False, allowedValues = [True, False]) so one would expect it to be False by default. This probably means that the problem is in the QGIS plugin, or ? - It is not clear where the order of parameter in the GUI comes from: notably those that are not directly defined by the model (e.g. ns and ew resolution), but it would make for a better experience to have them more logically grouped. When generating the GRASS module WPS XML process description and based upon this the PyWPS processes i had in mind that for literal boolean inputs False should be the default. And i had in mind that the input order is adapted from the GRASS command line interface, i.e: the ns and ew resolution as well as the band number are the last entries for inputs. But the WPS server seems to have its own order logic (dicts?) and i do not want to implement input/output ordering in QGIS plugin because the logic of the order of is GRASS module specific. - When running v.buffer, I get the following error: ows:Exception exceptionCode=NoApplicableCode ows:ExceptionTextFailed to execute WPS process [v.buffer]: Error while executing the grass module. The following error message was logged: Buffering lines... 2%...5%...etc [snip] Deleting boundaries... 2%... 5%...etc /ows:ExceptionText /ows:Exception So, it seems to catch all the output of G_percent and G_message and interpret it as an error... This should not happen. In case a module does not run correctly (decision is made based on the return value) and exception will be raised and the stderr output of the module will be attached. Seems like a bug in wps-grass-bridge to me. - The WPS plugin shouldn't be modular, i.e. you should be able to use other commands with the plugin open (e.g. add another layer) Yes indeed, this is an important missing feature. There are many features missing i.e: non blocking waiting for long running processes, a status bar for long running processes, efficient vector data handling, GUI enhancement, better text/plain output handling, better error handling and so on ... . Unfortunately i am not a QGIS nor a QT4 programmer, so i do not have the knowledge to implement such important features by myself, so any help is appreciated. Best regards Soeren Once again, thanks for all the work and for your support ! Moritz ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Development of an Open Cloud GIS with GRASS GIS, PyWPS, the wps-grass-bridge and QGIS
Hello, Just to inform you: GRASS GIS 7 is running in the cloud as web processing service backend. Have a look at: http://www.youtube.com/watch_popup?v=jg2pb_Xjq8Yvq=hd720#t=19 I have set up an Open Cloud GIS in a private amazon compatible cloud environment using: - Ubuntu 10.04 LTS and 10.10 cloud server edition - Eucalyptus Cloud - GRASS GIS 7 latest svn - PyWPS latest svn - wps-grass-bridge latest svn - QGIS 1.7 with a modified QWPS plugin Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] commits to tags, r45707
Hi, 2011/3/27 Hamish hamis...@yahoo.com: Hi, in svn tagged releases, as snapshots in time, would ideally be set to be read only/svn admin only, but as it is they are regular files and rely on discipline not to be modified. unfortunately r45707 was committed to the wrong place and now must be carefully reverted. https://trac.osgeo.org/grass/changeset/45707 I am really sorry for that, it was my fault. I made my changes within the wrong source directory and committed it. not sure of the best approach to wind that back. I hopefully reverted it correctly: https://trac.osgeo.org/grass/changeset/45793 Best regards Soeren thanks, Hamish ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] SIMWE update in grass7
Hi, i have reactivated a long disabled feature in the grass7 SIMWE library. Now the generation of walker vector point output is available again. I have updated the spearfish example located in the r.sim.water directory, using paraview makes it very easy to visualize the time step output of the spearfish example. Please test the modification and report any issue. Still missing is the support of observation points logging the water depth and discharge and a general code cleanup and re-fracturing. Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: GRASS GIS 7 and QGIS connected using the Web Processing Service (WPS)
Hi, i would like to inform you that GRASS GIS 7 is now available via the ZOO WPS server using the wps-grass-bridge. Gérald Fenoy the main developer of the ZOO project has made a nice video using QGIS with WPS plugin: http://www.youtube.com/watch?v=DzgFIox3FA8 The ZOO project: http://zoo-project.org/ Now hundreds of GRASS GIS 7 vector and raster modules can be attached out of the box to ZOO WPS, PyWPS and 52N WPS using the wps-grass-bridge. :) Best regards Soeren 2011/1/28 Soeren Gebbert soerengebb...@googlemail.com: Dear developers, i would just like to inform you about the latest development in WPS, GRASS GIS 7 and QGIS. Using the latest svn versions of GRASS GIS 7, PyWPS, wps-grass-bridge and the WPS Plugin from Horst Düster for QGIS it is possible to run more than hundred GRASS modules from QGIS out of the box. Have a look at this simple demo video: http://www.youtube.com/watch?v=zCnax9w1bCs How to attach GRASS modules to PyWPS using the wps-grass-bridge is documented here: http://code.google.com/p/wps-grass-bridge/wiki/PyWPS_Integration Well, the code isn't stable yet but it demonstrates the possibilities of this approach. Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] GRASS GIS 7 and QGIS connected using the Web Processing Service (WPS)
Dear developers, i would just like to inform you about the latest development in WPS, GRASS GIS 7 and QGIS. Using the latest svn versions of GRASS GIS 7, PyWPS, wps-grass-bridge and the WPS Plugin from Horst Düster for QGIS it is possible to run more than hundred GRASS modules from QGIS out of the box. Have a look at this simple demo video: http://www.youtube.com/watch?v=zCnax9w1bCs How to attach GRASS modules to PyWPS using the wps-grass-bridge is documented here: http://code.google.com/p/wps-grass-bridge/wiki/PyWPS_Integration Well, the code isn't stable yet but it demonstrates the possibilities of this approach. Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Recent changes in v.out.ogr grass7 trunk
Hi Martin, thanks for the patch it works for me now. Best Soeren 2011/1/25 Martin Landa landa.mar...@gmail.com: Hi, 2011/1/23 Soeren Gebbert soerengebb...@googlemail.com: because there was no response how to solve this in code, i would like to apply the following patch to main.c to fix the overwrite/append issue in v.out.ogr: right, it's a bug. Please try out r45180. Martin -- Martin Landa landa.martin gmail.com * http://geo.fsv.cvut.cz/~landa ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Recent changes in v.out.ogr grass7 trunk
Hi,
because there was no response how to solve this in code, i would like
to apply the following patch to main.c to fix the overwrite/append
issue in v.out.ogr:
Index: main.c
===
--- main.c (Revision 45168)
+++ main.c (Arbeitskopie)
@@ -408,9 +408,8 @@
G_fatal_error(_(OGR driver %s not found), options.format-answer);
Ogr_driver = OGRGetDriver(drn);
-overwrite = G_check_overwrite(argc, argv);
-if (overwrite || flags.append-answer) {
- G_verbose_message(_(Overwrite/Append existing layer requires
+if (flags.append-answer) {
+ G_verbose_message(_(Append existing layer requires
opening data source in update mode,
forcing '-%c' flag),
flags.update-key);
flags.update-answer = TRUE;
@@ -433,6 +432,7 @@
options.dsn-answer);
/* check if OGR layer exists */
+overwrite = G_check_overwrite(argc, argv);
if (flags.append-answer || overwrite) {
if (OGR_DS_GetLayerByName(Ogr_ds, options.layer-answer)) {
if (!overwrite !flags.append-answer) {
Best regards
Soeren
2011/1/13 Benjamin Ducke benjamin.du...@oxfordarch.co.uk:
Yes, that would be a problem.
I think here the semantics of the GRASS --overwrite flag
collide with the separate OGR concepts of datasource
and layer. In the case of GRASS maps, the meaning is
always clear: overwrite existing map. But OGR output
formats can be more complex. E.g. the user might want
to overwrite one of many spatial tables (OGR layers)
in an existing Spatialite DBMS (OGR datasource). For this to
work, the datasource must of course exist.
So I think we may need two different overwrite operations
in the special case of v.out.ogr:
one for OGR layers and one for OGR datasources?
Ben
On 12/18/2010 07:52 PM, Soeren Gebbert wrote:
Hi,
i am facing a problem with v.out.ogr which was introduced in revision
r44484 implementing the
ability of v.out.ogr to update existing files.
I have a python script which sets the overwrite flag as default to
True for all modules it calls (in
gisrc), to avoid overwrite errors when working in temporally created
grass locations. This works fine for all grass modules.
Unfortunately the last changes in v.out.ogr switches from overwrite
mode to update mode (main.c line 285) and expects an existing file.
Therefor v.out.ogr dies with the error message: ERROR: Unable to open
OGR data source '/tmp/voronoi_diagram.kml' in case there is no file.
IMHO using the overwrite flag should not fail in case there is no file
to overwrite. Updating files should be managed with a special flag and
should not depend on the overwrite flag.
I am open for suggestions and discussion. :)
Best regards
Soeren
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
--
Files attached to this email may be in ISO 26300 format (OASIS Open Document
Format). If you have difficulty opening them, please visit
http://iso26300.info for more information.
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] adding 2 more modules to G7 SVNimagery
Hello Yann, while adding new i.eb.* modules it would be great if you can check this ticket: http://trac.osgeo.org/grass/ticket/899 Maybe it is obsolete now? Best regards Soeren 2011/1/23 Yann Chemin yann.che...@gmail.com: Yes theoretically same for i.eb.* modules. Practically, it will be some work and maybe creating maintenance trouble. When I was thinking about i.eb.toolbox two issues were there first: 1-Coding blocks from each module to be kept separated somehow for maintenance/bug fixing reasons 2-possibility to have a module tab dedicated to each type of tool, with the possibility of input parameters to appear in two or more tabs. Rest is sweat and debugging and unforeseen problems I would appreciate comments from experienced coders about integrating many tools in one. Yann Yann Chemin www.csu.edu.au On 23/01/2011, at 21:05, Markus Neteler nete...@osgeo.org wrote: Hi Yann, On 1/23/11, Yann Chemin yann.che...@gmail.com wrote: Hello, I would like to complete the i.evapo.* series of modules from grass-addons/imagery/gipe with these two last ones. i.evapo.potrad: potential radiation (Bastiaanssen, 1995) i.evapo.senay: Actual evapotranspiration (Senay, 2007) would it be possible to make one single i.evapo module and manage the methods via method=xx parameter? Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Recent changes in v.out.ogr grass7 trunk
Hi, i am facing a problem with v.out.ogr which was introduced in revision r44484 implementing the ability of v.out.ogr to update existing files. I have a python script which sets the overwrite flag as default to True for all modules it calls (in gisrc), to avoid overwrite errors when working in temporally created grass locations. This works fine for all grass modules. Unfortunately the last changes in v.out.ogr switches from overwrite mode to update mode (main.c line 285) and expects an existing file. Therefor v.out.ogr dies with the error message: ERROR: Unable to open OGR data source '/tmp/voronoi_diagram.kml' in case there is no file. IMHO using the overwrite flag should not fail in case there is no file to overwrite. Updating files should be managed with a special flag and should not depend on the overwrite flag. I am open for suggestions and discussion. :) Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] swig
Hello Mohammed, maybe you can use the vtk-grass-bridge? The vtk-grass-bridge is a VTK/C++ wrapper around grass gis to provide direct access to grass library in C++, Python and Java. So many GRASS GIS library functions are available using this interface in Python, currently the raster-, vector-, voxel-, gis- and dbmi-libraries. Have a look at: http://code.google.com/p/vtk-grass-bridge/ Here are some unit tests which show how to handle raster, vector and the dbmi interface using the vtk-grass-bridge: This is a test for raster map read - write access: http://code.google.com/p/vtk-grass-bridge/source/browse/trunk/Raster/Testing/Python/GRASSRasterMapReaderWriterTest.py This is a test for vector read access: http://code.google.com/p/vtk-grass-bridge/source/browse/trunk/Vector/Testing/Python/GRASSVectorMapTopoReader.py This test shows how to use the dbmi interface: http://code.google.com/p/vtk-grass-bridge/source/browse/trunk/DBMI/Testing/Python/vtkGRASSDbmiInterfaceReaderTest.py This test shows the command line interface functionality: http://code.google.com/p/vtk-grass-bridge/source/browse/trunk/Common/Testing/Python/GRASSModuleTest.py Here the C++ code of a VTK class which reads grass vector maps as vtkPolyData and adds vector table columns as cell data. I hope this may help you a bit. But i must warn you, GRASS GIS library and so the vtk-grass-bridge is not designed to implement persistent applications like a GUI using the library functions directly. Feel free to ask any questions about the vtk-grass-bridge. Best regards Soeren 2010/9/21 Mohammed Rashad mohammedrasha...@gmail.com: Hi all, sorry for off-topic. kindly forgive me I would like to make a python gui toolkit based on FOX C++ Gui toolkit. I had some basic knowledge in swig based on tutorials given in the swig.org homepage. But my knowledge is not enough to make swig files for a whole GUI toolkit. So anyone please give me some links on making swig interface for python. Is there any good books for the making of swig interface. Treat me as a newbie to swig. Is there any users who develop swig interface for python based on GRASS C/C++ library? All I need is good start on making swig interfaces Any help will be greatly appreciated. Thanks in advance. -- Rashad -- Rashad ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] r.sim.water at debug level 5
Hello, IMHO your region resolution is much to large for r.sim.water: 33008x33443 cells - 1.1 billion cells. You can use a coarse resolution or you can split your dataset in smaller pieces for computation. r.sim.water loads all raster maps into your memory and creates additionally a maximum of 700 walkers. I would suggest to use about 100 cells for computation. Best Soeren 2010/9/8 Hanlie Pretorius hanlie.pretor...@gmail.com: Hi, I've been posting about problems with r.sim.water. I have compiled GRASS64 SVN with debugging symbols and set the GRASS DEBUG environment variable to level 5. I'm also running GRASS from gdb. I ran r.sim.water on a region with 1103886544 cells and it bombed out quickly. The output was: - GRASS 6.4.0svn (c83a_tm_29deg_e):~ r.sim.water --overwrite elevin=c83a_dem_...@permanent dxin=c83a_dem_25m_clipped...@rsimwater_toetse dyin=c83a_dem_25m_clipped...@rsimwater_toetse rain=rain...@rsimwater_toetse infil=infil...@rsimwater_toetse manin=manning_0...@rsimwater_toetse depth=depth_1mres_rain50_infil0_nite...@rsimwater_toetse disch=disch_1mres_rain50_infil0_nite...@rsimwater_toetse D2/5: G__read_Cell_head D2/5: G__read_Cell_head_array D3/5: region item: proj: 99 D3/5: region item: zone: 0 D3/5: region item: north: -319.69982072 D3/5: region item: south: -3155672.30728643 D3/5: region item: east: -33474.90929218 D3/5: region item: west: -66483.27873573 D3/5: region item: cols: 33008 D3/5: region item: rows: 33443 D3/5: region item: e-w resol: 1.1119 D3/5: region item: n-s resol: 0.8826 D3/5: region item: top: 0.5 D3/5: region item: bottom: -0.5 D3/5: region item: cols3: 33008 D3/5: region item: rows3: 33443 D3/5: region item: depths: 1 D3/5: region item: e-w resol3: 1.1119 D3/5: region item: n-s resol3: 0.8826 D3/5: region item: t-b resol: 1 D3/5: rain_val is set to: -999.99 D3/5: manin_val is set to: -999.99 D3/5: infil_val is set to: -999.99 default nwalk=-2087194208, rwalk=2207773088.00 Killed - Since GRASS itself doesn't exit abnormally, gdb doesn't seem to notice that anything is wrong. Is there anywhere else (a log file perhaps) where I can try and find out what is going on? Thanks Hanlie ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] r.sim.water at debug level 5
Hello, 2010/9/8 Hanlie Pretorius hanlie.pretor...@gmail.com: Thanks for the reply, Soeren. I have had similar results on rasters with fewer than 1 million cells. The process runs for a litte longer but then also gets killed. I made the resolution very fine on purpose because I was trying to crash r.sim.water to get to the debugging messages. Does that make sense? Unfortunately not really. The process get killed because it tries to allocate much to much memory and this may be not related to the problems you face when computing with a fewer number of cells. The debug messages will always be printed if the environmental variable was set correctly. Maybe using breakpoints while debugging will point to the problem? Best Soeren 2010/9/8, Soeren Gebbert soerengebb...@googlemail.com: Hello, IMHO your region resolution is much to large for r.sim.water: 33008x33443 cells - 1.1 billion cells. You can use a coarse resolution or you can split your dataset in smaller pieces for computation. r.sim.water loads all raster maps into your memory and creates additionally a maximum of 700 walkers. I would suggest to use about 100 cells for computation. Best Soeren 2010/9/8 Hanlie Pretorius hanlie.pretor...@gmail.com: Hi, I've been posting about problems with r.sim.water. I have compiled GRASS64 SVN with debugging symbols and set the GRASS DEBUG environment variable to level 5. I'm also running GRASS from gdb. I ran r.sim.water on a region with 1103886544 cells and it bombed out quickly. The output was: - GRASS 6.4.0svn (c83a_tm_29deg_e):~ r.sim.water --overwrite elevin=c83a_dem_...@permanent dxin=c83a_dem_25m_clipped...@rsimwater_toetse dyin=c83a_dem_25m_clipped...@rsimwater_toetse rain=rain...@rsimwater_toetse infil=infil...@rsimwater_toetse manin=manning_0...@rsimwater_toetse depth=depth_1mres_rain50_infil0_nite...@rsimwater_toetse disch=disch_1mres_rain50_infil0_nite...@rsimwater_toetse D2/5: G__read_Cell_head D2/5: G__read_Cell_head_array D3/5: region item: proj: 99 D3/5: region item: zone: 0 D3/5: region item: north: -319.69982072 D3/5: region item: south: -3155672.30728643 D3/5: region item: east: -33474.90929218 D3/5: region item: west: -66483.27873573 D3/5: region item: cols: 33008 D3/5: region item: rows: 33443 D3/5: region item: e-w resol: 1.1119 D3/5: region item: n-s resol: 0.8826 D3/5: region item: top: 0.5 D3/5: region item: bottom: -0.5 D3/5: region item: cols3: 33008 D3/5: region item: rows3: 33443 D3/5: region item: depths: 1 D3/5: region item: e-w resol3: 1.1119 D3/5: region item: n-s resol3: 0.8826 D3/5: region item: t-b resol: 1 D3/5: rain_val is set to: -999.99 D3/5: manin_val is set to: -999.99 D3/5: infil_val is set to: -999.99 default nwalk=-2087194208, rwalk=2207773088.00 Killed - Since GRASS itself doesn't exit abnormally, gdb doesn't seem to notice that anything is wrong. Is there anywhere else (a log file perhaps) where I can try and find out what is going on? Thanks Hanlie ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Voxel support in vtk-grass-bridge
Hello, JFYI, i have added voxel support to the vtk-grass-bridge[1]. Now, besides of raster and vector data, grass voxel data can be accessed from C++, Python and Java using a simple object oriented approach. It is possible to read grass voxel data directly as vtkImage data and to write vtkImage data as grass voxel data. Hence all vtkImage filtering classes can be used to process gis grass voxel data, here an example module which reads and writes grass voxel data and uses the vtkImageGaussianSmooth class from VTK to process the data: http://code.google.com/p/vtk-grass-bridge/source/browse/trunk/Modules/Python/r3.gauss.smooth.py Because of grass vector read and write support in vtk-grass-bridge, any kind of grass vector data can be converted into grass voxel data using VTK filtering classes. The vtk-grass-bridge is still under developing and not stable, but ready for experiments with grass and VTK. Good night and good luck Soeren [1] http://code.google.com/p/vtk-grass-bridge ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] grass + vtk , which vtk version
Hello Massimo, i'm the developer of the vtk-grass-bridge, you can write any questions about the project directly to me. :) 2010/8/19 Massimo Di Stefano massimodisa...@gmail.com: Hi All, i'm intersted to use the vtk-grass-bridge [1] , i've few questins about it . Can i use the latest stable release : VTK - 5.6 It is fully compatible. instead of the suggested version ( VTK - 5.4) ? ... have you any clue if this beatifull work will be ported to run under grass6.x ? Unfortunately, i do not have the time to support booth versions of grass, so grass7 is the only option. Please notice the vtk-grass-bridge is a work in progress and not stable enough to be used in production environments. There is also a massive lack of examples, tutorials and documentation. Only the source code is partly well documented. But feel free to try it out and don't bother to ask any questions. Best regards Soeren thanks! Massimo. [1] http://code.google.com/p/vtk-grass-bridge/___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] grass7 on mac OSX
Hello Massimo, can you please make a svn update?The gmath function G_math_backward_solving has been renamed into G_math_backward_substitution a while ago. Maybe something got mixed up? Best Soeren 2010/5/27 Massimo Di Stefano massimodisa...@gmail.com: i see ctypesgen is included in grass7 (not ctype itself) to check if my local copy of ctypesgen is the reason of the error, i temporary moved ctypesgen from my computer : sudo mv /Library/Python/2.6/site-packages/ctypesgencore /Library/Python/2.6/site-packages/ctypesgencore__ sudo mv /usr/local/bin/ctypesgen.py /usr/local/bin/ctypesgen__.py i can't remove ctype, is is used by other apps then in the grass_trunk source : make distclean svn up ./configure ... make but the problem persists : In [2]: import os, sys In [3]: from grass.lib import raster Traceback (most recent call last): File ipython console, line 1, in module File /Users/Shared/source/grass_trunk/dist.i386-apple-darwin10.3.0/etc/python/grass/lib/__init__.py, line 3, in module # $Id$ File /Users/Shared/source/grass_trunk/dist.i386-apple-darwin10.3.0/etc/python/grass/lib/math.py, line 7, in module ImportError: dlopen(/Applications/GRASS-7.0.app/Contents/MacOS/etc/python/grass/lib/_math.so, 2): Symbol not found: _G_math_backward_solving Referenced from: /Applications/GRASS-7.0.app/Contents/MacOS/etc/python/grass/lib/_math.so Expected in: /Applications/GRASS-7.0.app/Contents/MacOS/lib/libgrass_gmath.dylib in /Applications/GRASS-7.0.app/Contents/MacOS/etc/python/grass/lib/_math.so Massimo. Il giorno 27/mag/2010, alle ore 06.28, Markus Neteler ha scritto: I thought that ctypes are now *included*? Perhaps it is now conflicting for you if you have two copies? Markus Michael Barton wrote: Yes. I installed ctypes. No problem there. He said that users of the binary I compiled do not have to have ctypes installed separately. But I wonder if it has something to do with this. Michael C. Michael Barton Director, Center for Social Dynamics Complexity Professor of Anthropology, School of Human Evolution Social Change Arizona State University voice: 480-965-6262 (SHESC), 480-727-9746 (CSDC) fax: 480-965-7671 (SHESC), 480-727-0709 (CSDC) www: www.public.asu.edu/~cmbarton, http://csdc.asu.edu On May 26, 2010, at 9:11 PM, Markus Neteler wrote: On Thu, May 27, 2010 at 6:07 AM, Michael Bartonmichael.bar...@asu.edu wrote: ... I did nothing special to make it work. I'm also using the standard system python 2.6.1. I did a build and it worked fine. I posted my binaries for students and colleagues. A student downloaded it and it won't run. She has a very similar machine. Sorry I'm no help. I think it is something else wrong and that this error is probably not what is really wrong. Do you/they use latest GRASS 7? Glynn included a Ctypes* copy in SVN, this won't still work? Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Some ideas about future GRASS interface development
Hello Michael, i would like to add my 2c. More below 2010/5/11 Michael Barton michael.bar...@asu.edu: I'll start a thread on this. If there is enough interest, we should move it to the WIKI. If not, it can quietly fade away. Last month, I was at the AAG (a first for me) in sessions talking about advanced geospatial and other modeling, and jointly with Helena in a session on advances in open source tools for GI Science. Seeing some of the papers and talking with participants set me to thinking again about GUI concepts. I've tried to advocate for thinking outside the standard model in designing interfaces for GIS. Granted that it is difficult to get too cutting edge when we are all avocational developers away from our real jobs or studies. In spite of this (or perhaps because of it), I think we've done a good job so far in creating a flexible and very useable interface to a very complicated piece of software. Most people I show GRASS to are very impressed--and they are comparing it to the commercial market leader that pays a lot of money for software development. If we have such a great interface--and one that we've only developed to a point of stability in the past year--why should we think about something different. The reason is that the nature of computing is changing rapidly, and some of the changes are especially relevant for spatial technology. I've always felt that the normal windows/menus interface (and typing commands) is not really an appropriate design for software that is intended to manipulate, analyze, and display multidimensional geospatial data--but I've had only had a somewhat vague idea of what might be better. At the same time, the idea of ubiquitous computing and combining data from multiple sources over the internet is becoming a reality. The latter is especially important as internet-based repositories of complex geospatial data continue to grow. This brief intro leads me to toss out a couple of rather different ideas for comment. Think of them as GRASS 8 and beyond concepts. 1. GRASS was originally designed to access data from multiple sources and server. However, for most people, GRASS now lives on individual computers and accesses data there. But new tools in GRASS are changing that. There are of course initial tools to access WMS data. More interesting is Glynn's proposal to access all data through an r.external/v.external type interface linked to gdal. If we go that way, GRASS becomes much more flexible, accessing data wherever it is located accessible to a workstation. Some people still want to use GRASS remotely, but have considerable difficulty running our very nice GUI in that way--a GUI that is designed to run on a user's local computer. So we might want to think about making GRASS run in a browser for some future version. I see some kind of synergy between the kind of interface created by software like OpenLayers or MapServer and the data accessing capabilities of Glynn's idea--maybe also incorporating WMS and other such internet data. This might help to make GRASS transparently portable across all computing platforms, and reduce the amount of GUI development work needed for cross-platform compatibility. It also would open the possibility of making GRASS accessible via cloud computing. I don't know how we do this, but the fact that some kind of interface is doable in a browser via OpenLayers, MapServer, and other such internet mapping tools, means that this is possible. We'd need to develop a more sophisticated interface than is now available on existing internet mapping software, but the tools to do this exist and it may be worth considering. The easiest way to make GIS GRASS available as geo-data processing backend for many users is by supporting WPS. This approach allows the use of GRASS functionality by any desktop GIS application on MacOS, Windows or Linux which supports WPS (ie: uDig, OpenJump with WPS plugins from 52n [1] and Qgis 1.4 via WPS Python plugin from Horst Düster[2]) or any Web framework supporting WPS. Some of them support the download and visuailzation of WMS, WFS and WCS data directly, so WCS and WFS data can be used as GRASS module inputs and the results can be displayed together with WMS and WFS data. Hence a wide range of more or less sophisticated GUI's will be available to access GIS GRASS functionality over WPS. I.e.: PyWPS is available for many years to enable GIS GRASS as WPS backend. Making GRASS easier to integrate in other existing and future WPS server is an ongoing project of mine. I develop currently the integration of GRASS in ZOO-WPS server and support the 52n WPS integration. One important step was the generation of WPS XML process description documents by lib/gis/parser for grass7. So for any GRASS module a valid and usable WPS XML process description can be generated. The second important step is to provide a frame-work to start GRASS modules without the need to care about
Re: [GRASS-dev] Re: v.surf.rst
Helo, 2010/5/3 Abhishek Shukla abhishekii...@gmail.com: On Mon, May 3, 2010 at 1:45 AM, Soeren Gebbert soerengebb...@googlemail.com wrote: Hello, LU is used to solve the linear equation system, because the matrix is not positive definite. There is a zero at the diagonal. You can try to sort the rows and cols, but i'm not sure if the matrix will get positive definite. Maybe an LDL decomposition can be used instead of LU. I have tried to solve the linear equation system with a parallele BiCGStab algorithm, but most of the time the linear equation system is of very bad condition, so the BiCGStab solver fails. An LU algorithm with pivoting is needed. Hence, reordering rows and cols to avoid a zero at the diagonal and parallel LDL decomposition may be the best choice. I believe you meant LDU decomposition here. Nope, i mean LDL. Ok, the mathematical correct description is LDL^T. http://www.cise.ufl.edu/research/sparse/ldl/ Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] lidar tools update in grass7
Hello Markus, 2010/5/3 Markus Metz markus.metz.gisw...@googlemail.com: On Wed, Apr 28, 2010, Soeren Gebbert wrote: Hmm, if I understand the code right, only the innermost for loop [of the tchol solver] can be executed in parallel because the first two for-loops need results of previous runs (not possible to run i + 1 before i, same for j). But I don't know that parallel stuff, I would in any case first optimize the code without parallelization, only then add multithreading. I fully agree. The band cholesky solver is well suited for the job but not designed for parallelization. Thus i have implemented an iterative Conjugate Gradient (CG) solver for band matrices (just by replacing the matrix - vector multiplication) to replace the cholesky band solver. Well, the cholesky solver out-performes the CG solver by a factor of 10+. I have partly parallelized the CG solver for band matrices, but even the speedup on a 8 core system is to small to compete with the cholesky band solver. Besides of that, the created band matrices are of bad condition, so the CG solver may fail for subregions. IMHO, it doesn't really make sense to replace a function with another function that's 10+ times slower and produces worse results... If you need a parallelized version and a 10+ core system to achieve the same speed, and the results are not as good as with the original tchol solver, I would suggest to keep using the original tchol solver until the CG solver is faster and produces identical results (also for asymmetrical matrices). Sorry for my misunderstanding words, i not suggest to use the parallel CG algorithm instead of the cholesky. I just wanted to say that i tried to find an alternative and failed. Hence, i would suggest to use MPI parallelization or something else on process base, to concurrently compute the subregions which are created by default. Maybe a python script using overlapping tiling and subprocess can do the job? The handling of the overlapping tiles is deeply embedded in the module code, at first glance it seems to me that you would have to completely translate the module to python and then call the interpolation routines from python? Seems messy to me. The subregion tiling is a bit tricky and I spent quite some time fixing and optimizing it. I don't intent to move the tiling from the C-code into the Python-code, but to have a Python module which is able to tile huge regions into smaller subregions avoiding edge effects using overlapping strategies. These subregions should be still large enough so that the lidar tools can use there tiling strategies. No interpolation is called from the python modules. If that mean to re-implement the lidar tiling mechanism in Python using spline estimation and all the fancy algorithms would be to messy indeed. I was hoping that there are simpler overlapping strategies avoiding edge effects (large overlapping edges which are cut to smaller ones when patching the large tiles using a pixel averaging strategy ...). [snip] As above, IMHO that would be a complete rewrite of the modules in python. Maybe because I am more familiar with C. Changing the C code so that tiles can be processed in parallel should not be too hard. I see. That will mean to re-fracture the C-code, putting the consolidated tiling algorithms in the lidar library and implement MPI parallelism in the lidarlib and each module. Thats IMHO a lot of work too. :) [snip] r.tileset is not doing the same like the subregion tiling done in the lidartools because of the overlapping. The outermost overlapping edge is used for interpolation but the values interpolated there are never used, it exists to avoid edge effects along the borders of the subregion. The interpolation results of the second overlapping zone are used but weighted according to their distance to the core zone. The interpolation results of the core zone are used as is. The extends of the overlapping zones dramatically influence accuracy and speed, and the current design should be kept in order to avoid edge effects. AFAICT, the interpolated surfaces are seamless, no edge effects whatsoever visible in various diagnostic maps calculated. Please excuse my ignorance, i am not familiar with the lidar tiling algorithm. So i thought the size of the overlapping edges can be computed in python and used as parameter for r.tileset (parameter overlap). After the lidar interpolation, the created tiles can be patched by cutting the overlapping edges and using pixel averaging. This will, as you mentioned above not be sufficient to avoid edge effects. So there will be no good alternative to a native MPI lidar version? Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] lidar tools update in grass7
Hi Hamish, 2010/5/1 Hamish hamis...@yahoo.com: Soeren wrote: The computation time of overlapping regions should be the same in python as in C. IMHO it is easier to program a python module which computes the tiles, split up the vector points based on the tiles and distribute the data across a grid or cloud for computation. After the distributed processes are finished, the script can gather the data and merge it together. If you implement this into the lidar modules, each module needs to be changed to support MPI. Using MPI is IMHO much more complex, then just running the modules on different datasets on different nodes, gathering and patching the data. ... Maybe the overlapping algorithm can be simplified in the python version? In my vision, the LIDAR computation could be run in an elastic computation cloud (EC2) as WPS backend using recursive tiling algorithm for data distribution. thinking big! Thinking practical. :) Ignoring the WPS overhead, such an approach may work for every lidar module and other SIMD (single instruction, multiple data) type modules, where the computational effort is much larger than the tiling, distribution and gathering effort (rst - interpolation, ...). A tiling - distribution - gathering approach can be implemented in python very simple (multiprocessing, pympi ...). for my 2c (fwiw I run about 160 computational-hours per day split up and dynamically sent out to a couple of clusters using MPI), I would first ask how much processing are we talking about? If the typical large-case lidar tools job only took ~3 hours to run on a modern PC, I'd much prefer the simplicity of OpenMP and run it on the same quad-core desktop or 8-core Xeon server that the GIS is running on. If the jobs could take 12-24 hours and have to happen often I would more seriously consider using MPI. Setting up the mpd daemon and organizing the mpirun jobs is a lot of overhead, which for smaller jobs seems slightly counter-productive to me. In case the GRASS project has developers which have the time and the knowledge to parallelize the lidar modules using OpenMP or MPI that's fine. But i do not see that we have such resources. I do not have the knowledge and time to implement it. IMHO many modules would benefit from a more general, generic approach described above. Such an approach can be run on multi-core machines, a cluster or in a cloud. I do not have a cluster available, only a 4 core machine. If i need to run a huge LIDAR job in short time (which i never did before!), i would need to buy additional computational power. I.e: Amazon EC2 is a quite reasonable choice for that. also, I think it is quite natural to have the control and management scripts in python, but get worried about splitting up low-level processing into a hybrid python/C mix. for one thing it makes the code harder to follow/understand, for another you are at risk from different component versions interacting in different ways. (see wxNviz, vdigit problems on WinGrass..) Using i.e: MPI for parallelism means often to implement two versions of the program, a serial and a parallel version. OpenMP works only on multi-core or ccNUMA machines (SGI Altix) not in a cluster or cloud. Many paralleized programs are more complex and mostly rewritten than the serial version they based on. IMHO to implement a single python approach with additional python dependencies is not as risky as to implement and maintain many modules of highly complex parallel C code with additional serial versions. Using a tiling - distribution . gathering python approach will have a much lower speedup as pure OpenMP or MPI versions. But the benefit will raise with the size of the computational problem. Best regards Soeren (and of course this has to be an optional extra, and I'm all for exploring all these different methods .. we don't know which one will win out/survive in a few years time when 16-core workstations are the norm) regards, Hamish ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] lidar tools update in grass7
Hello Markus, 2010/4/28 Markus Metz markus.metz.gisw...@googlemail.com: Soeren Gebbert wrote: Hello, A while back I did some crude profiling of the v.lidar tools and found that it was spending lots and lots of time in the Tcholetsky decomposition loop (3-for loops deep). That's better now because the matrix is a bit smaller. Yes, its a nice symmetric band matrix. Was a bit puzzling for me to understand the creation of the matrix without any documentation! I added documentation where I understood the code but honestly, the inner workins of the lidar_lib are still a mystery to me. Thanks for adding documentation, that makes maintaining the modules much easier! The bilinear and bicubic interpolation is a mystery for me too ... at module and theoretical level. So, i have implemented some conversion functions into the gmath library while moving the tchol* code from lidar lib into the gmath lib There are still bugs in the code, mostly minor concerning warning and error messages, one major in v.surf.bspline for very large raster output regions. I'm busy fixing it, also converting it to a new module r.resamp.bspline that does particularly well for filling NULL cells, a real alternative to (the much slower) r.fillnulls. I have renamed some variable (italian - english) in the lidar ilb and modules, to get a better understanding of what is happening in the code. I have committed the changes to svn, in case the changes are wrong, please tell me, i will revert it. [...] i would suggest to use MPI parallelization or something else on process base, to concurrently compute the subregions which are created by default. Maybe a python script using overlapping tiling and subprocess can do the job? I think that should be integrated into the current lidar tools because they make use of the current subregion tiling and C code is still faster than Python code (right?). Treatment of overlapping zones of The computation time of overlapping regions should be the same in python as in C. IMHO it is easier to program a python module which computes the tiles, split up the vector points based on the tiles and distribute the data across a grid or cloud for computation. After the distributed processes are finished, the script can gather the data and merge it together. If you implement this into the lidar modules, each module needs to be changed to support MPI. Using MPI is IMHO much more complex, then just running the modules on different datasets on different nodes, gathering and patching the data. There are already modules which support this process (g.region, r.tileset, r.patch ... ). Well, is there a module which supports the extraction of vector points of the current region without topology support? neighbouring subregions is done sometimes in the lidar_lib, sometimes by the modules themselves. It took me quite some time to understand it and fix it, therefore I would advise against code porting/duplication. Maybe the overlapping algorithm can be simplified in the python version? In my vision, the LIDAR computation could be run in an elastic computation cloud (EC2) as WPS backend using recursive tiling algorithm for data distribution. Best regards Soeren My 2c, Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] lidar tools update in grass7
Hello, Slightly off-topic, but while we are thinking about this code ... A while back I did some crude profiling of the v.lidar tools and found that it was spending lots and lots of time in the Tcholetsky decomposition loop (3-for loops deep). That's better now because the matrix is a bit smaller. Yes, its a nice symmetric band matrix. Was a bit puzzling for me to understand the creation of the matrix without any documentation! So, i have implemented some conversion functions into the gmath library while moving the tchol* code from lidar lib into the gmath lib: now transformation of qudratic - band - sparse matrices can be performed. A new CG band solver, the matrix type conversion and a new band matrix -vector multiplication function are available with tests in the svn-trunk of grass 7. It seemed like a good simple test case for using OpenMP to multi-thread it, but I got stuck with it segfaulting. AFAIR the problem was that OpenMP wanted you to malloc the entire thing before starting, and this could get big. if it interests you, see the attached patch and https://trac.osgeo.org/grass/ticket/657 http://lists.osgeo.org/pipermail/grass-dev/2009-June/044709.html http://lists.osgeo.org/pipermail/grass-dev/2009-June/044705.html http://grass.osgeo.org/wiki/OpenMP Hmm, if I understand the code right, only the innermost for loop can be executed in parallel because the first two for-loops need results of previous runs (not possible to run i + 1 before i, same for j). But I don't know that parallel stuff, I would in any case first optimize the code without parallelization, only then add multithreading. I fully agree. The band cholesky solver is well suited for the job but not designed for parallelization. Thus i have implemented an iterative Conjugate Gradient (CG) solver for band matrices (just by replacing the matrix - vector multiplication) to replace the cholesky band solver. Well, the cholesky solver out-performes the CG solver by a factor of 10+. I have partly parallelized the CG solver for band matrices, but even the speedup on a 8 core system is to small to compete with the cholesky band solver. Besides of that, the created band matrices are of bad condition, so the CG solver may fail for subregions. Hence, i would suggest to use MPI parallelization or something else on process base, to concurrently compute the subregions which are created by default. Maybe a python script using overlapping tiling and subprocess can do the job? Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] makefile for Valgrind
Adding debug options should be sufficient to enable code line traceback. AFAIK other compile options are not needed. Best regards 2010/4/25 Jarek Jasiewicz jar...@amu.edu.pl: Sorry for that question can someone tell me how shall I modify makefile to test my program with Valgrind? adding CFLAGS=-g is enough or shall I do something else? thanks Jarek ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] raster engine in grass7
Hello, 2010/4/25 Glynn Clements gl...@gclements.plus.com: Martin Landa wrote: I have moved some dev-oriented stuff relating to raster library to trac [1]. What is really planned for GRASS 7? Any changes to the raster format or structure are dependent upon a mechanism for reading (and possibly writing) older formats. The raster I/O code is already quite messy due to having to read a variety of different formats. Any new format/structure should replace the existing code rather than adding to it. My preferred approach would be for someone to write a GRASS driver for GDAL which doesn't depend upon the GRASS libraries. Then, we could eliminate the support from the libraries and rely upon r.external[.out] for accessing legacy data. This is a very interesting approach. I hope this is not a silly question and i dont know how to implement a GDLA driver, but i wonder if this approach is able to handle concurrent simultaneously read/write access to multi band maps? How would such a driver look like? Direct read and write access to the existing grass raster file structure or the design of a new raster file format? Maybe we can reuse an existing driver? Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] wxGUI modeler
Hello Martin, this is really great. The GRASS community is waiting for such a tool for many, many years. At work i have implemented a back-end for a graphical modeler which is based on XSD/XML and Python. It is designed to automatically compute and consider the process dependencies and to execute independent processes in parallel. It can process huge, complex process chains with multiple start- and end-points. If you are interested in my experience, don't hesitate to ask. I may be able to help a bit. :) Best regards Soeren 2010/4/14 Martin Landa landa.mar...@gmail.com: Hi, some days ago I started working on the new graphical modeler for wxGUI [1]. Modeler is currently available in GRASS 7.0 and GRASS 6.5 [2]. It's aplha version with very limited functionality. See also flash example how to create a simple model [3]. Martin [1] http://grass.osgeo.org/wiki/WxGUI_Modeler [2] https://svn.osgeo.org/grass/grass/trunk/gui/wxpython/gui_modules/gmodeler.py [3] file:///home/martin/wxgui-modeler-0.html -- Martin Landa landa.martin gmail.com * http://gama.fsv.cvut.cz/~landa ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] GNU scientific library in GRASS
Hello, 2010/4/13 Jarosław Jasiewicz jar...@amu.edu.pl: Hi All! What about using GNU scientific library (gsl) in GRASS module? Will it create additional dependencies? Yes. What about compiling on windows? Any other contraindications?? In general what I want to use I cannot find in gmath library. What kind of algorithms are missing in gmath library? GRASS7 already includes a numerical library called ccmath. Have a look in the gmath-ccmath wrapper in lib/gmath. Currently only a few algorithms from ccmath are used: eigenvalue computation, LU decomposition and SVD solver. Have a look at the ccmath library (http://freshmeat.net/projects/ccmath/). Maybe the algorithms you are searching for is present there? If this is the case we can add it to grass7. Sören thanks for any answer J. ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] r.watershed broken in grass7 trunk?
Hi, it seems to me that r.watershed is broken in the latest grass7 trunk version? This may be related to r41705? Example from the r.watershed manual with spearfish data produces an error: grass_dev/grass_trunk g.list rast -- raster files available in mapset PERMANENT: aspect erosion1quads soils strm.dist bugsitesfields railroads soils.Kfactor texture density geology roads soils.Tfactor tractids elevation.10m landcover.30m rstrct.areassoils.phtransport.misc elevation.dem landcover.orig rushmoresoils.range trn.sites elevation.dted landuse slope spot.image uparea erode.index owner soil.br.depth streams vegcover grass_dev/grass_trunk g.region rast=elevation.dem grass_dev/grass_trunk r.watershed elev=elevation.dem accum=rwater.accum ERROR: USAGE for basin delineation: /1/gebbert/src/grass_dev/grass_trunk/dist.i686-pc-linux-gnu/etc/r.watershed.ram -4 elevation=elevation_map threshold=swale_threshold [flow=overland_flow_map] [drainage=drain_direction_map] [depression=depression_map] [accumulation=accumulation_map] [basin=watershed_basin_map] [stream=stream_segment_map] USAGE for slope length determination: /1/gebbert/src/grass_dev/grass_trunk/dist.i686-pc-linux-gnu/etc/r.watershed.ram [-4] elevation=elevation_map threshold=swale_threshold [drainage=drain_direction_map] [depression=depression_map] [accumulation=accumulation_map] [max_slope_length=max_slope_length] [blocking=overland_blocking_map] [slope_steepness=slope_steepness_map] length_slope=length_slope_map [disturbed_land=rill_erosion_map] [slope_deposition=slope_deposition value or map]USAGE for ARMSED FILE creation: /1/gebbert/src/grass_dev/grass_trunk/dist.i686-pc-linux-gnu/etc/r.watershed.ram [-4] elevation=elevation_map threshold=swale_threshold [flow=overland_flow_map] [drainage=drain_direction_map] [depression=depression_map] [accumulation=accumulation_map] [basin=watershed_basin_map] [stream=stream_segment_map] [half_basin=half_basin_map] ar=ARMSED_file_name WARNING: Subprocess failed with exit code 1 WARNING: category information for [rwater.accum] in [user1] missing or invalid Any suggestions how to fix that? Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Unable to find grass7 start script after svn update/recompile
Hello, i have a problem starting grass7 from the dist directory. I recently made an svn update and recompiled grass7 from scratch. Now the grass7 start script is missing in the dist directory. What can i do. I would like to avoid a make install to test my code changes? Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Unable to find grass7 start script after svn update/recompile
Hello Martin, 2010/4/6 Martin Landa landa.mar...@gmail.com: Hi, 2010/4/6 Soeren Gebbert soerengebb...@googlemail.com: i have a problem starting grass7 from the dist directory. I recently made an svn update and recompiled grass7 from scratch. Now the grass7 start script is missing in the dist directory. you can find start script in bin directory (`grass70`). Unfortunately not: grass_trunk find dist.i686-pc-linux-gnu/ -name \*grass70\* grass_trunk It looks to me that the start script is not generated in the dist directory. Soeren Martin -- Martin Landa landa.martin gmail.com * http://gama.fsv.cvut.cz/~landa ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Unable to find grass7 start script after svn update/recompile
Hi Martin, 2010/4/6 Martin Landa landa.mar...@gmail.com: 2010/4/6 Soeren Gebbert soerengebb...@googlemail.com: you can find start script in bin directory (`grass70`). Unfortunately not: grass_trunk find dist.i686-pc-linux-gnu/ -name \*grass70\* grass_trunk find bin.i686-pc-linux-gnu/ -name \*grass70\* start script (pythonized) has been moved from `dist` to `bin` directory. Ok, got it ... i am slow to catch on. ;) Thanks Martin. Soeren Martin -- Martin Landa landa.martin gmail.com * http://gama.fsv.cvut.cz/~landa ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] r.gwflow
Problem solved in Revision 41496.
2010/3/19 Soeren Gebbert soerengebb...@googlemail.com:
Hello,
i can reproduce this error with grass7, latest svn version.
But i am completely lost why this happens.
parser.c fails after the following option is defined in r.gflow main.c:
param.type = G_define_option();
param.type-key = type;
param.type-type = TYPE_STRING;
param.type-required = NO;
param.type-answer = confined; - this parameter is the reason
for the segfault
param.type-options = confined,unconfined;
param.type-description = _(The type of groundwater flow);
param.type-answer is corrupted and this results the segfault in
G_recreate_command():
Starting program:
/home/soeren/src/grass7.0/grass_trunk/dist.i686-pc-linux-gnu/bin/r.gwflow
[Thread debugging using libthread_db enabled]
Program received signal SIGSEGV, Segmentation fault.
0xb7f82956 in G_recreate_command () at parser.c:634
635 if (opt-answer opt-answers[0]) {
(gdb) bt
#0 0xb7f82956 in G_recreate_command () at parser.c:634
#1 0xb7f82fd7 in module_gui_wx () at parser.c:784
#2 0xb7f82276 in G_parser (argc=1, argv=0xbfffea94) at parser.c:422
#3 0x080498f2 in main (argc=1, argv=0xbfffea94) at main.c:260
Why does this happen. I thought using the answer filed in an options
defines the default value of this option?
Any help appreciated.
Soeren
2010/3/19 Glynn Clements gl...@gclements.plus.com:
Massimo Di Stefano wrote:
it's my first attempt to use r.gwflow
so i have to read it's manual page
with more carefully.
i tried a fast try following the help page, but seems i did something
wrong, see the logs :
http://www.geofemengineering.it/data/r_gwflow.txt
Says:
Sorry, vx is not a valid parameter
Sorry, vy is not a valid parameter
Sorry, budget is not a valid parameter
These options are only present in 7.0, not in 6.5.
--
Glynn Clements gl...@gclements.plus.com
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] r.gwflow
2010/3/20 Glynn Clements gl...@gclements.plus.com: Soeren Gebbert wrote: Problem solved in Revision 41496. So ... param.type = G_define_option(); param.type-key = type; param.type-type = TYPE_STRING; param.type-required = NO; param.type-answer = confined; - this parameter is the reason for the segfault ... is this statement incorrect? First i thought my code is wrong, but now i think this is correct. The reason for the segfault was in parser.c. I reproduced it with other modules setting a default answer. Because this appears to have nothing to do with r41496. The segfault in parser.c only appeared in case a module set a default answer. So IMHO this is related to the problem which was reported by Massimo. Now r.gwflow works as expected. Soeren -- Glynn Clements gl...@gclements.plus.com ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] r.gwflow
Hello Massimo,
unfortunately i am not able to reproduce the segfault, because i do
not run a 64bit system. Does this error also occurs when you start
r.gwflow with parameters, so without gui generation?
Best regards
Soeren
2010/3/19 Massimo Di Stefano massimodisa...@gmail.com:
Hi,
curiose i tried to compile grass on debian enabling glw supprt
to give a try to the module r.gwflow
the build go ahead without errors but ring to run r.gwflow i received a
segfault
teing with gdm i have this log :
GRASS 6.5.svn (WGS84_33N):~ gdb r.gwflow
GNU gdb (GDB) 7.0.1-debian
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type show copying
and show warranty for details.
This GDB was configured as x86_64-linux-gnu.
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/...
Reading symbols from /usr/local/grass-6.5.svn/bin/r.gwflow...done.
(gdb) run
Starting program: /usr/local/grass-6.5.svn/bin/r.gwflow
[Thread debugging using libthread_db enabled]
Program received signal SIGSEGV, Segmentation fault.
0x77bbab71 in G_recreate_command () at parser.c:2857
2857 if (opt-answer != '\0' opt-answers[0] != NULL) {
(gdb)
(gdb)
i'm running grass 6.5 svn on debian
sid.___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] r.gwflow
Hello,
i can reproduce this error with grass7, latest svn version.
But i am completely lost why this happens.
parser.c fails after the following option is defined in r.gflow main.c:
param.type = G_define_option();
param.type-key = type;
param.type-type = TYPE_STRING;
param.type-required = NO;
param.type-answer = confined; - this parameter is the reason
for the segfault
param.type-options = confined,unconfined;
param.type-description = _(The type of groundwater flow);
param.type-answer is corrupted and this results the segfault in
G_recreate_command():
Starting program:
/home/soeren/src/grass7.0/grass_trunk/dist.i686-pc-linux-gnu/bin/r.gwflow
[Thread debugging using libthread_db enabled]
Program received signal SIGSEGV, Segmentation fault.
0xb7f82956 in G_recreate_command () at parser.c:634
635 if (opt-answer opt-answers[0]) {
(gdb) bt
#0 0xb7f82956 in G_recreate_command () at parser.c:634
#1 0xb7f82fd7 in module_gui_wx () at parser.c:784
#2 0xb7f82276 in G_parser (argc=1, argv=0xbfffea94) at parser.c:422
#3 0x080498f2 in main (argc=1, argv=0xbfffea94) at main.c:260
Why does this happen. I thought using the answer filed in an options
defines the default value of this option?
Any help appreciated.
Soeren
2010/3/19 Glynn Clements gl...@gclements.plus.com:
Massimo Di Stefano wrote:
it's my first attempt to use r.gwflow
so i have to read it's manual page
with more carefully.
i tried a fast try following the help page, but seems i did something wrong,
see the logs :
http://www.geofemengineering.it/data/r_gwflow.txt
Says:
Sorry, vx is not a valid parameter
Sorry, vy is not a valid parameter
Sorry, budget is not a valid parameter
These options are only present in 7.0, not in 6.5.
--
Glynn Clements gl...@gclements.plus.com
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] lidar tools update in grass7
Hello, 2010/2/8 Hamish hamis...@yahoo.com: Markus Metz wrote: The core of the lidar tools is the lidarlib, that is AFAICT robust and working. Bugs are most likely in modules, so I'll try to add comments there. I have reorganized the code for v.surf.bspline in the hope that it is now easier to read. Slightly off-topic, but while we are thinking about this code ... A while back I did some crude profiling of the v.lidar tools and found that it was spending lots and lots of time in the Tcholetsky decomposition loop (3-for loops deep). It seemed like a good simple test case for using OpenMP to multi-thread it, but I got stuck with it segfaulting. AFAIR the problem was that OpenMP wanted you to malloc the entire thing before starting, and this could get big. There is an OpenMP version of a bandwidth optimized cholesky decomposition algorithm in gmathlib. I will have a look on the lidarlib to evaluate if we can use the existing OpenMP tuned cholesky. Besides of that, i will move the lidar linear equation solver and the inversion algorithm into the gmath library, so more modules may benefit from it. I have made some performance tests with the OpenMP tuned linear equation solver (cholesky, gauss, lu) in gmathlib, there is unfortunately not much speedup, which is related to the design of the solver algorithms. The solver benchmarks are available in the grass7 lib/gmath/tests directory. I have had only measurable speedup with the intel compiler (which is free for open source projects), the speedup with the gcc OpenMP implementation is little. Additionally it would be interesting to consider the use of the OpenMP tuned conjugate gradient solver from gmath within the lidar tools. This solver has an excellent linear speedup (related to the parallelzied matrix-vector multiplication), is quite fast (non-linear convergence) and works with sparse matrices. Best regards Soeren if it interests you, see the attached patch and https://trac.osgeo.org/grass/ticket/657 http://lists.osgeo.org/pipermail/grass-dev/2009-June/044709.html http://lists.osgeo.org/pipermail/grass-dev/2009-June/044705.html http://grass.osgeo.org/wiki/OpenMP I forgot to mention, there is a problem with sqlite, it is much slower than dbf, no idea if this is a problem of my system or of the grass sqlite driver or of the way auxiliary tables are accessed by the lidar tools. that sounds vaguely familiar, but as a general thing not necessarily to do with v.lidar. probably something about it in the archives? Hamish ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: List of new features in GRASS 7 (in trac)
Hello Markus, new IMHO noticeable features in GRASS 7 are: gmath: * Replacement of most of the numerical recipes code with ccmtah functionality (lu solver is work in progress) * Implementation of blas level 1,2 and 3 functions which are in use by all gmath solver, the gpde library and many modules. * ATLAS wrapper for blas level 1 functions (level 2, and 3 are work in progress) raster: r.neighbours - added new option to process only marked cells (i.e: selective elevation map improvement) r(3).gwflow - Added support for water budget calculation, documentation largely improved r.solute.transport - New module to compute the solute transport of groundwater in porous media using an implicit finite volume approach imagery: i.cca - supports now much more raster maps for canonical analyses (multi-spectral (8) - hyper-spectral (thousands)), converted from Fortran style to C-style :) Just my 2c Soeren 2010/1/26 Markus Neteler nete...@osgeo.org: On Mon, Jan 25, 2010 at 11:17 AM, Markus Neteler nete...@osgeo.org wrote: Hi, I have started the draft list of new features in GRASS 7: http://trac.osgeo.org/grass/wiki/Grass7/NewFeatures This will help to better advertise our progress. Please add missing items (there will be many). I'll give a talk in 23hs from now - please tell me about things missing in the features list :) You can also just respond here and I'll update the Wiki. Just to avoid that central improvements are not mentioned. cheers Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: List of new features in GRASS 7 (in trac)
Hello, sorry for the noise, i forgot to mention: I have ported r.sim.water and r.sim.sediment to be functional in grass 7, but they are still not compiled by default. Vector point support is still missing, the old site code is only commented out. Much more testing is needed. WPS: libgis supports the automatic generation of a WPS 1.0.0 process description XML document for each module (c, c++, python, shell) Best regards Soeren 2010/1/26 Markus Neteler nete...@osgeo.org: On Mon, Jan 25, 2010 at 11:17 AM, Markus Neteler nete...@osgeo.org wrote: Hi, I have started the draft list of new features in GRASS 7: http://trac.osgeo.org/grass/wiki/Grass7/NewFeatures This will help to better advertise our progress. Please add missing items (there will be many). I'll give a talk in 23hs from now - please tell me about things missing in the features list :) You can also just respond here and I'll update the Wiki. Just to avoid that central improvements are not mentioned. cheers Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] New option in r.neighbors
Hi Glynn, 2010/1/19 Glynn Clements gl...@gclements.plus.com: Soeren Gebbert wrote: JFYI: i have added a new option selection to r.neighbors. A few points about the change: 1. Human-readable strings (option descriptions, messages) should be localised with _(...). Sorry, i will check other modules, which i modified, to make sure i did not forget the localization. 2. There's no need to read the map using its native type (CELL/FCELL/DCELL); reading as DCELL will suffice, and requires less Ok. code. Better still, as you only care whether or not cells are null, you can use Rast_get_null_value_row(). I see, i never used Rast_get_null_value_row() before, thanks for your advise. 3. It reads the input a second time (at its native type, then converts to DCELL) when the input row will already be in ncb.buf[ncb.dist] (as DCELL). 4. There are a couple of deviations from the GRASS formatting conventions (see tools/grass_indent.sh), specifically space between if and ( and else on a separate line to a preceding closing brace. I german we call the coding which i have done verschlimmbessern, translated into english this means Trying to improve things but actually make them worse. Please check r40541. Code looks much better now, thanks. Soeren -- Glynn Clements gl...@gclements.plus.com ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] grass7 problems with MASK
Hello, 2010/1/13 Glynn Clements gl...@gclements.plus.com: Soeren Gebbert wrote: Therefor i would like to suggest to establish an environmental variable to enable G_fatal_error() to segfault for better error debugging. I was thinking of a variable in gisrc, but this will result in an infinite G_fatal_error() call-loop, because G_fatal_error() is called in case the environmental variable is not present in gisrc. What would be the best way to implement such an behavior of G_fatal_error()? Any suggestions are welcome. Use an environment variable rather than a GRASS ($GISRC) variable, e.g. if (getenv(GRASS_ABORT_ON_ERROR)) abort(); An abort is preferable to trying to create a segfault, as abort() is specified by the ANSI/ISO C standard, while generating a segfault relies upon undefined behaviour (which means that there is no robust way to force a segfault to occur; the most obvious approach is prone to being optimised-out). The default behaviour for SIGABRT is to terminate and dump core, the same as for SIGSEGV, so I'm not sure that there's any benefit to a segfault. If you really need SIGSEGV, raise(SIGSEGV) might be better (but does this signal exist on Windows?). Regarding the Microsoft Run-Time-Library documentation it should exist: http://msdn.microsoft.com/de-de/library/5s651ehs(VS.80).aspx I will add these lines to lib/gis/error.c in grass7. if (getenv(GRASS_SIGSEGV_ON_ERROR)) raise(SIGSEGV); Where is the best place to document the new grass environment variable? Thanks Soeren -- Glynn Clements gl...@gclements.plus.com ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] New option in r.neighbors
Hello, JFYI: i have added a new option selection to r.neighbors. The user can optionally specify a selection map, to compute new values only where the raster cells of the selection map are not NULL. In case of NULL cells, the values from the input map are copied into the output map. This may useful to smooth only parts of an elevation map (pits, peaks, ...). Example how to use a selection map with method=average: input map: 1 1 1 1 1 1 1 1 1 1 1 1 10 1 1 1 1 1 1 1 1 1 1 1 1 selection map, NULL values are marked as *: * * * * * * * 1 * * * 1 1 1 * * * 1 * * * * * * * The output map: 1 1 1 1 1 1 1 2 1 1 1 2 2 2 1 1 1 2 1 1 1 1 1 1 1 Without using the selection map, the output map would look like this: 1 1 1 1 1 1 2 2 2 1 1 2 2 2 1 1 2 2 2 1 1 1 1 1 1 Have fun Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS GIS] #869: Compile libs with -fexception
I use in vtk.grass-bridge (C++ -VTK GRASS wrapper) a different approach.
http://code.google.com/p/vtk-grass-bridge/
Replacing the G_fatal_error() function and using the setjmp/longjmp
construct allows you to catch all errors from grass, inclusively stack
restoration. Glynn made this suggestion. :)
This function replace G_fatal_error(), to set with
G_set_error_routine(vgb_error_handler);:
int vgb_error_handler(const char *msg, int fatal)
{
if (fatal == 0)
{
fprintf(stderr, %s\n, msg);
return 1;
}
else
{
fprintf(stderr, \n## Exceptiont called ###\n);
vgb_error_message = msg;
longjmp(vgb_stack_buffer, 1);
}
return 1;
}
This is an example how to handle fatal errors (old example, still
check for return value):
if (!setjmp(vgb_stack_buffer))
{
if (Rast_get_row(this-Map, this-RasterBuff, idx, this-MapType) 0)
{
error = 1;
}
}
else
{
this-InsertNextError(vgb_error_message);
return NULL;
}
Code examples from:
http://code.google.com/p/vtk-grass-bridge/source/browse/trunk/Common/vtkGRASSDefines.h
http://code.google.com/p/vtk-grass-bridge/source/browse/trunk/Common/vtkGRASSInit.cxx
http://code.google.com/p/vtk-grass-bridge/source/browse/trunk/Raster/vtkGRASSRasterMapBase.cxx
Soeren
2010/1/14 GRASS GIS t...@osgeo.org:
#869: Compile libs with -fexception
--+-
Reporter: rblazek | Owner: grass-...@lists.osgeo.org
Type: defect | Status: closed
Priority: normal | Milestone: 6.5.0
Component: Compiling | Version: unspecified
Resolution: worksforme | Keywords:
Platform: Unspecified | Cpu: All
--+-
Comment (by rblazek):
Replying to [comment:8 hamish]:
fwiw, wrt. -fexceptions, Francesco commented on the DebianGIS list:
{{{
This would be required for any C library linked by Qgis. This is out of
discussion.
Qgis has to manage correctly C libraries by providing wrappers to raise
execeptions
when C calls fail, else people will start asking that libc too supports
exceptions, soon or later...
}}}
Probably he does not know, that GRASS libraries can call exit(). No
wrapper can help in that situation AFAIK.
--
Ticket URL: https://trac.osgeo.org/grass/ticket/869#comment:9
GRASS GIS http://grass.osgeo.org
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] grass7 problems with MASK
Hi,
i have had the same problem and implemented an identical solution.
But the problem was hard to figure out, because G_fatal_error() is now called
if opening or closing of maps fails.
The problem appeared when i started r.sim.water after creating a mask.
So i searched the error first in r.sim.water, because i changed the
memory allocation
recently and enabled the compilation of this module.
After a while i implemented an artificial segfault in G_fatal_error()
to enable debugging with gdb and valgrind and found the problem.
Therefor i would like to suggest to establish an environmental
variable to enable
G_fatal_error() to segfault for better error debugging. I was thinking
of a variable
in gisrc, but this will result in an infinite G_fatal_error()
call-loop, because G_fatal_error() is called in case the environmental
variable is not present in gisrc.
What would be the best way to implement such an behavior of G_fatal_error()?
Any suggestions are welcome.
Best
Soeren
2010/1/13 Markus Metz markus.metz.gisw...@googlemail.com:
Because of r40217, Rast_close() and Rast_unopen() can no longer be called
unconditionally. That gives me errors when a MASK is present:
ERROR: Invalid descriptor: 0
The following patch to lib/raster works for me but I'm not sure if this is
the right way
Index: init.c
===
--- init.c (revision 40418)
+++ init.c (working copy)
@@ -83,6 +83,7 @@
/* Set masking flag unknown */
R__.auto_mask = -1;
+ R__.mask_fd = -1;
R__.nbytes = sizeof(CELL);
R__.compression_type = getenv(GRASS_INT_ZLIB) ? 2 : 1;
Index: auto_mask.c
===
--- auto_mask.c (revision 40418)
+++ auto_mask.c (working copy)
@@ -61,7 +61,9 @@
return 0;
}
- Rast_unopen(R__.mask_fd);
+ if (R__.mask_fd = 0)
+ Rast_unopen(R__.mask_fd);
+ R__.mask_fd = Rast__open_old(MASK, G_mapset());
if (R__.mask_fd 0) {
R__.auto_mask = 0;
Alternatively, just return if a raster map can't be unopened/closed?
Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Python script to call grass 7 modules as service
Dear developers, JFYI: I have implemented a Python script which allows to call grass 7 modules as a service (e.g.: from a WPS server). This script works currently only with grass raster modules. Vector modules are work in progress. The processing is based on grass version 7. The clue is that the user provides the name of the module, some parameters and the input data as png or geotiff files and receives geotiff or gml files as output. All the location/mapset creation as well as the import and export is done automatically by the script. The coordinate system of the first input file is used to create the grass work location. Next features is, that all the input files are linked via r.external into the grass work location. No explicit import is needed. Currently the output is exported via r.out.gdal and v.out.ogr, but r.external.out will be used in the future, so that raster data will be processed via gdla access without import or export (thanks to Glynn and Frank!). The Python script (some parts are based on pyWPS from Jachym Cepicky): http://code.google.com/p/vtk-grass-bridge/source/browse/trunk/WPS/GrassModuleStarter.py Example input files: http://code.google.com/p/vtk-grass-bridge/source/browse/trunk/WPS/Testing/Python/r.contour_input.txt http://code.google.com/p/vtk-grass-bridge/source/browse/trunk/WPS/Testing/Python/r.watershed_input.txt I hope this script will simplify the use of grass as WPS server back-end. The script is still under development and provides only basic features. Better error and message handling is needed, as well as vector support and testing. Any comments, suggestions, criticism are welcome. :) Have fun Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS-SVN] r40078 - in grass/trunk/raster: . r.solute.transport
Hi Martin, sorry, you are right. I will convert them. Best Soeren 2009/12/19 Martin Landa landa.mar...@gmail.com: Hi, 2009/12/19 svn_gr...@osgeo.org: Author: huhabla Date: 2009-12-19 16:25:49 -0500 (Sat, 19 Dec 2009) New Revision: 40078 Added: grass/trunk/raster/r.solute.transport/ grass/trunk/raster/r.solute.transport/Disper_calc.sh grass/trunk/raster/r.solute.transport/Makefile grass/trunk/raster/r.solute.transport/example.sh grass/trunk/raster/r.solute.transport/example2.sh grass/trunk/raster/r.solute.transport/main.c grass/trunk/raster/r.solute.transport/r.solute.transport.html grass/trunk/raster/r.solute.transport/seguin_verify.sh grass/trunk/raster/r.solute.transport/seguin_verify_well.sh Log: New module to compute solute transport in porous media, including several examples and verification shell scripts. Shell script are not supported in GRASS7. Please convert the scripts to Python. Martin -- Martin Landa landa.martin gmail.com * http://gama.fsv.cvut.cz/~landa ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] [i.spec.unmix]- Compiling spec.unmix to GRASS6.4
Hello António, you need to compile la.c. But la.c depends on blas and lapack: la.c line 36 #if defined(HAVE_LIBLAPACK) defined(HAVE_LIBBLAS) defined(HAVE_G2C_H) Additionally the Fortran to C header file g2c.h must be present. AFAICT there where problems compiling Fortran to C support with gcc4, but i do not remember correctly. If la.c is not compiled your get undefined reference errors. To avoid this you can install the lapack and blas libraries (i have no clue how to do this on windows), or you can use the native blas level 1,2 and 3 function in grass 6.5 and grass 7: http://download.osgeo.org/grass/grass6_progman/blas__level__1_8c.html http://download.osgeo.org/grass/grass6_progman/blas__level__2_8c.html http://download.osgeo.org/grass/grass6_progman/blas__level__3_8c.html The memory allocation is different from the la.c version: http://download.osgeo.org/grass/grass6_progman/dalloc_8c.html http://download.osgeo.org/grass/grass6_progman/ialloc_8c.html http://download.osgeo.org/grass/grass6_progman/sparse__matrix_8c.html There are several linear equation solver and eigenvalue solver available in grass, which are independent from lapack: http://download.osgeo.org/grass/grass6_progman/ccmath__grass__wrapper_8c.html http://download.osgeo.org/grass/grass6_progman/solvers__krylov_8c.html http://download.osgeo.org/grass/grass6_progman/solvers__direct_8c.html IMHO replacing the lapack dependent functions with the native grass versions in i.spec.unmix should be preferred. Best regards Soeren 2009/12/10 António Rocha antonio.ro...@deimos.com.pt: Greetings all A couple of weeks ago I sent an email regarding i.spec.unmix compilation for GRASS 6.4. I analyzed the question a bit more and: 1- I included #include grass/la.h at global.h and main.c. I suppose that this files are compiled in WinGRASS64 binaries. Am I wrong? 2- While compiling i.spec.unmix I got this error from main.c and open.c: /undefined reference to `G_matrix_init' undefined reference to `G_matrix_get_element' undefined reference to `G_matrix_set_element' undefined reference to `G_matrix_transpose' undefined reference to `G_matrix_free' undefined reference to `G_matrix_read' undefined reference to `G_matvect_get_column' undefined reference to `G_vector_norm_maxval' undefined reference to `G_vector_free' undefined reference to `G_vector_init' undefined reference to `G_vector_sub' undefined reference to `G_vector_norm_euclid' undefined reference to `G_vector_copy' undefined reference to `G_vector_product' undefined reference to `G_vector_norm1' /These functions are defined in la.c and la.h. Since I included la.h in header file, I want to know what is not right because compiler is not recognizing these functions? Thanks Best regards, Antonio Rocha __ Information from ESET NOD32 Antivirus, version of virus signature database 4675 (20091210) __ The message was checked by ESET NOD32 Antivirus. http://www.eset.com ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Maintenance QGIS GRASS plugin / provider
Hi Marco, i would like to help maintaining the grass plugin in qgis. But unfortunately i am not able to start right now. I am much to busy. I hope to start working on it in the 2. quater of 2010. I have curently little experience with qgis. I did not use it very often. But i am a C/C++ and Python developer with some Qt experience and have had a look on the grass-plugin sources. :) Best regards Soeren 2009/12/5 Marco Hugentobler ma...@hugis.net: Dear GRASS devs I'm writing to you as the technical advisor of the QGIS project steering committee. As such, I'm trying to make sure all the parts of the program have a dedicated maintainer. Unfortunately, the GRASS plugin in QGIS has no maintainer for quite some time now. The GRASS plugin was written by Radim in 2004. He did a great job, maintained and improved the plugin successively until he left the team in 2006. Since then, the plugin has no maintainer. Even if there was some work on it after that, the GRASS plugin/provider is now the category with the most open bugs in the bugtracker(35). I think this is a pitty, because the plugin has a large user base. Especially (but not only) users new to GIS, are not familiar with the GRASS command line. So they are happy to have the QGIS GUI and the analytical power of GRASS in a graphical way. If there is someone from the GRASS community who is interested in helping to maintain the plugin, it would be greatly appreciated. In case of technical questions and problems, we are available to help. Best regards, Marco -- Dr. Marco Hugentobler HUGIS - GIS programming and consulting Webereistr. 66 CH-8134 Zürich ma...@hugis.net http://homepage.hispeed.ch/hugis/ Technical Advisor QGIS Project Steering Committee ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [tlaronde: ticket #542: vector for GRASS 7]
Hello, thanks for the very interesting discussion. My two cents regarding the delaunay triangulation: Triangulation, Delaunay are a critical part of what should be here in a GIS (GPL GRASS has things as far as I know; KerGIS not for the moment). modern grass has those things, but they need some finishing touches AFAIU. see past Google Summer of Code projects. if you wanted to be a hero :) you could work on a BSD version of this: http://www.cs.cmu.edu/~quake/triangle.html (very well implemented but non-free for commercial use*, so sadly mostly unusable..) [*] I think that's fair enough if the author wants it, except for the NSF (public) funding of it. I have implemented a connection layer between GRASS and VTK (www.vtk.org). Therefor all of the VTK vector and image processing algorithms can process natively GRASS datasets. And so the very good delaunay triangulation algorithms for 2d and 3d data: http://www.vtk.org/doc/release/5.4/html/a00404.html http://www.vtk.org/doc/release/5.4/html/a00405.html No need to re-implement the wheel. ;) Example: The delaunay triangulation grass-vtk python module: http://code.google.com/p/vtk-grass-bridge/source/browse/trunk/Examples/v.delaunay2d.py Computation time is quite equal to v.delaunay. But it ships better documentation and several options to optimize the output. You can also convert raster maps into vector TIN's: Here an example how to convert an elevation raster map into a TIN using a VTK algorithm: http://code.google.com/p/vtk-grass-bridge/source/browse/trunk/Examples/r.to.tin.py * Results: http://code.google.com/p/vtk-grass-bridge/wiki/rToTinResults I have tested this algorithm with large datasets with more than 64 million cells creating a TIN with about 3 million triangles. Memory usage is about 1.5 Gig. Computation time ~20min on Intel core2 with 2.1 Ghz. VTK is licensed under a BSD license. Therefor you can implement commercial algorithms in VTK and use it in GRASS via the vtk-grass connection layer. Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] SVN server problem
Hi folks, i am currently unable to checkout or commit to the grass 7 trunk. I get this error while commiting: svn: Commit failed (details follow): svn: Server sent unexpected return value (400 Bad Request) in response to CHECKOUT request for '/grass/!svn/bln/39802' And this error while checkout: svn: PROPFIND von »/grass/!svn/vcc/default«: Could not read status line: connection was closed by server (https://svn.osgeo.org) Anybody facing the same problem? Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] SVN server problem
H, looks like a problem with my Internet provider ... . 2009/11/26 Markus Neteler nete...@osgeo.org: On Thu, Nov 26, 2009 at 1:32 PM, Soeren Gebbert soerengebb...@googlemail.com wrote: Hi folks, i am currently unable to checkout or commit to the grass 7 trunk. I get this error while commiting: svn: Commit failed (details follow): svn: Server sent unexpected return value (400 Bad Request) in response to CHECKOUT request for '/grass/!svn/bln/39802' And this error while checkout: svn: PROPFIND von »/grass/!svn/vcc/default«: Could not read status line: connection was closed by server (https://svn.osgeo.org) Anybody facing the same problem? I have updated from SVN, no problem right now. Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] SVN server problem
Thanks for the reply, now it magically works :) I have committed the changes to lib/gis/parser.c. Soeren 2009/11/26 Maris Nartiss maris@gmail.com: Recently there where problems with osgeo server. Last update I heard - FrankW was rebooting box/processes when server started to lag (few times a day!). After restart for some time everything works just fine - needs a bit of luck to get things done :) Maris. 2009/11/26, Soeren Gebbert soerengebb...@googlemail.com: H, looks like a problem with my Internet provider ... . 2009/11/26 Markus Neteler nete...@osgeo.org: On Thu, Nov 26, 2009 at 1:32 PM, Soeren Gebbert soerengebb...@googlemail.com wrote: Hi folks, i am currently unable to checkout or commit to the grass 7 trunk. I get this error while commiting: svn: Commit failed (details follow): svn: Server sent unexpected return value (400 Bad Request) in response to CHECKOUT request for '/grass/!svn/bln/39802' And this error while checkout: svn: PROPFIND von »/grass/!svn/vcc/default«: Could not read status line: connection was closed by server (https://svn.osgeo.org) Anybody facing the same problem? I have updated from SVN, no problem right now. Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] WPS XML process description added to lib/gis/parser.c
Hello, i while ago i discussed with markus how to support a better integration of grass in WPS server. He suggested that it would be the best to add WPS XML process description generation directly into grass. So here we are. :) I have added the generation of WPS process description XML documents to lib/gis/parser.c in grass7 . If a module is started with the parameter --wps-process-description the process description XML document is written to stdout and the module will exit. Currently only raster and vector modules are supported, but the generation works with any module (more or less meaningful). Most of the input options are catched: * single and multiple raster and vector maps * single and multiple string, float and integer data with default values and value options (range is missing) Flags are supported as boolean values. The mime types for vector maps are GML 3.1 and grass ascii and binary vectors. mime type: application/grass-vector-ascii - a text file generated with v.out.asci Example.: urn:file:///path/name mime type: application/grass-vector-binary - the binary vectors must be addressed with a non standard urn: Example: urn:grass:vector:location/mapset/name The mime types for raster maps are tiff and png as well as grass ascii and binary raster maps, following the same scheme as the vector maps The mime types are reflecting the capabilities of gdal and may be extended. The support for specific grass formats is just a crazy idea of mine. o_O Only single maps are supported as output or plain text on stdout. BoundignBox support is currently not available for inputs and outputs. Literal data output (string, float or integer) is currently not supported. A lot of work must still be done to verify the correct generation of WPS process descriptions, so please do not use them right now for WPS server input. ;) I am afraid that to fully support the WPS XML process description generation, which is IMO very important for grass, the option, flag and module structures which are used to describe the input and output parameter for each module, must be reviewed and may be extended. Any hints, criticism and suggestions are welcome. Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] GRASS programmer's manual now with cool search function
Thats a great feature, thanks a lot Markus and Martin! Best regards Soeren 2009/11/1 Maris Nartiss maris@gmail.com: Thanks Markus and Martin! This is really cool feature :) Maris. 2009/11/1, Markus Neteler nete...@osgeo.org: Thanks to Martin's idea I have updated doxygen on grass.osgeo.org. Now the GRASS programmer's manual comes with a cool search function: http://download.osgeo.org/grass/grass6_progman/ http://download.osgeo.org/grass/grass7_progman/ enjoy, Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] New TIN algorithm in vtkGRASSBridge
Hello, JFYI i have created a grass python module which uses the vtkGreedyTerrainDecimation algorithm from VTK 5.4 to create a TIN from a raster height map. This algorithm can approximate a raster hight map based on several criterias i.e.: the number of triangles used to approximate the height field of the raster map. It may be quite useful for TIN creation and visualization pruposes of large raster height maps. The created TIN is written back to the grass vector database. The module is called r.to.in.py. An example with screenshots is available here: http://code.google.com/p/vtk-grass-bridge/wiki/rToTinResults The Code is available here: http://code.google.com/p/vtk-grass-bridge/source/browse/trunk/Examples/r.to.tin.py Have fun Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] 3D Vector to Voxel
Hi Ben, 2009/9/30 Benjamin Ducke benjamin.du...@oxfordarch.co.uk: That looks very straight-forward. I don't know Python either, though. Could you write a little example script that runs the VTK Delaunay 2D filter on a GRASS points map, as well? ;) Here we are: http://code.google.com/p/vtk-grass-bridge/source/browse/trunk/Examples/v.delaunay2d.py?spec=svn24r=24 Best reagards Soeren btw: Some additional informations: The help: GRASS 7.0.svn (nc_spm_08):~/src/vtkGRASSBridge/vtk-grass-bridge/Examples python v.delaunay2d.py --help Description: Delaunay 2d triangulation using the vtkDelaunay2D class. The created triangles will be converted into the grass vector feature face. Keywords: vector, delaunay Usage: v.delaunay2d.py [-tw] input=name output=name [alpha=alpha] [tolerance=tolerance] [--overwrite] [--verbose] [--quiet] Flags: -t Build vector topology -w Write the triangulated map as vtk xml file to the current working directoy named output.vtk --o Allow output files to overwrite existing files --v Verbose module output --q Quiet module output Parameters: input Name of input vector map output Name for output vector map alpha Specify alpha (or distance) value to control output of this filter. For a non-zero alpha value, only edges or triangles contained within a sphere centered at mesh vertices will be output. Otherwise, only triangles will be output. default: 0 tolerance Specify a tolerance to control discarding of closely spaced points. This tolerance is specified as a fraction of the diagonal length of the bounding box of the points. default: 0 Now some Computation. VTK version with topo and vtk xml output: GRASS 7.0.svn (nc_spm_08):~/src/vtkGRASSBridge/vtk-grass-bridge/Examples time python v.delaunay2d.py input=elev_lid792_rand...@user1 output=delaunay2d --o -tw Vector map delaunay2d already exists and will be overwritten Building topology for vector map delaunay2d... Registering primitives... 11912 primitives registered 35736 vertices registered Building areas... 100% 0 areas built 0 isles built Attaching islands... Attaching centroids... 100% Number of nodes: 5773 Number of primitives: 11912 Number of points: 0 Number of lines: 0 Number of boundaries: 0 Number of centroids: 0 Number of faces: 11912 Number of areas: 0 Number of isles: 0 real0m1.750s user0m1.564s sys 0m0.144s VTK version without topo and vtk xml output: GRASS 7.0.svn (nc_spm_08):~/src/vtkGRASSBridge/vtk-grass-bridge/Examples time python v.delaunay2d.py input=elev_lid792_rand...@user1 output=delaunay2d --o Vector map delaunay2d already exists and will be overwritten real0m1.167s user0m1.004s sys 0m0.144s Comparsion to v.delaunay with huge map, notice its a 3d vector map: VTK version without grass topology build: GRASS 7.0.svn (nc_spm_08):~/src/vtkGRASSBridge/vtk-grass-bridge/Examples time python v.delaunay2d.py input=elev_ned10m_cont10m output=delaunay2d --o Vector map delaunay2d already exists and will be overwritten real0m27.422s user0m25.230s sys 0m1.904s VTK version with grass topology build: GRASS 7.0.svn (nc_spm_08):~/src/vtkGRASSBridge/vtk-grass-bridge/Examples time python v.delaunay2d.py input=elev_ned10m_cont10m output=delaunay2d --o -t Vector map delaunay2d already exists and will be overwritten Building topology for vector map delaunay2d... Registering primitives... 369281 primitives registered 1107843 vertices registered Building areas... 100% 0 areas built 0 isles built Attaching islands... Attaching centroids... 100% Number of nodes: 151776 Number of primitives: 369281 Number of points: 0 Number of lines: 0 Number of boundaries: 0 Number of centroids: 0 Number of faces: 369281 Number of areas: 0 Number of isles: 0 real0m45.035s user0m42.243s sys 0m2.464s Now the grass version: GRASS 7.0.svn (nc_spm_08):~/src/vtkGRASSBridge/vtk-grass-bridge/Examples time v.delaunay input=elev_ned10m_cont...@user1 output=delaunay2d --o WARNING: Vector map delaunay2d already exists and will be overwritten Building topology for vector map delaunay2d... Registering primitives... 0 primitives registered 0 vertices registered Building areas... 0 areas built 0 isles built Number of nodes: 0 Number of primitives: 0 Number of points: 0 Number of lines: 0 Number of boundaries: 0 Number of centroids: 0 Number of areas: 0 Number of isles: 0 Building topology for vector map delaunay2d... Attaching islands... Attaching centroids... Number of nodes: 0 Number of primitives: 0 Number of points: 0 Number of lines: 0 Number of boundaries: 0 Number of centroids: 0 Number of areas: 0 Number of isles: 0 real0m0.053s user0m0.028s sys 0m0.020s That was fast! Well looks like v.delaunay only works with 2d vector maps??? Ben - Original Message - From: Soeren Gebbert soerengebb...@googlemail.com To: Benjamin Ducke benjamin.du...@oxfordarch.co.uk Cc: GRASS developers list
Re: [GRASS-dev] 3D Vector to Voxel
Sorry, i posted a wrong version. Here is the correct link: http://code.google.com/p/vtk-grass-bridge/source/browse/trunk/Examples/v.delaunay2d.py best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] grass7 ccmath, gmath and gpde libraries svn update
Hello, i have added the ccmath library to grass7 trunk, the patched/enhanced versions of the gmath and gpde libraries and updated versions of r/r3.gwflow, i.cca, i.pca and i.smap. Its a port from grass6.5. It is a major update and I have tested it several times, but only on a suse linux 11.1 system. So hopefully i did not break anything. :) Any feedback is welcome. Best regards Soeren ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] grass7 ccmath, gmath and gpde libraries svn update
Hi Martin, 2009/10/3 Martin Landa landa.mar...@gmail.com: Hi, 2009/10/3 Soeren Gebbert soerengebb...@googlemail.com: i have added the ccmath library to grass7 trunk, the patched/enhanced versions of the gmath and gpde libraries and updated versions of r/r3.gwflow, i.cca, i.pca and i.smap. Thanks! Its a port from grass6.5. It is a major update and I have tested it several times, but only on a suse linux 11.1 system. So hopefully i did not break anything. :) Just cosmetics, missing EXTRA_LIBS in Makefile (fixed). Thanks, i guess i need to update the gmath Makefile for the same reason too. Any feedback is welcome. I am just thinking why it came first to devbr6 and not trunk;-) Probably we should focus on trunk and GRASS 7... I was working on this patch since 2007. IIRC grass7 was not established at this time. So patching grass6.5 was the first target on my mind. :) Additionally it was important for me to remove the numerical recipes code from grass6 too, which will be IMHO for the next years the standard version for many users. And i am still not finished, gmath/lu.c must be replaced too and many modules must be patched to use the new gmath solver and the blas functions ... . Soeren Martin -- Martin Landa landa.martin gmail.com * http://gama.fsv.cvut.cz/~landa ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Using GRASS in long running and multithreaded applications Was: Re: The tomcat shut down ...
Hi Glynn,
thanks a lot for your response.
After reading some documentation and asking silly questions to
my poor informatics colleagues, i understand the concept of thread local and
the setjmp()/longjmp() approach a bit better.
I would suggest to add longjmp() to G_fatal_error().
It should be set at runtime by an application if longjmp() should be
chosen or not.
So G_fatal_error() will either call longjmp() or exit().
The setjmp() code goes into the application which calls the grass
library functions,
except if nested setjmp()/longjmp() calls are needed in grass to clean
up data, or
to close open file descriptors.
The linux threaded errno definition scared me, so i have chosen a
different approach.
We define thread local support and two extern variables in gis.h to
choose at runtime if
G_fatal_error() will call exit() or longjmp() and to add thread local support.
Example which works for me in my test code:
/*Thread local and setjmp() exception support*/
#include setjmp.h
#ifdef WIN32
#define Thread __declspec( thread )
#else
#define Thread __thread
#endif
extern Thread jmp_buf G_stack_buffer;/*to save the most important
CPU register for each thread*/
extern int G_long_run;/*Set to 1 to choose the setjmp() version of
G_fatal_error()*/
The G_long_run variable will be initialized in gisinit.c and so the
G_stack_buff:
int G_long_run;
Thread jmp_buf G_stack_buffer;
...
void G__gisinit(const char *version, const char *pgm)
{
const char *mapset;
if (initialized)
return;
G_long_run = 0;
...
The application has to set G_long_run right after
calling G_gisinit() from a single thread (i.e: a singleton).
Now we need to patch error.c to use longjmp() or exit():
void G_fatal_error(const char *msg, ...)
{
va_list ap;
va_start(ap, msg);
vfprint_error(ERR, msg, ap);
va_end(ap);
if(G_long_run == 1)
longjmp(G_stack_buffer, 1);
else
exit(EXIT_FAILURE);
}
The C++ application code may look like this:
extern C {
#include grass/gis.h
}
...
int G_long_run;
Thread jmp_buf G_stack_buffer;
vtkGRASSInit::vtkGRASSInit() {
G_gisinit(vtkGRASSBridge);
// Set the long run variable to provide long run support in grass
libraries
G_long_run = 1;
}
...
/*Open a vector map*/
...
if(!setjmp(G_stack_buffer))
{
if (1 Vect_open_new(this-map, name, with_z))
{
fprintf(stderr, class: %s line: %i Unable to open
vector map %s.,
this-GetClassName(), __LINE__,
name);
return false;
}
} else {
fprintf(stderr, class: %s line: %i Unable to open vector map
%s.,
this-GetClassName(), __LINE__, name);;
return false;
}
...
That's all.
Is this approach ok or to simple or just naive? :)
If this is ok, i would like to test this approach to identify possible
nested setjmp()/longjmp()
calls in libgis, libraster and libvector.
Additionally i will try to make most of the static variable thread local.
Best regards
Soeren
2009/9/27 Glynn Clements gl...@gclements.plus.com:
Soeren Gebbert wrote:
I'm not suggesting making all of the functions take a pointer to the
state as a parameter, just making it thread-local.
Ok.
To my shame i have to admit that i never heard of the thread-local
mechanism before.
After a quick look at wikipedia i understand the principal and it sounds
great!
This will speed up things a lot.
I guess we need to use the pthread version of thread-local to support
other compiler than gcc and windows too?
You would need to conditionalise it. The usual mechanism is like that
used for errno. In a single-threaded implementation, it's just a
variable. In a multi-threaded implementation, it's a macro which
expands to (*errno_location()), where errno_location() retrieves the
address using pthread_getspecific().
However, the error handling is probably a bigger issue. Pushing error
handling onto the modules isn't an acceptable solution.
Indeed. This was the next issue i would like to talk about.
Simply allowing the fatal error handler to longjmp() out then resume
using the GRASS libraries would be non-trivial, as you would have to
repair any inconsistencies in the library state.
Is there an alternative to longjmp() and setjmp()?
Not really.
It seems to be quite complex, the man page warns about the usage.
And i never used it before.
longjmp() is conceptually similar to raising an exception in C++,
while setjmp() is equivalent to establishing a try/catch block.
The details are quite simple if you understand how C is implemented in
terms of machine code. setjmp() essentially saves the most important
CPU registers (including the program counter, stack pointer, and frame
pointer), while longjmp() restores them. So setjmp() records the
current execution
Re: [GRASS-dev] Using GRASS in long running and multithreaded applications Was: Re: The tomcat shut down ...
Hello,
Example which works for me in my test code:
/*Thread local and setjmp() exception support*/
#include setjmp.h
#ifdef WIN32
#define Thread __declspec( thread )
#else
#define Thread __thread
#endif
The __thread qualifier is a gcc extension.
Yes, wikipedia says it works for:
Sun Studio C/C++, IBM XL C/C++, GNU C and Intel C/C++ (Linux systems)
Using the pthreads implementation will be a better solution?
extern Thread jmp_buf G_stack_buffer; /*to save the most important
CPU register for each thread*/
In order to use a GRASS library from multiple threads, the entire
library state needs to be thread-specific. That would include the
current error handler.
Also: do you actually need to resume a thread in which an error
occurs? If not, you can just make the error handler terminate the
current thread. That will prevent the error handler from returning and
thus prevent exit() being called.
I am not sure about this. If the grass libs are called from within an
EJB container, then there will be no way to terminate the current
thread. But it should work from within a servlet container or a RCP
container.
Is this approach ok or to simple or just naive? :)
It's too invasive. The longjmp() should go into an application-defined
error handler, rather than the GRASS libraries.
Ok. The error handler looks like that for now:
int vgb_error_handler(const char *msg, int fatal)
{
if (fatal == 0)
{
fprintf(stderr, %s\n, msg);
return 1;
}
else if (fatal == 1)
{
fprintf(stderr, WARNING: %s\n, msg);
longjmp(vgb_stack_buffer, 1);
}
else if (fatal == 2)
{
fprintf(stderr, ERROR: %s\n, msg);
longjmp(vgb_stack_buffer, 2);
}
return 1;
}
vtkGRASSInit::vtkGRASSInit()
{
G_gisinit(vtkGRASSInit);
// Set the error routine
G_set_error_routine(vgb_error_handler);
}
The only changes to the GRASS libraries regarding error handling
should be to ensure that it is safe to continue using them after the
application recovers from a fatal error.
Supporting thread-local state requires some changes to the GRASS
libraries, i.e. moving most[1] static variables for a library into a
state structure, and referencing all such variables through a pointer.
I'd prefer to avoid putting any setjmp()s into the GRASS libraries.
I'd rather see low-level functions split into an internal version
which returns an error status, and a public version which calls
G_fatal_error(), and have intermediate functions use the internal
version if they need to catch errors.
Ok, i agree. I will implement the setjmp()/longjmp() only within my application.
Ultimately, the GRASS libraries exist for the benefit of GRASS
modules. Minor changes to facilitate other uses (e.g. persistent
applications) are reasonable, but I'm opposed to substantial
architectural changes, particularly if it makes the coding process
more complex.
Very much agreed.
Thanks
Soeren
--
Glynn Clements gl...@gclements.plus.com
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] 3D Vector to Voxel
Hello Ben, 2009/9/30 Benjamin Ducke benjamin.du...@oxfordarch.co.uk: That looks very straight-forward. I don't know Python either, though. Could you write a little example script that runs the VTK Delaunay 2D filter on a GRASS points map, as well? ;) The grass vector to VTK polydata writer is right on its way. There is already a brand new grass vector writer class in vtk-grass-bridge svn. :) When the writer is ready i will provide a VTK Delaunay 2D filter module for grass vector maps. And if you ask for, i can provide the VTK and vtkGRASSBridge shared libraries (compiled on SUSE Linux 11.1), so you can test the module without compiling VTK and vtkGRASSBridge on your machine. Best regards Soeren btw: I can bring an USB stick on 15 Oct. 2009. :D Ben - Original Message - From: Soeren Gebbert soerengebb...@googlemail.com To: Benjamin Ducke benjamin.du...@oxfordarch.co.uk Cc: GRASS developers list grass-dev@lists.osgeo.org Sent: Tuesday, September 29, 2009 5:38:12 PM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: Re: [GRASS-dev] 3D Vector to Voxel You don't need to learn C++. You can use Python or Java to connect grass, vtkGRASSBridge and VTK. And even TCL/TK if im going crazy and support the vtkTckTkWrapper. ;) A simple Python example which will work right out of the box: http://code.google.com/p/vtk-grass-bridge/source/browse/trunk/Examples/r.gauss.smooth On command line call within the nc dataset: python r.gauss.smooth input=lsat5_1987_10 output=test --o Thats all :) Beń - Original Message - From: Soeren Gebbert soerengebb...@googlemail.com To: Benjamin Ducke benjamin.du...@oxfordarch.co.uk Cc: GRASS developers list grass-dev@lists.osgeo.org Sent: Tuesday, September 29, 2009 2:49:53 PM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: Re: [GRASS-dev] 3D Vector to Voxel Hello Ben, 2009/9/21 Benjamin Ducke benjamin.du...@oxfordarch.co.uk: Hi all, I just saw a little GPL'd sofwtare that may provide a base for a v.to.r3 module that could actually cast arbitrary vector geometries to a voxel grid: Have a look at the vtkVoxelModeller class : http://www.vtk.org/doc/release/5.4/html/a01861.html or the vtkImplicitModeller class: http://www.vtk.org/doc/release/5.4/html/a00859.html Booth classes convert arbitrary datasets to a voxel representations. I have just committed a grass vector topology reader to the vtkGRASSBridge repository. Maybe you can now create voxel representations of 3d grass vector data, using GRAS, VTK and vtkGRASSBridge? Best regards Soeren http://www.cs.princeton.edu/~min/binvox/ maybe someone would like to look at possibilities for integrating this into GRASS? Best, Ben -- Files attached to this email may be in ISO 26300 format (OASIS Open Document Format). If you have difficulty opening them, please visit http://iso26300.info for more information. ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev -- Files attached to this email may be in ISO 26300 format (OASIS Open Document Format). If you have difficulty opening them, please visit http://iso26300.info for more information. ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev -- Files attached to this email may be in ISO 26300 format (OASIS Open Document Format). If you have difficulty opening them, please visit http://iso26300.info for more information. ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] 3D Vector to Voxel
Hello Ben, 2009/9/21 Benjamin Ducke benjamin.du...@oxfordarch.co.uk: Hi all, I just saw a little GPL'd sofwtare that may provide a base for a v.to.r3 module that could actually cast arbitrary vector geometries to a voxel grid: Have a look at the vtkVoxelModeller class : http://www.vtk.org/doc/release/5.4/html/a01861.html or the vtkImplicitModeller class: http://www.vtk.org/doc/release/5.4/html/a00859.html Booth classes convert arbitrary datasets to a voxel representations. I have just committed a grass vector topology reader to the vtkGRASSBridge repository. Maybe you can now create voxel representations of 3d grass vector data, using GRAS, VTK and vtkGRASSBridge? Best regards Soeren http://www.cs.princeton.edu/~min/binvox/ maybe someone would like to look at possibilities for integrating this into GRASS? Best, Ben -- Files attached to this email may be in ISO 26300 format (OASIS Open Document Format). If you have difficulty opening them, please visit http://iso26300.info for more information. ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] 3D Vector to Voxel
Hello Ben, 2009/9/29 Benjamin Ducke benjamin.du...@oxfordarch.co.uk: That looks really useful. There is probably a myriad of other classes in VTK that could be used by GRASS modules, too. Indeed. I guess the best way would be to write some add-on modules that depend on VTK and the GRASS bridge and expose some VTK classes to the GRASS CLI. However, then I would have to learn C++ ;) You don't need to learn C++. You can use Python or Java to connect grass, vtkGRASSBridge and VTK. And even TCL/TK if im going crazy and support the vtkTckTkWrapper. ;) A simple Python example which will work right out of the box: http://code.google.com/p/vtk-grass-bridge/source/browse/trunk/Examples/r.gauss.smooth On command line call within the nc dataset: python r.gauss.smooth input=lsat5_1987_10 output=test --o Thats all :) How about the OpenGL coordinate precision problem? I guess the problem is still there, i haven't tested it recently. Best regards Soeren Beń - Original Message - From: Soeren Gebbert soerengebb...@googlemail.com To: Benjamin Ducke benjamin.du...@oxfordarch.co.uk Cc: GRASS developers list grass-dev@lists.osgeo.org Sent: Tuesday, September 29, 2009 2:49:53 PM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: Re: [GRASS-dev] 3D Vector to Voxel Hello Ben, 2009/9/21 Benjamin Ducke benjamin.du...@oxfordarch.co.uk: Hi all, I just saw a little GPL'd sofwtare that may provide a base for a v.to.r3 module that could actually cast arbitrary vector geometries to a voxel grid: Have a look at the vtkVoxelModeller class : http://www.vtk.org/doc/release/5.4/html/a01861.html or the vtkImplicitModeller class: http://www.vtk.org/doc/release/5.4/html/a00859.html Booth classes convert arbitrary datasets to a voxel representations. I have just committed a grass vector topology reader to the vtkGRASSBridge repository. Maybe you can now create voxel representations of 3d grass vector data, using GRAS, VTK and vtkGRASSBridge? Best regards Soeren http://www.cs.princeton.edu/~min/binvox/ maybe someone would like to look at possibilities for integrating this into GRASS? Best, Ben -- Files attached to this email may be in ISO 26300 format (OASIS Open Document Format). If you have difficulty opening them, please visit http://iso26300.info for more information. ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev -- Files attached to this email may be in ISO 26300 format (OASIS Open Document Format). If you have difficulty opening them, please visit http://iso26300.info for more information. ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] vtkGRASSBridge update
Hello Markus, 2009/9/26 Markus Neteler nete...@osgeo.org: On Sat, Sep 26, 2009 at 5:01 PM, Soeren Gebbert soerengebb...@googlemail.com wrote: Dear grass devs, JFYI, i have updated the vtkGRASSBridge to work only with grass7 and added vector support without topology handling. The vtkGRASSBridge is hosted on code.google.com as svn repository: http://code.google.com/p/vtk-grass-bridge/ Congratulations! I have taken liberty to create a WIKI page: http://grass.osgeo.org/wiki/GRASS_and_VTK Thanks :) I will maybe add some content about the project there. But first i need to add some content to the google code wiki. ;) And my time is sooo limited ... :( Here a small python example to triangulate vector points from the nc demo data set with the VTK delaunay filter class: Curiosity: how much faster is this compared to v.delaunay or likewise? The VTK Version is twice as fast, but does not write the data back to grass and therefor do not build a topology in grass. The VTK output is base64 XML written to /tmp. GRASS output: GRASS 7.0.svn (nc_spm_08):~/src/vtkGRASSBridge/vtk-grass-bridge/Examples time v.delaunay -r input=elev_lid792_rand...@user1 output=test2 --o WARNING: Vector map test2 already exists and will be overwritten Building topology for vector map test2... Registering primitives... 17948 primitives registered 35896 vertices registered Building areas... 100% 11949 areas built 1 isles built Number of nodes: 6000 Number of primitives: 17948 Number of points: 0 Number of lines: 0 Number of boundaries: 17948 Number of centroids: 0 Number of areas: 11949 Number of isles: 1 Number of areas without centroid: 11949 100% Building topology for vector map test2... Attaching islands... 100% Attaching centroids... 100% Number of nodes: 17949 Number of primitives: 29897 Number of points: 0 Number of lines: 0 Number of boundaries: 17948 Number of centroids: 11949 Number of areas: 11949 Number of isles: 1 real0m2.543s user0m2.100s sys 0m0.392s GRASS 7.0.svn (nc_spm_08):~/src/vtkGRASSBridge/vtk-grass-bridge/Examples time python VectorDelaunyTriangulation.py 'vector/elev_lid792_randpts' was found in more mapsets (also found in user1)Using elev_lid792_rand...@permanentDebug: Off Modified Time: 254 Reference Count: 1 Registered Events: (none) Is Open: True: Organisation: NC OneMap CreationDate: Person: hmitaso Title: Rural area (in tile 792), random points from lidar-based DEM FullName: elev_lid792_rand...@permanent Mapset: PERMANENT MapDate: Mon Mar 19 00:37:45 2007 Scale: 1 Comment: Zone: 0 Threshold: 0 Projection: 99 ProjectionName: Lambert Conformal Conic Is 3d: No Total number of coor points: 6000 real0m1.107s user0m1.048s sys 0m0.040s Best regards Soeren Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] vtkGRASSBridge update
Hello Ben, 2009/9/26 Benjamin Ducke benjamin.du...@oxfordarch.co.uk: Great stuff indeed, Soeren! Thanks a lot. I hope this code will help to do something useful in the future. Curiosity: how much faster is this compared to v.delaunay or likewise? To me, it's not so much the speed that matters but functionality. VTK's 2D Delaunay has a very useful alpha threshold setting to constrain the triangle (by maximum triangle size). In addition, it's capable of fitting a plane of reference through the data points and does not assume that they lie perfectly in the X-Y plane. This gives much better results for point clouds that describe e.g. micro relief on a slope. You can try this already, even without the VTK bridge: just export some 3D points to VTK, load into ParaView (3.6) and choose Delaunay 2D from the Filter menu. Now, if only we had a simple way of getting VTK data back into GRASS... This is work in progress there will be of course a vtkGRASSVectorPolyDataWriter() in ... some ... weeks. :) Best regards Soeren Ben Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev -- Benjamin Ducke Senior Applications Support and Development Officer Oxford Archaeological Unit Limited Janus House Osney Mead OX2 0ES Oxford, U.K. Tel: +44 (0)1865 263 800 (switchboard) Tel: +44 (0)1865 980 758 (direct) Fax :+44 (0)1865 793 496 benjamin.du...@oxfordarch.co.uk -- Files attached to this email may be in ISO 26300 format (OASIS Open Document Format). If you have difficulty opening them, please visit http://iso26300.info for more information. ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
