[GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-05-22 Thread Anne Ghisla
Hello all, Sören and me have discussed today about creating a comprehensive test suite for GRASS. We wrote down the draft [0] and I plan to work on the implementation as soon as possible. I hope it is of interest of the developers team, and of course more opinions are welcome, as well as coding h

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-05-22 Thread Glynn Clements
Anne Ghisla wrote: > Sören and me have discussed today about creating a comprehensive test > suite for GRASS. We wrote down the draft [0] and I plan to work on the > implementation as soon as possible. > > I hope it is of interest of the developers team, and of course more > opinions are welcome

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-05-23 Thread Anne Ghisla
Hello Glynn, all, On Sun, 2011-05-22 at 20:56 +0100, Glynn Clements wrote: > Anne Ghisla wrote: > > > Sören and me have discussed today about creating a comprehensive test > > suite for GRASS. We wrote down the draft [0] and I plan to work on the > > implementation as soon as possible. [...] > >

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-05-23 Thread Glynn Clements
Anne Ghisla wrote: > > I'd suggest adding a "test" target to the various *.make files, so you > > can do e.g.: > > > > make -C display test > > make -C display/d.rast test > > Good idea. Sadly I don't know makefile system well, so most of my > questions are basic. Your suggestion is to

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-05-24 Thread Markus Metz
On Mon, May 23, 2011 at 4:04 PM, Glynn Clements wrote: > > Anne Ghisla wrote: > >> > I'd suggest adding a "test" target to the various *.make files, so you >> > can do e.g.: >> > >> >     make -C display test >> >     make -C display/d.rast test >> >> Good idea. Sadly I don't know makefile system

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-05-24 Thread Soeren Gebbert
Hi, [snip] >> It saves us writing lots of functions that would compare datatypes; but >> OTOH it would add dependencies of other GRASS modules. Then, it will be >> necessary (or at least better) to test first the modules without >> dependencies. We would need to check if a test includes more than

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-06-02 Thread Soeren Gebbert
Hi, i have updated the wiki and added a simple Python example of a hypothetical r.series test. Please have a look at: http://grass.osgeo.org/wiki/Test_Suite#Test_framework The sample Python code shows in principle how a test case would look like and what kind of methods should be provided by the f

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-06-03 Thread Glynn Clements
Soeren Gebbert wrote: > i have updated the wiki and added a simple Python example of a > hypothetical r.series test. Please have a look at: > http://grass.osgeo.org/wiki/Test_Suite#Test_framework > > The sample Python code shows in principle how a test case would look > like and what kind of met

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-06-03 Thread Soeren Gebbert
Hi Glynn, 2011/6/3 Glynn Clements : > > Soeren Gebbert wrote: > >> i have updated the wiki and added a simple Python example of a >> hypothetical r.series test. Please have a look at: >> http://grass.osgeo.org/wiki/Test_Suite#Test_framework >> >> The sample Python code shows in principle how a tes

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-06-03 Thread Glynn Clements
Soeren Gebbert wrote: > I was thinking about a similar approach, but the effort to parse the > modules XML interface description to identify the command line > arguments to compare the created data was to much effort for me. I don't see a need to parse the command; just execute it and see what f

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-06-08 Thread Soeren Gebbert
Hello Glynn, i was thinking a lot about your and my approach and decided finally to try your approach first with the hope it will be sufficient for any kind of test cases. I still have concerns about the comparison of floating point data regarding precision, i.e: coordinates, region settings, FCELL

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-06-09 Thread Glynn Clements
Soeren Gebbert wrote: > > Dependencies aren't really an issue. You build all of GRASS first, > > then test. Any modules which are used for generating test maps or > > analysing data are assumed to be correct (they will have test cases of > > their own; the most that's required is that such module

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-06-09 Thread Moritz Lennert
On 09/06/11 10:01, Glynn Clements wrote: Soeren Gebbert wrote: v.random output=random_points n=100 v.info -t random_points> result If the framework understands vector maps, it shouldn't be necessary to use v.info; it should be able to compare the random_points map to random_points.ref.

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-06-09 Thread Soeren Gebbert
Hi, Thats why I compare the topology information and not the position/coordinates. In any cases 100 points must be generated. Best Soeren Am 09.06.2011 12:21 schrieb "Moritz Lennert" : > On 09/06/11 10:01, Glynn Clements wrote: >> >> Soeren Gebbert wrote: > >>> v.random output=random_points n=100

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-06-09 Thread Hamish
Moritz wrote: > But how to compare two randomly generated maps ? Unless the > system is not really random, the maps _should_ be different. Good eye, but the system is not really random, only pseudo-random. Use the GRASS_RND_SEED enviro variable to get the same result each time by using the same s

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-06-09 Thread Glynn Clements
Moritz Lennert wrote: > >> v.random output=random_points n=100 > >> v.info -t random_points> result > > > > If the framework understands vector maps, it shouldn't be necessary to > > use v.info; it should be able to compare the random_points map to > > random_points.ref. > > But how to compare

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-06-10 Thread Moritz Lennert
On 10/06/11 02:29, Glynn Clements wrote: Moritz Lennert wrote: v.random output=random_points n=100 v.info -t random_points> result If the framework understands vector maps, it shouldn't be necessary to use v.info; it should be able to compare the random_points map to random_points.ref. But

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-06-10 Thread Glynn Clements
Moritz Lennert wrote: > So which of the two option (a seed= parameter / disabling seeding by PID > vs GRASS_RND_SEED) is preferrable ? I would think that we should try to > be consistent across modules, but r.mapcalc uses GRASS_RND_SEED and > v.perturb uses 'seed=' (but also doesn't use rand()).

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-06-11 Thread Hamish
Hi, wrt, but not limited to the test suite, see also wish #618 "rfe: r.md5sum" https://trac.osgeo.org/grass/ticket/618 maybe it is a useful tool to have in the mix. Hamish ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-06-13 Thread Soeren Gebbert
Hello, >> I assume such critical modules are coded in the framework, not in the >> test scripts? > > I was thinking about a directive (e.g. "@critical") in the test script > so that any failure during the test would generate a more prominent Such kind of annotations/directives are a great idea. I

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-06-14 Thread Glynn Clements
Soeren Gebbert wrote: > > message. If any such errors occured as a result of "make test", you > > would ignore all the other failures. In the same way that if error.log > > has an error for e.g. lib/gis, you wouldn't bother about all of the > > subsequent errors and focus on what was wrong with l

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-06-16 Thread Soeren Gebbert
Hi all, i have summarized the ideas of Anne, Glynn and me in the wiki: http://grass.osgeo.org/wiki/Test_Suite Hopefully most of the ideas a listed there. While reviewing the g3d library code and related modules i implemented prototypes of test cases for several r3.* modules. Some of them are add

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-06-16 Thread Pawel Netzel
Hi, By my opinion, sections are not clearly separated in the examples on wiki page. It will be something like "good practise" to write: # description of test # . # #@preprocessing # ... some comments command1 command2 # #@test # some comments command3 command4 or a

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-06-17 Thread Soeren Gebbert
Hello Pawel, 2011/6/17 Pawel Netzel : > Hi, > > By my opinion, sections are not clearly separated in the examples on wiki > page. This is intention. IMHO the annotation should be part of the documentation so the test writer need to formulate sentence using predefined words. > It will be somethi

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-06-17 Thread Soeren Gebbert
... > Ugh; v.random should have a seed= option (or something) to allow > repeatability. Or at least an option to disable seeding with the PID. The seed option for v.random is now available in r46730 including tests. Cheers Soeren ___ grass-dev mailing l

Re: [GRASS-dev] Test suite for GRASS - proposal, discussion welcome

2011-06-18 Thread Anne Ghisla
On Thu, 2011-06-16 at 22:26 +0200, Soeren Gebbert wrote: > Hi all, > i have summarized the ideas of Anne, Glynn and me in the wiki: > > http://grass.osgeo.org/wiki/Test_Suite > > Hopefully most of the ideas a listed there. > > While reviewing the g3d library code and related modules i implemente