Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Glynn Clements]

Sören Gebbert wrote:

> > Vector lib does not call G_fatal_error(), it was written in parallel
> > with QGIS plugin with vector editing in mind. I hope it is still true.
> 
> I fear that might be not the case anymore. For example, all memory
> allocation functions G_calloc(), G_malloc() and G_realloc() will call
> G_fatal_error() in case the memory allocation fails. As far as i can
> tell the vector library make often use of this functions. So we have
> the problem that basic core functions call G_fatal_error() that are
> used all over the place in GRASS GIS.

Memory allocation is largely a theoretical problem. If you're out of
memory, there's no guarantee that whatever error-handling mechanism
you come up with can function (e.g. printing an error message may
result in a malloc() within libc's stdio functions).

> If you patch GRASS to use setjmp/longjmp to implement an exception
> like behavior then you need to write clean up code for static
> variables and dynamically allocated memory. A G_fatal_error() call can
> be deeply nested in a library function. Cleaning only the static
> variables to be in usable state will lead to problematic memory leaks.
> So it is a complex task in GRASS.

Memory leaks are another area where GRASS libraries aren't
particularly suitable for persistent applications.

In general, any leak which has no significance for a command-line
module is considered not significant, period.

Many strings inside the library are dynamically allocated and never
freed (rather, they're implicitly freed upon process termination). 
This means that the library can just return a pointer and nothing
needs to worry about that pointer becoming invalid (because it will
remain valid for the lifetime of the process).

Allowing such data to be freed would require either keeping track of
usage (e.g. reference counting) or duplicating the underlying data
rather than simply returning a pointer to it.

> Using setjmp/longjmp in QGIS wrapping all GRASS functions that might
> call G_fatal_error() is complex, since a G_fatal_error() call can be
> deeply nested in a library function. You need to keep track of this
> resulting in plenty setjmp wrappers around GRASS functions.
> You need to cleanup global static structures as well and you will have
> no control over pointers pointing to dynamic allocated memory that got
> lost in a longjmp.

If it can be gotten to the point where memory leaks are the only
problem, there isn't really that much of a problem, given that we're
talking about the behaviour in "exceptional" conditions.

The more fundamental problem is e.g. allocating a "slot" from an
array, starting to fill in its fields, and aborting (longjmp,
exception, etc) part-way through. Or doing something similar with
groups of related variables.

Anything which iterates over the array is going to encounter a slot
where some of the fields contain garbage. If those fields are
pointers, offsets, lengths, etc, the likely result is a segfault.

> Having a kind of garbage collector in GRASS, implemented in the
> G_*alloc() functions, may solve many problems. But i have no idea
> howto implement the garbage collector to detect pointers that got
> orphaned in a longjmp to automagically clean them.

I don't think that's realistic.

There are simpler things which would help in many cases; e.g. in the
"slots" case, not allocating the slot until after you've obtained all
of the values which are needed to initialise it.

Beyond that, we have to deal with C's lack of destructors. We may want
to maintain a stack of cleanup handlers (i.e. destructors) which would
be executed in the event of a fatal error. Code which needs to ensure
some form of cleanup occurs would push handlers as resources are
allocated, and pop them prior to a normal return.

This would avoid having to implement nested try/catch constructs using
setjmp/longjmp and G_set_error_routine. In fact, that currently isn't
possible, as you can't get at the existing fatal error handler in
order to save and restore it around an local handler.

-- 
Glynn Clements 
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Radim Blazek]

On Wed, Apr 23, 2014 at 4:02 PM, Sören Gebbert
>> Vector lib does not call G_fatal_error(), it was written in parallel
>> with QGIS plugin with vector editing in mind. I hope it is still true.
>
> I fear that might be not the case anymore. For example, all memory
> allocation functions G_calloc(), G_malloc() and G_realloc() will call
> G_fatal_error() in case the memory allocation fails. As far as i can
> tell the vector library make often use of this functions. So we have
> the problem that basic core functions call G_fatal_error() that are
> used all over the place in GRASS GIS.

I don't care about allocation error. If it really runs out of memory
then exit() is maybe the best. Any other action (saving project,
saving changes) will also need memory and thus fail. Are you often
running out of memory?

It made me curious how Qt handles allocation problem, because I think
that allocation results are not checked in the whole QGIS at all. For
example QString::realloc(int) on the pointer returned by malloc()
calls Q_CHECK_PTR() macro which prints error if pointer is 0. Then it
continues execution even if allocation failed.

> If you patch GRASS to use setjmp/longjmp to implement an exception
> like behavior then you need to write clean up code for static
> variables and dynamically allocated memory. A G_fatal_error() call can
> be deeply nested in a library function. Cleaning only the static
> variables to be in usable state will lead to problematic memory leaks.
> So it is a complex task in GRASS.

I don't want to patch GRASS until it is really inevitable. The next
one is my choice:

> Using setjmp/longjmp in QGIS wrapping all GRASS functions that might
> call G_fatal_error() is complex, since a G_fatal_error() call can be
> deeply nested in a library function. You need to keep track of this
> resulting in plenty setjmp wrappers around GRASS functions.

setjmp will be called just before the top level function we need.

> You need to cleanup global static structures as well and you will have
> no control over pointers pointing to dynamic allocated memory that got
> lost in a longjmp.

>>> and will not prevent QGIS from
>>> terminating or segfaulting eventually in case of a fatal error.
>>
>> Why?
>
> Because of the undefined state of global variables and structures and
> because of potential memory leaks that may sum up making QGIS
> unusable.

In theory maybe, in practice it will work and it DOES work with GRASS
6. I don't care about few bytes lost. G_fatal_error() is really rare
in QGIS provider/plugin, it should be rare.

Radim
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Sören Gebbert]

Hi Radim,

2014-04-23 14:50 GMT+02:00 Radim Blazek :
> On Tue, Apr 22, 2014 at 4:49 PM, Sören Gebbert
>  wrote:
 IMHO we GRASS developers are  too stubborn to change the design of
 GRASS GIS to work as library with persistent applications. This would
 require rewriting plenty of core functionality and modification of all
 C-modules. But there is a solution for persistent applications, called
 Remote Procedure Call (RPC), more below.
>>>
>>> It sounds interesting but it seems to be a huge work an far future. I
>>
>> Its far less work then modifying GRASS libraries to run with
>> persistent applications.
>
> We have already the providers and the plugin working with GRASS 6, I
> just want to upgrade it to GRASS 7. The only problem I found so far is
> that G__get_window() and Rast_get_cellhd() call G_fatal_error(). I can
> only dedicate few hours/max days of my free time to that, so I have to
> consider well what is less work, RPC is not.

Yes, i absolutely understand this.

>
>> We need
>> such an approach anyway, since vector editing is a GUI task -> GUI is
>> a persistent
>> application -> we need an exit safe interface to the GRASS libraries
>> to implement a reliable
>> vector editing GUI in GRASS itself.
>
> Vector lib does not call G_fatal_error(), it was written in parallel
> with QGIS plugin with vector editing in mind. I hope it is still true.

I fear that might be not the case anymore. For example, all memory
allocation functions G_calloc(), G_malloc() and G_realloc() will call
G_fatal_error() in case the memory allocation fails. As far as i can
tell the vector library make often use of this functions. So we have
the problem that basic core functions call G_fatal_error() that are
used all over the place in GRASS GIS.

>
>>> am also concerned about complexity, e.g. starting / keeping running
>>> servers on various platforms. I can imagine endless bug reports about
>>> "server not running". I am looking for something which will not
>>
>> Since only QGIS is spawning the "server" process, it is always able to
>> check if the
>> process is running and can restart it when needed. We already make use
>> of this concept in GRASS
>> and it works nicely[1]. Spawning of processes and keeping track of
>> them is an easy task with Qt,
>> and it is completely OS independent. The Qt Bitcoin client for example
>> makes use of RPC
>> and runs on Windows, Mac OS and various Linux systems. So i don't see
>> a problem here.
>
> The raster provider is already using something like that. Raster data
> are read from executed GRASS module output (a module written for that
> in QGIS in fact) and identify tool keeps another GRASS module running
> and it is communicating with that through a  pipe. I wrote that as an
> experiment and because I know, how much unnecessary additional code
> and potential problems it is, I don't want to continue in that
> direction.

Ok.

>
>>> require permanent assistance. It is not KISS enough for me.
>>
>> What would be a better KISS solution?
>>
>> IMO the RPC approach will make the implementation of the QGIS-GRASS
>> data provider and the
>> plugin simpler. There will be no mixing of C++ and C-code anymore.
>
> We are no starting the plugin from scratch, it is already written, we
> just need to make it work with GRASS 7.
>
>> No setjmp/longjmp/exception workaround that is really complex
>
> Why?

If you patch GRASS to use setjmp/longjmp to implement an exception
like behavior then you need to write clean up code for static
variables and dynamically allocated memory. A G_fatal_error() call can
be deeply nested in a library function. Cleaning only the static
variables to be in usable state will lead to problematic memory leaks.
So it is a complex task in GRASS.

Using setjmp/longjmp in QGIS wrapping all GRASS functions that might
call G_fatal_error() is complex, since a G_fatal_error() call can be
deeply nested in a library function. You need to keep track of this
resulting in plenty setjmp wrappers around GRASS functions.
You need to cleanup global static structures as well and you will have
no control over pointers pointing to dynamic allocated memory that got
lost in a longjmp.

IMHO booth cases require plenty of work to make them usable.

Having a kind of garbage collector in GRASS, implemented in the
G_*alloc() functions, may solve many problems. But i have no idea
howto implement the garbage collector to detect pointers that got
orphaned in a longjmp to automagically clean them.

The next step would be to centralize global variables to clean them up
more easily (as Glynn mentioned).

>
>> and will not prevent QGIS from
>> terminating or segfaulting eventually in case of a fatal error.
>
> Why?

Because of the undefined state of global variables and structures and
because of potential memory leaks that may sum up making QGIS
unusable.

Best regards
Soeren

>
> Radim
>
>> The QGIS GRASS data provider/plugin will be a clean C++ implementation
>> using the thrift library.
>

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Radim Blazek]

On Tue, Apr 22, 2014 at 4:49 PM, Sören Gebbert
 wrote:
>>> IMHO we GRASS developers are  too stubborn to change the design of
>>> GRASS GIS to work as library with persistent applications. This would
>>> require rewriting plenty of core functionality and modification of all
>>> C-modules. But there is a solution for persistent applications, called
>>> Remote Procedure Call (RPC), more below.
>>
>> It sounds interesting but it seems to be a huge work an far future. I
>
> Its far less work then modifying GRASS libraries to run with
> persistent applications.

We have already the providers and the plugin working with GRASS 6, I
just want to upgrade it to GRASS 7. The only problem I found so far is
that G__get_window() and Rast_get_cellhd() call G_fatal_error(). I can
only dedicate few hours/max days of my free time to that, so I have to
consider well what is less work, RPC is not.

> We need
> such an approach anyway, since vector editing is a GUI task -> GUI is
> a persistent
> application -> we need an exit safe interface to the GRASS libraries
> to implement a reliable
> vector editing GUI in GRASS itself.

Vector lib does not call G_fatal_error(), it was written in parallel
with QGIS plugin with vector editing in mind. I hope it is still true.

>> am also concerned about complexity, e.g. starting / keeping running
>> servers on various platforms. I can imagine endless bug reports about
>> "server not running". I am looking for something which will not
>
> Since only QGIS is spawning the "server" process, it is always able to
> check if the
> process is running and can restart it when needed. We already make use
> of this concept in GRASS
> and it works nicely[1]. Spawning of processes and keeping track of
> them is an easy task with Qt,
> and it is completely OS independent. The Qt Bitcoin client for example
> makes use of RPC
> and runs on Windows, Mac OS and various Linux systems. So i don't see
> a problem here.

The raster provider is already using something like that. Raster data
are read from executed GRASS module output (a module written for that
in QGIS in fact) and identify tool keeps another GRASS module running
and it is communicating with that through a  pipe. I wrote that as an
experiment and because I know, how much unnecessary additional code
and potential problems it is, I don't want to continue in that
direction.

>> require permanent assistance. It is not KISS enough for me.
>
> What would be a better KISS solution?
>
> IMO the RPC approach will make the implementation of the QGIS-GRASS
> data provider and the
> plugin simpler. There will be no mixing of C++ and C-code anymore.

We are no starting the plugin from scratch, it is already written, we
just need to make it work with GRASS 7.

> No setjmp/longjmp/exception workaround that is really complex

Why?

> and will not prevent QGIS from
> terminating or segfaulting eventually in case of a fatal error.

Why?

Radim

> The QGIS GRASS data provider/plugin will be a clean C++ implementation
> using the thrift library.
> It will be better maintainable since developers do not need to know
> all the different
> GRASS libraries (gis, raster, vector, database) but only the RPC
> interface that make
> use of a subset of the GRASS library functions, bundling them in an
> object oriented API.
> QGIS do not need to link against GRASS libraries anymore, future GRASS
> versions will be supported out of the box.
>
> This is much more KISS then the current approach.
>
> Best regards
> Soeren
>
> [1] 
> http://grass.osgeo.org/programming7/namespacepython_1_1temporal_1_1c__libraries__interface.html
>>
>> Radim
>>
>>> 2014-04-18 11:31 GMT+02:00 Radim Blazek :
 I have upgraded  the vector provider to GRASS 7, layers may be added
 by drag from browser. The raster and the plugin are disabled. Be
 careful about multiple versions on the same system
 (LD_LIBRARY_PATH..., check with ldd if does not work).

 Unfortunately GRASS 7 moved ahead towards its aim "to make life harder
 for anyone trying to use the GRASS libraries" [1]. Basically more and
 more low level functions are calling exit() instead of returning error
 code if something failed. As I am not willing to implement GRASS
 module call for each simple function, we have to think again about
 hacks we can use:

 1) add a requirement that GRASS 7 used with QGIS must be compiled with
 -fexceptions

 2) add a requirement that a patch (it is a single line comment in
 fact) must be applied to GRASS 7 to make it usable with QGIS

 3) use setjmp()/longjmp()

 4) let QGIS crash whenever GRASS lib function fails
>>>
>>> I fear that none of these suggestions are good working solutions,
>>> since in case a fatal error or a segfault occurs nothing (no
>>> exception, no setjmp) will prevent QGIS from crashing. So i would like
>>> to suggest the following:
>>>
>>> 5.) Using a RPC interface for map metadata, vector and raster map
>>> ac

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Sören Gebbert]

2014-04-20 11:38 GMT+02:00 Martin Landa :
> Hi,
>
> 2014-04-20 1:34 GMT+02:00 Sören Gebbert :
>
>> Hence the wrapper will not only do simple function wrapping, it will
>> also implement a higher level interface:
>
> [...]
>
> what about to create a trac wiki page dedicated to RFC and collect
> this notes there... ?

https://trac.osgeo.org/grass/wiki/Grass7/RPCInterface

>
> Martin
>
> --
> Martin Landa * http://geo.fsv.cvut.cz/gwiki/Landa
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Sören Gebbert]

Hi Radim,

2014-04-22 16:03 GMT+02:00 Radim Blazek :
> On Fri, Apr 18, 2014 at 2:19 PM, Sören Gebbert
>  wrote:
>> Hi Radim,
>> IMHO we GRASS developers are  too stubborn to change the design of
>> GRASS GIS to work as library with persistent applications. This would
>> require rewriting plenty of core functionality and modification of all
>> C-modules. But there is a solution for persistent applications, called
>> Remote Procedure Call (RPC), more below.
>
> It sounds interesting but it seems to be a huge work an far future. I

Its far less work then modifying GRASS libraries to run with
persistent applications. We need
such an approach anyway, since vector editing is a GUI task -> GUI is
a persistent
application -> we need an exit safe interface to the GRASS libraries
to implement a reliable
vector editing GUI in GRASS itself.

> am also concerned about complexity, e.g. starting / keeping running
> servers on various platforms. I can imagine endless bug reports about
> "server not running". I am looking for something which will not

Since only QGIS is spawning the "server" process, it is always able to
check if the
process is running and can restart it when needed. We already make use
of this concept in GRASS
and it works nicely[1]. Spawning of processes and keeping track of
them is an easy task with Qt,
and it is completely OS independent. The Qt Bitcoin client for example
makes use of RPC
and runs on Windows, Mac OS and various Linux systems. So i don't see
a problem here.

> require permanent assistance. It is not KISS enough for me.

What would be a better KISS solution?

IMO the RPC approach will make the implementation of the QGIS-GRASS
data provider and the
plugin simpler. There will be no mixing of C++ and C-code anymore. No
setjmp/longjmp/exception
workaround that is really complex and will not prevent QGIS from
terminating or segfaulting eventually in case of a fatal error.
The QGIS GRASS data provider/plugin will be a clean C++ implementation
using the thrift library.
It will be better maintainable since developers do not need to know
all the different
GRASS libraries (gis, raster, vector, database) but only the RPC
interface that make
use of a subset of the GRASS library functions, bundling them in an
object oriented API.
QGIS do not need to link against GRASS libraries anymore, future GRASS
versions will be supported out of the box.

This is much more KISS then the current approach.

Best regards
Soeren

[1] 
http://grass.osgeo.org/programming7/namespacepython_1_1temporal_1_1c__libraries__interface.html
>
> Radim
>
>> 2014-04-18 11:31 GMT+02:00 Radim Blazek :
>>> I have upgraded  the vector provider to GRASS 7, layers may be added
>>> by drag from browser. The raster and the plugin are disabled. Be
>>> careful about multiple versions on the same system
>>> (LD_LIBRARY_PATH..., check with ldd if does not work).
>>>
>>> Unfortunately GRASS 7 moved ahead towards its aim "to make life harder
>>> for anyone trying to use the GRASS libraries" [1]. Basically more and
>>> more low level functions are calling exit() instead of returning error
>>> code if something failed. As I am not willing to implement GRASS
>>> module call for each simple function, we have to think again about
>>> hacks we can use:
>>>
>>> 1) add a requirement that GRASS 7 used with QGIS must be compiled with
>>> -fexceptions
>>>
>>> 2) add a requirement that a patch (it is a single line comment in
>>> fact) must be applied to GRASS 7 to make it usable with QGIS
>>>
>>> 3) use setjmp()/longjmp()
>>>
>>> 4) let QGIS crash whenever GRASS lib function fails
>>
>> I fear that none of these suggestions are good working solutions,
>> since in case a fatal error or a segfault occurs nothing (no
>> exception, no setjmp) will prevent QGIS from crashing. So i would like
>> to suggest the following:
>>
>> 5.) Using a RPC interface for map metadata, vector and raster map
>> access. Hence one or several GRASS "server" processes provide an RPC
>> interface to access GRASS library functions that are needed in a
>> persistent application. Most important is the vector editing, vector
>> reading/writing, database access, raster reading and map metadata
>> support. Everything else can be done using modules. Hence the RPC
>> interface will support only a limited subset of the GRASS library
>> functions. This RPC interface should only be used in the GRASS
>> provider classes, the GRASS plugin itself can be written in C++ or
>> Python.
>>
>> This approach will decouple GRASS from QGIS, since all communication
>> is done via Inter Process Communication (IPC) using pipes or sockets.
>> There is no need anymore to catch a fatal error or to link GRASS
>> libraries directly to QGIS. The GRASS plugin can be implemented GRASS
>> version independently and so the GRASS data provider, since they will
>> not use GRASS functions directly only the RPC interface. The RPC
>> implementation on the side of GRASS will provide a consistent
>> interface that will not chan

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Radim Blazek]

On Fri, Apr 18, 2014 at 2:19 PM, Sören Gebbert
 wrote:
> Hi Radim,
> IMHO we GRASS developers are  too stubborn to change the design of
> GRASS GIS to work as library with persistent applications. This would
> require rewriting plenty of core functionality and modification of all
> C-modules. But there is a solution for persistent applications, called
> Remote Procedure Call (RPC), more below.

It sounds interesting but it seems to be a huge work an far future. I
am also concerned about complexity, e.g. starting / keeping running
servers on various platforms. I can imagine endless bug reports about
"server not running". I am looking for something which will not
require permanent assistance. It is not KISS enough for me.

Radim

> 2014-04-18 11:31 GMT+02:00 Radim Blazek :
>> I have upgraded  the vector provider to GRASS 7, layers may be added
>> by drag from browser. The raster and the plugin are disabled. Be
>> careful about multiple versions on the same system
>> (LD_LIBRARY_PATH..., check with ldd if does not work).
>>
>> Unfortunately GRASS 7 moved ahead towards its aim "to make life harder
>> for anyone trying to use the GRASS libraries" [1]. Basically more and
>> more low level functions are calling exit() instead of returning error
>> code if something failed. As I am not willing to implement GRASS
>> module call for each simple function, we have to think again about
>> hacks we can use:
>>
>> 1) add a requirement that GRASS 7 used with QGIS must be compiled with
>> -fexceptions
>>
>> 2) add a requirement that a patch (it is a single line comment in
>> fact) must be applied to GRASS 7 to make it usable with QGIS
>>
>> 3) use setjmp()/longjmp()
>>
>> 4) let QGIS crash whenever GRASS lib function fails
>
> I fear that none of these suggestions are good working solutions,
> since in case a fatal error or a segfault occurs nothing (no
> exception, no setjmp) will prevent QGIS from crashing. So i would like
> to suggest the following:
>
> 5.) Using a RPC interface for map metadata, vector and raster map
> access. Hence one or several GRASS "server" processes provide an RPC
> interface to access GRASS library functions that are needed in a
> persistent application. Most important is the vector editing, vector
> reading/writing, database access, raster reading and map metadata
> support. Everything else can be done using modules. Hence the RPC
> interface will support only a limited subset of the GRASS library
> functions. This RPC interface should only be used in the GRASS
> provider classes, the GRASS plugin itself can be written in C++ or
> Python.
>
> This approach will decouple GRASS from QGIS, since all communication
> is done via Inter Process Communication (IPC) using pipes or sockets.
> There is no need anymore to catch a fatal error or to link GRASS
> libraries directly to QGIS. The GRASS plugin can be implemented GRASS
> version independently and so the GRASS data provider, since they will
> not use GRASS functions directly only the RPC interface. The RPC
> implementation on the side of GRASS will provide a consistent
> interface that will not change in case the underlying GRASS API
> changes.
>
> I strongly suggest to use an existing RPC framework to implement fast
> binary data exchange and the IPC. My favorite is apache thrift[1],
> since it supports plenty of programming languages (C/C++, Python,
> Java, JavaScript, ...) and provides operating system independent
> client and server functionality. It supports exceptions on client side
> that can be emitted in case a GRASS server process died because of a
> fatal error, SIGINT or segfault.
>
> I am absolutely willing to implement the RPC server side in GRASS
> using thrift and C++, providing plenty of Python unit tests that show
> howto use it on the client side. We just need to decide what kind of
> GRASS library functionality is needed as RPC interface and what can be
> done in QGIS with C++/Python directly (gisrc creation, GRASS
> environmental variable settings, ...) or using GRASS modules
> (g.region, g.gisenv, ...).
>
> [1] https://thrift.apache.org/
>
>
> Best regards
> Soeren
>
>>
>> Radim
>>
>> [1]https://trac.osgeo.org/grass/ticket/869#comment:1
>>
>> On Thu, Mar 27, 2014 at 11:18 AM, Paolo Cavallini  
>> wrote:
>>> Hi all.
>>> I learned during dinner that GRASS7 RC1 is due very soon. This opens the
>>> issue of its functioning in QGIS. IMHO:
>>>
>>> * the qgis-grass-plugin might stop working (this has to be tested)
>>> * some of the module options will be different
>>> * new modules will not be available in QGIS.
>>>
>>> I think we can deal with this in several ways:
>>>
>>> * dropping the plugin, and concentrate the work on Processing
>>> * upgrading both the plugin and Processing.
>>>
>>> In the first case, we have two major issues:
>>>
>>> * upgrading Processing GRASS modules
>>> * changing the current Processing behaviour, avoiding the import-export
>>> phase when piping consecutive GRASS commands; this makes GRASS modules
>>> sl

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Martin Landa]

Hi,

2014-04-20 1:34 GMT+02:00 Sören Gebbert :

> Hence the wrapper will not only do simple function wrapping, it will
> also implement a higher level interface:

[...]

what about to create a trac wiki page dedicated to RFC and collect
this notes there... ?

Martin

-- 
Martin Landa * http://geo.fsv.cvut.cz/gwiki/Landa
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Vaclav Petras]

>  > G_fatal_error() is too low level not only for messages but in general,
> for
> > Python libraries wrapping GRASS library, it does not allow to throw an
> > exception in case of an error. And the same applies for C++ wrappers.
>
> If you want to turn fatal errors into exceptions, the correct way to
> do it is to use setjmp/longjmp (and contribute patches to make any
> intervening code exception-safe).
>

As I wrote before, I hope that RPC approach suggested here by Soeren will
work for languages with exceptions and for persistent applications while
GRASS modules will benefit from the advantages of GRASS library with
G_fatal_error() with exit().

Vaclav
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Glynn Clements]

Vaclav Petras wrote:

> > The key feature of G_fatal_error() isn't printing error messages. The
> > key feature is that it doesn't return.
> >
> 
> This is one of the problems of G_fatal_error(), it does not gives good
> error messages to the user. User gets "G_malloc: unable to allocate %lu
> bytes of memory at %s:%d" but does not have any idea when and why this
> happened and what to do about it. If the module calling G_malloc() had the
> chance to report error by itself, it could suggest to user to change some
> parameter or use a smaller map.

It could, but 99% of the time it won't. Instead, it will either assume
that the returned pointer is valid, or provide some vacuous message
such as "out of memory" (without the reference to the source file,
line number or requested size that is given at present).

So in a handful of cases, you'd get a better error message, while the
rest will be even worse. At least the current situation ensures that
there are enough clues that the GRASS developers can provide
assistance to the user upon request.

When I replaced status returns with fatal errors in a number of core
functions (r40209, r40217, r40701), in some of the cases where the
function's return type changed from int to void, not a single error
was generated; IOW, not one of the callers actually bothered to check
whether the call succeeded.

Similarly, in cases where an int/pointer return value could be
negative or null to indicate an error, the cases where the (now
redundant) error tests needed to be removed from the callers were
often a minority, sometimes a very small minority, occasionally zero.

With G_fatal_error(), the requirement for the caller to check for
errors is removed, and with it the possibility that the caller will
either fail to check for errors, or produce an entirely useless error
message (although I think that we've now eliminated all of the
G_fatal_error("oops!") cases; no, I'm not joking).

> G_fatal_error() is too low level not only for messages but in general, for
> Python libraries wrapping GRASS library, it does not allow to throw an
> exception in case of an error. And the same applies for C++ wrappers.

If you want to turn fatal errors into exceptions, the correct way to
do it is to use setjmp/longjmp (and contribute patches to make any
intervening code exception-safe).

-- 
Glynn Clements 
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Glynn Clements]

Martin Landa wrote:

> > Returning from G_fatal_error() won't leave GRASS libraries in an
> > unpredictable state. It's "escaping" from G_fatal_error() via e.g.
> > Returning from G_fatal_error() will typically result in a
> > (more-or-less immediate) segfault when the caller tries to use data
> > structures which contain garbage (because the callee never initialised
> > them, but raised a fatal error instead).
> 
> Well, and what do you think that I meant by "unpredictable state"?

I assumed that you meant the same thing that I do when I point out
that longjmp()ing out of a fatal error handling will leave the
libraries' data in an unpredictable state, i.e. that a *subsequent*
call to a GRASS library function may result in a crash.

Having G_fatal_error() return will usually cause an immediate crash,
without needed any subsequent calls.

> > The simplest example is G__malloc():

> You chose very specific case of course.

I chose the simplest example.

But the more general case is:

1. Code performs a task which can either fail or succeed.

2a. On success, some form of result is stored in one or more
variables.

2b. On failure, the code calls G_fatal_error().

3. The code uses the values of the variables assigned in 2a, taking it
for granted that they contain valid values (because if 1 had failed,
the G_fatal_error() call in 2b ensures that we wouldn't be here).

If G_fatal_error() can return, practically any function which calls it
will either attempt to use garbage data, or will (silently) return
garbage data to its caller.

> There is probably something
> you don't want to understand. In the core libraries (gis, vector,
> raster, imagery) there are 553(!) places where G_fatal_error() is
> called. Some time ago you introduced G_fatal_error() even to the
> function which tries to open raster map [1]. So if the raster map
> doesn't exists the function is calling exit()! This is something which
> makes life harder for anyone who want to use GRASS libraries (as you
> already mentioned) with one exception - modules - standalone programs,

Modules aren't an exception, they're the rule. People wanting to use
GRASS libraries in a persistent application are the exception.

It's the latter category that's doing the "not understanding". 
Specifically, they don't understand the magnitude of the cost involved
in propagating errors all the way to the top-most level using status
returns at each step.

This cost is why modern languages have exceptions. Unfortunately:

1. C doesn't have exceptions. The closest thing it has is
setjmp/longjmp.

2. Exceptions eliminate much of the cost, but not all of it. You still
have to write exception-safe code. But the difference is significant
enough that I wouldn't object to the (comparatively minor) added
complexity resulting from making code exception-safe. Apart from
anything else, the added complexity would be limited to the
implementation of library functions; it doesn't impose a unreasonable
burden on the callers in the way that using status returns does.

-- 
Glynn Clements 
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Jürgen E . Fischer]

Hi Martin,

On Fri, 18. Apr 2014 at 12:20:15 +0200, Martin Landa wrote:
> [...] I still think that we should provide in GRASS API function like
> G_set_fatal_error() with default value G_FATAL_ERROR_EXIT (current
> behaviour). The second option would be G_FATAL_ERROR_RETURN with big big
> warning in the API manual that you are living GRASS libraries in completely
> unpredictable state and you should immediately stop an application which is
> using GRASS libraries.

I also don't think that would help much.  You would still need to use
G_set_error_routine to track that there was an fatal error and in turn avoid a
crash by not touching the result when the call returns - and keep away from
calling other library function because of the unpredictable state of the
library.

On Fri, 18. Apr 2014 at 23:59:25 +0100, Glynn Clements wrote:
> It achieves this by calling exit() (or optionally raising SIGSEGV; why it
> didn't use SIGABRT, I have no idea). Calling longjmp() would work.  If it was
> C++, raising an exception would work. Returning will *not* work.

I guess there are also library calls inside the libraries themselves and those
would not be aware that there is a need to handle invalid results and might
crash before the toplevel caller has a chance know of the error.

I think that's Glynn's keypoint.

Using setjmp()/longjmp() or throwing C++ exceptions (with the libraries built
with -fexceptions) in the error routine would take you back to the caller that
is aware of the error.   That avoids the crash, but also leaves the library in
an unpredictable state.

To avoid that there's no other way than returning and handling all the errors
properly.  Probably huge amount of work with not much gain for GRASS.

The RPC approach will IMHO be less work and also cause less friction.


Jürgen

-- 
Jürgen E. Fischer norBIT GmbH   Tel. +49-4931-918175-31
Dipl.-Inf. (FH)   Rheinstraße 13Fax. +49-4931-918175-50
Software Engineer D-26506 Norden   http://www.norbit.de
QGIS PSC member (RM)  Germany  IRC: jef on FreeNode 


-- 
norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH
Rheinstrasse 13, 26506 Norden
GF: Jelto Buurman, HR: Amtsgericht Emden, HRB 5502

___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Martin Landa]

2014-04-19 0:59 GMT+02:00 Glynn Clements :

[...]

> You're misunderstanding a key point.

"Misunderstanding" is usually a question of the point of view or level
of "doggedness". Especially when someone is speaking about "a key
point" ;-)

> Returning from G_fatal_error() won't leave GRASS libraries in an
> unpredictable state. It's "escaping" from G_fatal_error() via e.g.
> Returning from G_fatal_error() will typically result in a
> (more-or-less immediate) segfault when the caller tries to use data
> structures which contain garbage (because the callee never initialised
> them, but raised a fatal error instead).

Well, and what do you think that I meant by "unpredictable state"?

> The simplest example is G__malloc():
>
> buf = malloc(n);
> if (!buf) {
>
> ...
>
> G_fatal_error(_("G_malloc: unable to allocate %lu bytes of memory at 
> %s:%d"),
>   (unsigned long) n, file, line);
> }
>
> return buf;
>
> If malloc() returns NULL and the subsequent G_fatal_error() call
> returns, G__malloc() will return NULL to its caller. Which will
> segfault as soon as it tries to store something in the "allocated"
> memory.

You chose very specific case of course. There is probably something
you don't want to understand. In the core libraries (gis, vector,
raster, imagery) there are 553(!) places where G_fatal_error() is
called. Some time ago you introduced G_fatal_error() even to the
function which tries to open raster map [1]. So if the raster map
doesn't exists the function is calling exit()! This is something which
makes life harder for anyone who want to use GRASS libraries (as you
already mentioned) with one exception - modules - standalone programs,
where you don't need to specify `if` statement. You can hardly explain
to someone that our libraries call fatal exit even when raster map is
not found. Yes, you can if don't want to change your opinion
regardless any input from outside world. For such cases, like opening
raster map would G_set_fatal_error() simply make sense.

> It achieves this by calling exit() (or optionally raising SIGSEGV; why
> it didn't use SIGABRT, I have no idea). Calling longjmp() would work.
> If it was C++, raising an exception would work. Returning will *not*
> work.

[...]

Martin

[1] 
http://trac.osgeo.org/grass/changeset/40209/grass/trunk/lib/raster/opencell.c

-- 
Martin Landa * http://geo.fsv.cvut.cz/gwiki/Landa
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Vaclav Petras]

On Fri, Apr 18, 2014 at 6:59 PM, Glynn Clements wrote:

> The key feature of G_fatal_error() isn't printing error messages. The
> key feature is that it doesn't return.
>

This is one of the problems of G_fatal_error(), it does not gives good
error messages to the user. User gets "G_malloc: unable to allocate %lu
bytes of memory at %s:%d" but does not have any idea when and why this
happened and what to do about it. If the module calling G_malloc() had the
chance to report error by itself, it could suggest to user to change some
parameter or use a smaller map. With G_fatal_error() user has only the
possibility to try to guess from debug messages, ask on mailing list or
keep trying with random changes in input parameters. I don't believe that
this is efficient.

G_fatal_error() is too low level not only for messages but in general, for
Python libraries wrapping GRASS library, it does not allow to throw an
exception in case of an error. And the same applies for C++ wrappers.

As far as I understand, the RPC wrapper suggested earlier by Soeren, would
hopefully allow to have exceptions (and no exit()) in Python and C++ and
thus potentially higher level error messages and also general error
handling would be possible (for those not using the original library).
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Glynn Clements]

Martin Landa wrote:

> > Unfortunately GRASS 7 moved ahead towards its aim "to make life harder
> > for anyone trying to use the GRASS libraries" [1]. Basically more and
> 
> personally I am not fan of this approach. I still think that we should
> provide in GRASS API function like G_set_fatal_error() with default
> value G_FATAL_ERROR_EXIT (current behaviour). The second option would
> be G_FATAL_ERROR_RETURN with big big warning in the API manual that
> you are living GRASS libraries in completely unpredictable state

You're misunderstanding a key point.

Returning from G_fatal_error() won't leave GRASS libraries in an
unpredictable state. It's "escaping" from G_fatal_error() via e.g. 
longjmp() (so that control doesn't return to the caller) that will do
that.

Returning from G_fatal_error() will typically result in a
(more-or-less immediate) segfault when the caller tries to use data
structures which contain garbage (because the callee never initialised
them, but raised a fatal error instead).

The simplest example is G__malloc():

buf = malloc(n);
if (!buf) {

...

G_fatal_error(_("G_malloc: unable to allocate %lu bytes of memory at 
%s:%d"),
  (unsigned long) n, file, line);
}

return buf;

If malloc() returns NULL and the subsequent G_fatal_error() call
returns, G__malloc() will return NULL to its caller. Which will
segfault as soon as it tries to store something in the "allocated"
memory.

The key feature of G_fatal_error() isn't printing error messages. The
key feature is that it doesn't return.

It achieves this by calling exit() (or optionally raising SIGSEGV; why
it didn't use SIGABRT, I have no idea). Calling longjmp() would work. 
If it was C++, raising an exception would work. Returning will *not*
work.

-- 
Glynn Clements 
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Sören Gebbert]

Hi Radim,
IMHO we GRASS developers are  too stubborn to change the design of
GRASS GIS to work as library with persistent applications. This would
require rewriting plenty of core functionality and modification of all
C-modules. But there is a solution for persistent applications, called
Remote Procedure Call (RPC), more below.

2014-04-18 11:31 GMT+02:00 Radim Blazek :
> I have upgraded  the vector provider to GRASS 7, layers may be added
> by drag from browser. The raster and the plugin are disabled. Be
> careful about multiple versions on the same system
> (LD_LIBRARY_PATH..., check with ldd if does not work).
>
> Unfortunately GRASS 7 moved ahead towards its aim "to make life harder
> for anyone trying to use the GRASS libraries" [1]. Basically more and
> more low level functions are calling exit() instead of returning error
> code if something failed. As I am not willing to implement GRASS
> module call for each simple function, we have to think again about
> hacks we can use:
>
> 1) add a requirement that GRASS 7 used with QGIS must be compiled with
> -fexceptions
>
> 2) add a requirement that a patch (it is a single line comment in
> fact) must be applied to GRASS 7 to make it usable with QGIS
>
> 3) use setjmp()/longjmp()
>
> 4) let QGIS crash whenever GRASS lib function fails

I fear that none of these suggestions are good working solutions,
since in case a fatal error or a segfault occurs nothing (no
exception, no setjmp) will prevent QGIS from crashing. So i would like
to suggest the following:

5.) Using a RPC interface for map metadata, vector and raster map
access. Hence one or several GRASS "server" processes provide an RPC
interface to access GRASS library functions that are needed in a
persistent application. Most important is the vector editing, vector
reading/writing, database access, raster reading and map metadata
support. Everything else can be done using modules. Hence the RPC
interface will support only a limited subset of the GRASS library
functions. This RPC interface should only be used in the GRASS
provider classes, the GRASS plugin itself can be written in C++ or
Python.

This approach will decouple GRASS from QGIS, since all communication
is done via Inter Process Communication (IPC) using pipes or sockets.
There is no need anymore to catch a fatal error or to link GRASS
libraries directly to QGIS. The GRASS plugin can be implemented GRASS
version independently and so the GRASS data provider, since they will
not use GRASS functions directly only the RPC interface. The RPC
implementation on the side of GRASS will provide a consistent
interface that will not change in case the underlying GRASS API
changes.

I strongly suggest to use an existing RPC framework to implement fast
binary data exchange and the IPC. My favorite is apache thrift[1],
since it supports plenty of programming languages (C/C++, Python,
Java, JavaScript, ...) and provides operating system independent
client and server functionality. It supports exceptions on client side
that can be emitted in case a GRASS server process died because of a
fatal error, SIGINT or segfault.

I am absolutely willing to implement the RPC server side in GRASS
using thrift and C++, providing plenty of Python unit tests that show
howto use it on the client side. We just need to decide what kind of
GRASS library functionality is needed as RPC interface and what can be
done in QGIS with C++/Python directly (gisrc creation, GRASS
environmental variable settings, ...) or using GRASS modules
(g.region, g.gisenv, ...).

[1] https://thrift.apache.org/


Best regards
Soeren

>
> Radim
>
> [1]https://trac.osgeo.org/grass/ticket/869#comment:1
>
> On Thu, Mar 27, 2014 at 11:18 AM, Paolo Cavallini  
> wrote:
>> Hi all.
>> I learned during dinner that GRASS7 RC1 is due very soon. This opens the
>> issue of its functioning in QGIS. IMHO:
>>
>> * the qgis-grass-plugin might stop working (this has to be tested)
>> * some of the module options will be different
>> * new modules will not be available in QGIS.
>>
>> I think we can deal with this in several ways:
>>
>> * dropping the plugin, and concentrate the work on Processing
>> * upgrading both the plugin and Processing.
>>
>> In the first case, we have two major issues:
>>
>> * upgrading Processing GRASS modules
>> * changing the current Processing behaviour, avoiding the import-export
>> phase when piping consecutive GRASS commands; this makes GRASS modules
>> slower than the equivalent commands of other backends.
>>
>> While the first issue can be solved easily by a couple of people in a
>> few days, the second one is more tricky, and requires hard coding skills.
>> In addition, we'll no longer be able to edit GRASS vectors directly.
>>
>> In the second case, we'll have more work, and a not-so-nice duplication.
>>
>> I would like to have an open discussion on this, avoiding things to just
>> happen, with the possible negative consequences.
>>
>> All the best.
>> --
>> Paolo Cavallini - www.faunal

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Martin Landa]

Hi,

2014-04-18 11:31 GMT+02:00 Radim Blazek :

[]

> Unfortunately GRASS 7 moved ahead towards its aim "to make life harder
> for anyone trying to use the GRASS libraries" [1]. Basically more and

personally I am not fan of this approach. I still think that we should
provide in GRASS API function like G_set_fatal_error() with default
value G_FATAL_ERROR_EXIT (current behaviour). The second option would
be G_FATAL_ERROR_RETURN with big big warning in the API manual that
you are living GRASS libraries in completely unpredictable state and
you should immediately stop an application which is using GRASS
libraries. There no so big chance that someone will rewrite GRASS
libraries to be suitable for long running applications, on the other
hand we should allow in API possibility to avoid calling exit with big
warning to the programmer. Than it will be his/her responsibility if
he/she uses G_FATAL_ERROR_RETURN.

Martin
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Moritz Lennert]

On 08/04/14 15:31, Radim Blazek wrote:

On Tue, Apr 8, 2014 at 2:18 PM, Moritz Lennert
 wrote:

On 28/03/14 15:16, Paolo Cavallini wrote:


Il 28/03/2014 15:04, Martin Dobias ha scritto:


On Fri, Mar 28, 2014 at 11:48 AM, Paolo Cavallini 
wrote:


* to add GRASS browsing capabilities in QGIS file browser



The support for GRASS is already in the browser - if you enter a
directory that is a GRASS database, it will detect it and show
locations/mapsets/maps/layers.



That's great news, never tried. Works beautifully, thanks.



I cannot reproduce this. How does it work exactly ? When I browse into a
GRASS database directory, it just shows up like any other directory.


The GRASS location is added as another item with GRASS icon in the
tree on the same level where is the location dir. It means that the
location will appear twice, first as regular dir and then as a GRASS
location. The items do not seem to be sorted by name, so maybe you
have to scroll down.


That was it. I just never thought to go down further...

Thanks !

Moritz
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[William Kyngesburye]

This is how it looks for me (Mt Lion, should be same on Mavericks) - you see the grass mapsets inside the grass DB (I have some other junk in mine), the DB doesn't get any special treatment:On Apr 8, 2014, at 9:03 AM, Radim Blazek  wrote:On Tue, Apr 8, 2014 at 3:54 PM, Rainer M Krug  wrote:Radim Blazek  writes:On Tue, Apr 8, 2014 at 2:18 PM, Moritz Lennert wrote:On 28/03/14 15:16, Paolo Cavallini wrote:Il 28/03/2014 15:04, Martin Dobias ha scritto:On Fri, Mar 28, 2014 at 11:48 AM, Paolo Cavallini wrote:* to add GRASS browsing capabilities in QGIS file browserThe support for GRASS is already in the browser - if you enter adirectory that is a GRASS database, it will detect it and showlocations/mapsets/maps/layers.That's great news, never tried. Works beautifully, thanks.I cannot reproduce this. How does it work exactly ? When I browse into aGRASS database directory, it just shows up like any other directory.The GRASS location is added as another item with GRASS icon in thetree on the same level where is the location dir. It means that thelocation will appear twice, first as regular dir and then as a GRASSlocation. The items do not seem to be sorted by name, so maybe youhave to scroll down.Hm - I think I also need a step-by-step guide. I am using QGIS 2.2.0 onOSX Maverick and I don't get it to work.0) Open QGIS1) I open the browser (View - Panels - Browser)2) Then I go via the Home folder (top entry) to a GRASS database3) and now, where should I see what? What is the root in the treestructure? I can only see the folder structure.Probably a bug. It is checking for location/PERMANENT/DEFAULT_WIND.You have maybe lower case names? If so, can you test with upper?Please fill a new issue.RadimRainerRadimMoritz___grass-dev mailing listgrass-dev@lists.osgeo.orghttp://lists.osgeo.org/mailman/listinfo/grass-dev___grass-dev mailing listgrass-dev@lists.osgeo.orghttp://lists.osgeo.org/mailman/listinfo/grass-dev--Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, UCT), Dipl. Phys. (Germany)Centre of Excellence for Invasion BiologyStellenbosch UniversitySouth AfricaTel :   +33 - (0)9 53 10 27 44Cell:   +33 - (0)6 85 62 59 98Fax :   +33 - (0)9 58 10 27 44Fax (D):    +49 - (0)3 21 21 25 22 44email:  rai...@krugs.deSkype:  RMkrugPGP: 0x0F52F982___grass-dev mailing listgrass-dev@lists.osgeo.orghttp://lists.osgeo.org/mailman/listinfo/grass-dev-William Kyngesburye http://www.kyngchaos.com/"History is an illusion caused by the passage of time, and time is an illusion caused by the passage of history."- Hitchhiker's Guide to the Galaxy___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Radim Blazek]

On Tue, Apr 8, 2014 at 3:54 PM, Rainer M Krug  wrote:
> Radim Blazek  writes:
>
>> On Tue, Apr 8, 2014 at 2:18 PM, Moritz Lennert
>>  wrote:
>>> On 28/03/14 15:16, Paolo Cavallini wrote:

 Il 28/03/2014 15:04, Martin Dobias ha scritto:
>
> On Fri, Mar 28, 2014 at 11:48 AM, Paolo Cavallini 
> wrote:
>>
>> * to add GRASS browsing capabilities in QGIS file browser
>
>
> The support for GRASS is already in the browser - if you enter a
> directory that is a GRASS database, it will detect it and show
> locations/mapsets/maps/layers.


 That's great news, never tried. Works beautifully, thanks.
>>>
>>>
>>> I cannot reproduce this. How does it work exactly ? When I browse into a
>>> GRASS database directory, it just shows up like any other directory.
>>
>> The GRASS location is added as another item with GRASS icon in the
>> tree on the same level where is the location dir. It means that the
>> location will appear twice, first as regular dir and then as a GRASS
>> location. The items do not seem to be sorted by name, so maybe you
>> have to scroll down.
>
> Hm - I think I also need a step-by-step guide. I am using QGIS 2.2.0 on
> OSX Maverick and I don't get it to work.
>
> 0) Open QGIS
> 1) I open the browser (View - Panels - Browser)
> 2) Then I go via the Home folder (top entry) to a GRASS database
> 3) and now, where should I see what? What is the root in the tree
> structure? I can only see the folder structure.

Probably a bug. It is checking for location/PERMANENT/DEFAULT_WIND.
You have maybe lower case names? If so, can you test with upper?

Please fill a new issue.

Radim

> Rainer
>
>
>>
>> Radim
>>
>>
>>> Moritz
>>>
>>> ___
>>> grass-dev mailing list
>>> grass-dev@lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/grass-dev
>> ___
>> grass-dev mailing list
>> grass-dev@lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/grass-dev
>
> --
> Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, 
> UCT), Dipl. Phys. (Germany)
>
> Centre of Excellence for Invasion Biology
> Stellenbosch University
> South Africa
>
> Tel :   +33 - (0)9 53 10 27 44
> Cell:   +33 - (0)6 85 62 59 98
> Fax :   +33 - (0)9 58 10 27 44
>
> Fax (D):+49 - (0)3 21 21 25 22 44
>
> email:  rai...@krugs.de
>
> Skype:  RMkrug
>
> PGP: 0x0F52F982
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Radim Blazek]

On Tue, Apr 8, 2014 at 3:54 PM, Rainer M Krug  wrote:
> Radim Blazek  writes:
>
>> On Tue, Apr 8, 2014 at 2:18 PM, Moritz Lennert
>>  wrote:
>>> On 28/03/14 15:16, Paolo Cavallini wrote:

 Il 28/03/2014 15:04, Martin Dobias ha scritto:
>
> On Fri, Mar 28, 2014 at 11:48 AM, Paolo Cavallini 
> wrote:
>>
>> * to add GRASS browsing capabilities in QGIS file browser
>
>
> The support for GRASS is already in the browser - if you enter a
> directory that is a GRASS database, it will detect it and show
> locations/mapsets/maps/layers.


 That's great news, never tried. Works beautifully, thanks.
>>>
>>>
>>> I cannot reproduce this. How does it work exactly ? When I browse into a
>>> GRASS database directory, it just shows up like any other directory.
>>
>> The GRASS location is added as another item with GRASS icon in the
>> tree on the same level where is the location dir. It means that the
>> location will appear twice, first as regular dir and then as a GRASS
>> location. The items do not seem to be sorted by name, so maybe you
>> have to scroll down.
>
> Hm - I think I also need a step-by-step guide. I am using QGIS 2.2.0 on
> OSX Maverick and I don't get it to work.
>
> 0) Open QGIS
> 1) I open the browser (View - Panels - Browser)
> 2) Then I go via the Home folder (top entry) to a GRASS database
> 3) and now, where should I see what? What is the root in the tree
> structure? I can only see the folder structure.

Probably a bug. It is checking for location/PERMANENT/DEFAULT_WIND.
You have maybe lower case names? If so, can you test with upper?

Please fill a new issue.

Radim

> Rainer
>
>
>>
>> Radim
>>
>>
>>> Moritz
>>>
>>> ___
>>> grass-dev mailing list
>>> grass-dev@lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/grass-dev
>> ___
>> grass-dev mailing list
>> grass-dev@lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/grass-dev
>
> --
> Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, 
> UCT), Dipl. Phys. (Germany)
>
> Centre of Excellence for Invasion Biology
> Stellenbosch University
> South Africa
>
> Tel :   +33 - (0)9 53 10 27 44
> Cell:   +33 - (0)6 85 62 59 98
> Fax :   +33 - (0)9 58 10 27 44
>
> Fax (D):+49 - (0)3 21 21 25 22 44
>
> email:  rai...@krugs.de
>
> Skype:  RMkrug
>
> PGP: 0x0F52F982
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Rainer M Krug]

Radim Blazek  writes:

> On Tue, Apr 8, 2014 at 2:18 PM, Moritz Lennert
>  wrote:
>> On 28/03/14 15:16, Paolo Cavallini wrote:
>>>
>>> Il 28/03/2014 15:04, Martin Dobias ha scritto:

 On Fri, Mar 28, 2014 at 11:48 AM, Paolo Cavallini 
 wrote:
>
> * to add GRASS browsing capabilities in QGIS file browser


 The support for GRASS is already in the browser - if you enter a
 directory that is a GRASS database, it will detect it and show
 locations/mapsets/maps/layers.
>>>
>>>
>>> That's great news, never tried. Works beautifully, thanks.
>>
>>
>> I cannot reproduce this. How does it work exactly ? When I browse into a
>> GRASS database directory, it just shows up like any other directory.
>
> The GRASS location is added as another item with GRASS icon in the
> tree on the same level where is the location dir. It means that the
> location will appear twice, first as regular dir and then as a GRASS
> location. The items do not seem to be sorted by name, so maybe you
> have to scroll down.

Hm - I think I also need a step-by-step guide. I am using QGIS 2.2.0 on
OSX Maverick and I don't get it to work.

0) Open QGIS
1) I open the browser (View - Panels - Browser)
2) Then I go via the Home folder (top entry) to a GRASS database 
3) and now, where should I see what? What is the root in the tree
structure? I can only see the folder structure.

Rainer


>
> Radim
>
>
>> Moritz
>>
>> ___
>> grass-dev mailing list
>> grass-dev@lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/grass-dev
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-dev

-- 
Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, 
UCT), Dipl. Phys. (Germany)

Centre of Excellence for Invasion Biology
Stellenbosch University
South Africa

Tel :   +33 - (0)9 53 10 27 44
Cell:   +33 - (0)6 85 62 59 98
Fax :   +33 - (0)9 58 10 27 44

Fax (D):+49 - (0)3 21 21 25 22 44

email:  rai...@krugs.de

Skype:  RMkrug

PGP: 0x0F52F982
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Rainer M Krug]

Radim Blazek  writes:

> On Tue, Apr 8, 2014 at 2:18 PM, Moritz Lennert
>  wrote:
>> On 28/03/14 15:16, Paolo Cavallini wrote:
>>>
>>> Il 28/03/2014 15:04, Martin Dobias ha scritto:

 On Fri, Mar 28, 2014 at 11:48 AM, Paolo Cavallini 
 wrote:
>
> * to add GRASS browsing capabilities in QGIS file browser


 The support for GRASS is already in the browser - if you enter a
 directory that is a GRASS database, it will detect it and show
 locations/mapsets/maps/layers.
>>>
>>>
>>> That's great news, never tried. Works beautifully, thanks.
>>
>>
>> I cannot reproduce this. How does it work exactly ? When I browse into a
>> GRASS database directory, it just shows up like any other directory.
>
> The GRASS location is added as another item with GRASS icon in the
> tree on the same level where is the location dir. It means that the
> location will appear twice, first as regular dir and then as a GRASS
> location. The items do not seem to be sorted by name, so maybe you
> have to scroll down.

Hm - I think I also need a step-by-step guide. I am using QGIS 2.2.0 on
OSX Maverick and I don't get it to work.

0) Open QGIS
1) I open the browser (View - Panels - Browser)
2) Then I go via the Home folder (top entry) to a GRASS database 
3) and now, where should I see what? What is the root in the tree
structure? I can only see the folder structure.

Rainer


>
> Radim
>
>
>> Moritz
>>
>> ___
>> grass-dev mailing list
>> grass-dev@lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/grass-dev
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-dev

-- 
Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, 
UCT), Dipl. Phys. (Germany)

Centre of Excellence for Invasion Biology
Stellenbosch University
South Africa

Tel :   +33 - (0)9 53 10 27 44
Cell:   +33 - (0)6 85 62 59 98
Fax :   +33 - (0)9 58 10 27 44

Fax (D):+49 - (0)3 21 21 25 22 44

email:  rai...@krugs.de

Skype:  RMkrug

PGP: 0x0F52F982


pgpfVIFAv6fbL.pgp
Description: PGP signature
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Radim Blazek]

On Tue, Apr 8, 2014 at 2:18 PM, Moritz Lennert
 wrote:
> On 28/03/14 15:16, Paolo Cavallini wrote:
>>
>> Il 28/03/2014 15:04, Martin Dobias ha scritto:
>>>
>>> On Fri, Mar 28, 2014 at 11:48 AM, Paolo Cavallini 
>>> wrote:

 * to add GRASS browsing capabilities in QGIS file browser
>>>
>>>
>>> The support for GRASS is already in the browser - if you enter a
>>> directory that is a GRASS database, it will detect it and show
>>> locations/mapsets/maps/layers.
>>
>>
>> That's great news, never tried. Works beautifully, thanks.
>
>
> I cannot reproduce this. How does it work exactly ? When I browse into a
> GRASS database directory, it just shows up like any other directory.

The GRASS location is added as another item with GRASS icon in the
tree on the same level where is the location dir. It means that the
location will appear twice, first as regular dir and then as a GRASS
location. The items do not seem to be sorted by name, so maybe you
have to scroll down.

Radim


> Moritz
>
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Moritz Lennert]

On 28/03/14 15:16, Paolo Cavallini wrote:

Il 28/03/2014 15:04, Martin Dobias ha scritto:

On Fri, Mar 28, 2014 at 11:48 AM, Paolo Cavallini  wrote:

* to add GRASS browsing capabilities in QGIS file browser


The support for GRASS is already in the browser - if you enter a
directory that is a GRASS database, it will detect it and show
locations/mapsets/maps/layers.


That's great news, never tried. Works beautifully, thanks.


I cannot reproduce this. How does it work exactly ? When I browse into a 
GRASS database directory, it just shows up like any other directory.


Moritz
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Paolo Cavallini]

Il 28/03/2014 15:16, Paolo Cavallini ha scritto:
> Il 28/03/2014 15:04, Martin Dobias ha scritto:

>> The support for GRASS is already in the browser - if you enter a
>> directory that is a GRASS database, it will detect it and show
>> locations/mapsets/maps/layers.
> 
> In the GRASS plugin there are a few more features, i.e.:
> * showing history (important)
> * copy/rename/delete.
> If we can add these, I think the corresponding section of the plugin
> would be rather useless.
> Am I missing something?

What about writing down the minumum set of requirements to be preserved?
Please add it to this thread, we can note it on a wiki page once bolied
down.
All the best.
-- 
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Paolo Cavallini]

Il 28/03/2014 15:04, Martin Dobias ha scritto:
> On Fri, Mar 28, 2014 at 11:48 AM, Paolo Cavallini  
> wrote:
>> * to add GRASS browsing capabilities in QGIS file browser
> 
> The support for GRASS is already in the browser - if you enter a
> directory that is a GRASS database, it will detect it and show
> locations/mapsets/maps/layers.

That's great news, never tried. Works beautifully, thanks.
In the GRASS plugin there are a few more features, i.e.:
* showing history (important)
* copy/rename/delete.
If we can add these, I think the corresponding section of the plugin
would be rather useless.
Am I missing something?
All the best.
-- 
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Benjamin Ducke]

One last addition to this from my side:

Since, as I mentioned, we have evaluated many of the
issues discussed in this thread in the design of the
GRASS plug-in for SEXTANTE/gvSIG CE, here are some things
you might want to take into consideration:

On 28/03/14 11:48, Paolo Cavallini wrote:
> Il 28/03/2014 08:34, Paolo Cavallini ha scritto:
> 
>> Thanks to all for the interesting comments. Obviously we have a varied
>> and rich ecosystem here, that's why I want to preserve it.
>> Obviously we all want to have all. IMHO the points are:
>>
>> * can we (GRASS+QGIS) agree on a specific course of actions?
>> * can we find the resource to implement it?
>>
>> Of course we could think of sticking with GRASS6, but I think sooner or
>> later this will be untenable (e.g. major distros will switch to GRASS7
>> once out), so better prepare now.
> 
> Quick interesting chat with Luca Delucchi and Martin Landa from GRASS
> dev team.
> One interesting option would be:
> * to add GRASS browsing capabilities in QGIS file browser

Browsing capabilities for existing GRASS mapsets would be
nice, but having the plug-in directly manipulate the data
in there leads down a difficult path (see my notes further
below).

> * to add support to direct GRASS reading (possibly writing) in
> Processing, and avoid import/export whenever possible; v.external and
> v.out.external could be used for this; this is probably the thoughest part

As far as I understand, the v.out.* modules create read-only
datasets. They also create minimal topological data structures
that are not enough for many GRASS modules (correct me if any of
these statements is no longer true). Therefore, we decided against
this path and use v.in/out.ogr instead -- with all known problems.

But r.external seems to work fine, so that at least raster I/O is
efficient enough; with the one limitation being that GRASS does not
support multi-band rasters in single files (we do not have a good
solution for working around that problem yet).

> * of course modules should be upgraded anyway
> * GRASS direct digitizing can be skipped; heavy digitizer can use GRASS
> directly

(see below)

> * keep a GRASS shell?

This only works well with the original GRASS plug-in design, not
with the one derived from SEXTANTE:

Any plug-in design that allows direct manipulation of existing
GRASS data and/or runs GRASS modules on existing mapsets is
not very compatible with the SEXTANTE/Processing on-the-fly
"import/process/export/delete temp" approach. As long as the GRASS
mapset is guaranteed to be temporary, the above chain is clean
and simple. But if you give the plug-in access to existing mapset
data, then you need a nice, thick wrapper of additional safety
checks to make sure that the final "delete" step *never* deletes
any pre-existing user data! Not only would you have to keep track
of each and every bit of data that an intermediate processing step
creates (modules that call other modules!), but also make sure
that no stale data is left in the user's mapset if processing does
not complete.

This is all very, very tricky and normally the role of the GRASS user
who knows about GRASS data management and is in full control.
I.e. if you want to make this safe, then you have to expose the GRASS
mapset and management completely to the user --> and this in turn
defeats the idea of radically simplifying the use of GRASS modules.

This is the reason why in gvSIG CE we steered clear of existing
user mapsets; our conviction being that data safety is always more
important than convenience and that advanced users can/will just run
GRASS by themselves.

Best,

Ben

> * with all the above, upgrading the plugin may probably be skipped.
> 
> Any thoughts.
> 



-- 
Dr. Benjamin Ducke, M.A.
{*} Geospatial Consultant
{*} GIS Developer

  bendu...@fastmail.fm
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Paolo Cavallini]

Il 28/03/2014 08:34, Paolo Cavallini ha scritto:

> Thanks to all for the interesting comments. Obviously we have a varied
> and rich ecosystem here, that's why I want to preserve it.
> Obviously we all want to have all. IMHO the points are:
> 
> * can we (GRASS+QGIS) agree on a specific course of actions?
> * can we find the resource to implement it?
> 
> Of course we could think of sticking with GRASS6, but I think sooner or
> later this will be untenable (e.g. major distros will switch to GRASS7
> once out), so better prepare now.

Quick interesting chat with Luca Delucchi and Martin Landa from GRASS
dev team.
One interesting option would be:
* to add GRASS browsing capabilities in QGIS file browser
* to add support to direct GRASS reading (possibly writing) in
Processing, and avoid import/export whenever possible; v.external and
v.out.external could be used for this; this is probably the thoughest part
* of course modules should be upgraded anyway
* GRASS direct digitizing can be skipped; heavy digitizer can use GRASS
directly
* keep a GRASS shell?
* with all the above, upgrading the plugin may probably be skipped.

Any thoughts.
-- 
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Saber Razmjooei]

Hi all,

Here is my wish list:

- To be able to access GRASS data within QGIS. Similar to PostGIS and other
GDAL/OGR data sources, it should be up to users to create GRASS
geodatabase/location/mapsets. QGIS will be able to simply add layers for
viewing and editing 

- Having access to GRASS shell. It will be a very handy shortcut to run
GRASS modules without the need to open GRASS. GRASS modules which are
available in Processing toolbox can use the same UI. Otherwise, user has to
run it in shell.

- Simple import data to the mapsets...similar approach as drag-n-drop for
PostGIS databases. Right-click and save as... in QGIS does a great work for
Export.


In summary, Processing toolbox should be a good one-stop for one-off use of
a specific GRASS module. But if a user wants to dig more, GRASS already
offers all that. For those who have already got their GRASS set up, they
should be able to view and edit their data in QGIS with the added benefit of
having access to shell.

Some of the stuff can be ported to plugin(s) as mentioned by others: e.g.
creating location/mapsets, setting region from qgis canvas or other qgis
layers, etc.

Cheers,

Saber




-Original Message-
From: grass-dev-boun...@lists.osgeo.org
[mailto:grass-dev-boun...@lists.osgeo.org] On Behalf Of Benjamin Ducke
Sent: 27 March 2014 14:31
To: grass-dev
Subject: Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

In the gvSIG CE project, we had to make the same type of decisions and came
to our own conclusions.

Allow me to summarize our reasoning, maybe it will be useful for the QGIS
project, as well:

1. The number one cause of irritations among novice users is having to set
up a GRASS mapset and having to understand how GRASS data management works.

2. The number two cause of irritations are the effects of importing vector
data with bad topology into a GRASS mapset.

3. The original QGIS plugin does nothing to alleviate (1).
No plug-in, however cleverly designed, can do anything about (2): garbage
in, garbage out.

4. GRASS "power users" gain very little (if anything) from running GRASS
through a host GIS, such as QGIS or gvSIG. But they do lose functionality,
such as the
d.* family of modules.

Therefore, we gave up trying to design a plug-in for advanced users. We
assume that such users will use GRASS through its native CLI and/or native
GUI.

The resulting design of the original SEXTANTE-GRASS interface (which is now
also mirrored in the Python re-write that became QGIS' Processing) addresses
users that either don't care much for GRASS' CLI capabilities and internal
data management, or are willing to switch to native GRASS as needed.

If you want to change this and address another type of user, then you will
need to re-examine the entire design of the current SEXTANTE/Processing
approach, which is to use only temporary mapsets and perform data
import/export every time a GRASS module is run.

You can optimize the I/O performance of Processing by using r.external to
directly access raster input maps. But there is little you can do about
vector data with the current design, as GRASS needs to build its own
topological data structures (and rebuild them every time you run a GRASS
module on non-topological input!).

In any case, I do not think that it is worth maintaining the original QGIS
plugin, since it is neither very well suited for novice nor advanced users,
IMHO.

Best,

Ben


On 27/03/14 14:38, Blumentrath, Stefan wrote:
> That sounds very reasonable to me.
> 
> Proper GDAL/OGR handling for GRASS 7 would be very nice in any case (see
http://trac.osgeo.org/gdal/ticket/2953).
> 
> As for a "GRASS data browser", I think, a plugin would be required with
regards to user friendliness, because one needs to know what files to access
from a GRASS data base when using GDAL.
> 
> I also understand that at some point in time one will have to use GRASS
directly in order to access full functionality (e.g. ortho-rectification,
nviz, mapswipe, animation and stuff), which makes the way Moritz suggests
maybe even more reasonable...
> 
> Cheers
> Stefan
>  
> 
> 
> 
> -Original Message-
> From: Moritz Lennert [mailto:mlenn...@club.worldonline.be]
> Sent: 27. mars 2014 13:36
> To: Blumentrath, Stefan
> Cc: Paolo Cavallini; Nathan Woodrow; qgis-developer; grass-dev
> Subject: Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future
> 
> I'm not much of a QGIS-as-GRASS-frontend user, either, but from a 
> general point of view I also think that duplication is a problem and 
> that the current way to go in QGIS is the processing framework. So;
> 
> On 27/03/14 12:49, Blumentrath, Stefan wrote:
>> I understand well the point; however, the plugin has additional
functions, e.g.:
>> * a grass shell
> 
> couldn't this be implemented within the processing environment ?
> 
>> * 

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Benjamin Ducke]

In the gvSIG CE project, we had to make the same
type of decisions and came to our own conclusions.

Allow me to summarize our reasoning, maybe it will be
useful for the QGIS project, as well:

1. The number one cause of irritations among novice
users is having to set up a GRASS mapset and having
to understand how GRASS data management works.

2. The number two cause of irritations are the effects
of importing vector data with bad topology into a
GRASS mapset.

3. The original QGIS plugin does nothing to alleviate (1).
No plug-in, however cleverly designed, can do anything
about (2): garbage in, garbage out.

4. GRASS "power users" gain very little (if anything)
from running GRASS through a host GIS, such as QGIS
or gvSIG. But they do lose functionality, such as the
d.* family of modules.

Therefore, we gave up trying to design a plug-in for
advanced users. We assume that such users will use
GRASS through its native CLI and/or native GUI.

The resulting design of the original SEXTANTE-GRASS
interface (which is now also mirrored in the Python
re-write that became QGIS' Processing) addresses users
that either don't care much for GRASS' CLI capabilities
and internal data management, or are willing to switch to
native GRASS as needed.

If you want to change this and address another type
of user, then you will need to re-examine the entire design
of the current SEXTANTE/Processing approach, which is to use
only temporary mapsets and perform data import/export every
time a GRASS module is run.

You can optimize the I/O performance of Processing by using
r.external to directly access raster input maps. But there
is little you can do about vector data with the current
design, as GRASS needs to build its own topological data
structures (and rebuild them every time you run a GRASS module
on non-topological input!).

In any case, I do not think that it is worth maintaining
the original QGIS plugin, since it is neither very well
suited for novice nor advanced users, IMHO.

Best,

Ben


On 27/03/14 14:38, Blumentrath, Stefan wrote:
> That sounds very reasonable to me.
> 
> Proper GDAL/OGR handling for GRASS 7 would be very nice in any case (see 
> http://trac.osgeo.org/gdal/ticket/2953).
> 
> As for a "GRASS data browser", I think, a plugin would be required with 
> regards to user friendliness, because one needs to know what files to access 
> from a GRASS data base when using GDAL.
> 
> I also understand that at some point in time one will have to use GRASS 
> directly in order to access full functionality (e.g. ortho-rectification, 
> nviz, mapswipe, animation and stuff), which makes the way Moritz suggests 
> maybe even more reasonable...
> 
> Cheers
> Stefan
>  
> 
> 
> 
> -Original Message-
> From: Moritz Lennert [mailto:mlenn...@club.worldonline.be] 
> Sent: 27. mars 2014 13:36
> To: Blumentrath, Stefan
> Cc: Paolo Cavallini; Nathan Woodrow; qgis-developer; grass-dev
> Subject: Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future
> 
> I'm not much of a QGIS-as-GRASS-frontend user, either, but from a general 
> point of view I also think that duplication is a problem and that the current 
> way to go in QGIS is the processing framework. So;
> 
> On 27/03/14 12:49, Blumentrath, Stefan wrote:
>> I understand well the point; however, the plugin has additional functions, 
>> e.g.:
>> * a grass shell
> 
> couldn't this be implemented within the processing environment ?
> 
>> * a grass data browser
> 
> If we are talking about accessing GRASS data for loading into QGIS, wouldn't 
> it be a better idea to improve the GDAL/OGR handling in order to be able to 
> load GRASS data just like any other data format ?
> 
>> * a grass digitizing environment.
> 
> Maybe this could be split out into a specific plugin ?
> 
> Moritz
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-dev
> 



-- 
Dr. Benjamin Ducke, M.A.
{*} Geospatial Consultant
{*} GIS Developer

  bendu...@fastmail.fm
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Vaclav Petras]

On Thu, Mar 27, 2014 at 9:38 AM, Blumentrath, Stefan <
stefan.blumentr...@nina.no> wrote:

> I also understand that at some point in time one will have to use GRASS
> directly in order to access full functionality (e.g. ortho-rectification,
> nviz, mapswipe, animation and stuff), which makes the way Moritz suggests
> maybe even more reasonable...


I hope that we find the way to start these tools from QGIS. Tck/Tk nviz is
starting in this way. I hope that it will be possible to do the same for
g.gui.* modules.

Also, it would be nice if the QGIS GRASS vector digitizer (plugin/tool)
would use the same backend as vector digitizer in GRASS wxGUI (Python
subprocessing exit-safe wrapper). Now the first problem is that there is no
reusable backend in wxGUI.

Vaclav
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Blumentrath, Stefan]

That sounds very reasonable to me.

Proper GDAL/OGR handling for GRASS 7 would be very nice in any case (see 
http://trac.osgeo.org/gdal/ticket/2953).

As for a "GRASS data browser", I think, a plugin would be required with regards 
to user friendliness, because one needs to know what files to access from a 
GRASS data base when using GDAL.

I also understand that at some point in time one will have to use GRASS 
directly in order to access full functionality (e.g. ortho-rectification, nviz, 
mapswipe, animation and stuff), which makes the way Moritz suggests maybe even 
more reasonable...

Cheers
Stefan
 



-Original Message-
From: Moritz Lennert [mailto:mlenn...@club.worldonline.be] 
Sent: 27. mars 2014 13:36
To: Blumentrath, Stefan
Cc: Paolo Cavallini; Nathan Woodrow; qgis-developer; grass-dev
Subject: Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

I'm not much of a QGIS-as-GRASS-frontend user, either, but from a general point 
of view I also think that duplication is a problem and that the current way to 
go in QGIS is the processing framework. So;

On 27/03/14 12:49, Blumentrath, Stefan wrote:
> I understand well the point; however, the plugin has additional functions, 
> e.g.:
> * a grass shell

couldn't this be implemented within the processing environment ?

> * a grass data browser

If we are talking about accessing GRASS data for loading into QGIS, wouldn't it 
be a better idea to improve the GDAL/OGR handling in order to be able to load 
GRASS data just like any other data format ?

> * a grass digitizing environment.

Maybe this could be split out into a specific plugin ?

Moritz
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Moritz Lennert]

I'm not much of a QGIS-as-GRASS-frontend user, either, but from a 
general point of view I also think that duplication is a problem and 
that the current way to go in QGIS is the processing framework. So;


On 27/03/14 12:49, Blumentrath, Stefan wrote:

I understand well the point; however, the plugin has additional functions, e.g.:
* a grass shell


couldn't this be implemented within the processing environment ?


* a grass data browser


If we are talking about accessing GRASS data for loading into QGIS, 
wouldn't it be a better idea to improve the GDAL/OGR handling in order 
to be able to load GRASS data just like any other data format ?



* a grass digitizing environment.


Maybe this could be split out into a specific plugin ?

Moritz
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Blumentrath, Stefan]

Dear all,

>From my user perspective (I am using both GRASS and QGIS, or the other way 
>around, depends on topic), processing is not really a replacement for the 
>GRASS plugin.
It is handy and probably better for those who do not use GRASS but are 
interested in single functions / modules.

Yet, once you have a GRASS database with a considerable amount of data, the 
GRASS plugin is very valuable for accessing those data in QGIS (e.g. for 
cartography). Also users with no or little GRASS experience benefit from the 
GRASS plugin in cases where they have access to a GRASS database.
That way they can work on it without acquainting themselves with a "new" GIS in 
depth...

Concluding, if you have the possibility to maintain it, keep the GRASS plugin 
in QGIS!

Cheers
Stefan




-Original Message-
From: qgis-developer-boun...@lists.osgeo.org 
[mailto:qgis-developer-boun...@lists.osgeo.org] On Behalf Of Paolo Cavallini
Sent: 27. mars 2014 12:41
To: Nathan Woodrow
Cc: qgis-developer; grass-dev
Subject: Re: [Qgis-developer] GRASS & QGIS: the future

Il 27/03/2014 12:33, Nathan Woodrow ha scritto:
> I would vote for dropping the plugin and just updating the processing 
> plugin.  Having both ways is bad for us and bad for users, even worse 
> when some functions are missing from one but not in the other.

I understand well the point; however, the plugin has additional functions, e.g.:
* a grass shell
* a grass data browser
* a grass digitizing environment.
Whether these are important or not, it's a matter of users.
All the best.

--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
___
Qgis-developer mailing list
qgis-develo...@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

[Paolo Cavallini]

Il 27/03/2014 12:33, Nathan Woodrow ha scritto:
> I would vote for dropping the plugin and just updating the processing
> plugin.  Having both ways is bad for us and bad for users, even worse
> when some functions are missing from one but not in the other.

I understand well the point; however, the plugin has additional
functions, e.g.:
* a grass shell
* a grass data browser
* a grass digitizing environment.
Whether these are important or not, it's a matter of users.
All the best.

-- 
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev