Re: [GRASS-dev] Maintainer for rgrass needed by end 2024

[Steven Pawley via grass-dev]

Hi Roger,

Great to hear from you, I’ll take a look at the issue. I’ll also look at 
roxygen2 in a separate request. We predominantly work with GRASS temporary 
sessions/projects - especially for processing large grids, because many other 
GIS software/libraries use in-memory processing.

Kind regards,

Steven




> On Oct 4, 2024, at 5:22 AM, Roger Bivand  wrote:
> 
> Hi Steven,
> 
> Thanks for responding! Could I suggest following up at 
> https://github.com/rsbivand/rgrass/issues/96 ? This also of course is open to 
> any others interested or concerned. rgrass and Rsagacmd overlap a little, but 
> usage of rgrass has the possibility to spin up a temporary location (now 
> project?), and to run GRASS commands constructed through the 
> --interface-description argument, which has now been migrated from XML to 
> xml2. rgrass has also not been roxygenised, as it began well before such 
> tools. All these are things that can be cleared up, probably with the help of 
> others pinged in the github issue I just raised, and others wishing to 
> contribute.
> 
> Best wishes,
> 
> Roger
> 
> --
> Roger Bivand
> Emeritus Professor
> Norwegian School of Economics
> Postboks 3490 Ytre Sandviken, 5045 Bergen, Norway
> roger.biv...@nhh.no
> 
> 
> From: Steven Pawley 
> Sent: 04 October 2024 02:11
> To: Roger Bivand
> Cc: grass-dev@lists.osgeo.org; grass-st...@lists.osgeo.org
> Subject: Re: [GRASS-dev] Maintainer for rgrass needed by end 2024
> 
> [You don't often get email from dr.stevenpaw...@gmail.com. Learn why this is 
> important at https://aka.ms/LearnAboutSenderIdentification ]
> 
> Hi Roger,
> 
> I would be happy to support the R rgrass package. I’m the author of another 
> “bridge” package (Rsagacmd) on CRAN, and I’ve contributed a few add-ons to 
> GRASS, including the scikit-learn / raster add-on r.learn.ml2.
> 
> I've also been working on some R-GRASS integration but bringing in reticulate 
> so that the pygrass interface can be used from R, with the aim of being able 
> to use common generics (particularly ‘predict’) that can work directly with 
> GRASS RasterRow objects etc.
> 
> Irrespectively, happy to take a look at some of the open issues over the next 
> few days.
> 
> Kind regards,
> 
> Steven
> 
> Alberta Geological Survey
> 
>> On Oct 3, 2024, at 5:21 AM, Roger Bivand via grass-dev 
>>  wrote:
>> 
>> In connection with the transfer of the grass-dev list to Discourse, which I 
>> am not prepared to follow as it will absorb much more time than scanning the 
>> digested list headers, I will give up maintaining the R-GRASS interface 
>> package at the end of 2024.
>> 
>> The package lives in https://github.com/rsbivand/rgrass and the repo can be 
>> moved to the preferred location of the new maintainer, and the maintainer 
>> change notified to CRAN without difficulty.
>> 
>> The arrival of Discourse precipitates a change that was started before a 
>> workshop at FOSS4G in 2022 
>> https://rsbivand.github.io/foss4g_2022/modernizing_220822.html, and several 
>> GRASS users/developers then contributed to the workshop, and have helped 
>> improve the package over many years.
>> 
>> Before a hand-over, this group of contributors can go through issues not yet 
>> formally closed but mostly settled, to make sure that the new maintainer 
>> ramps up smoothly.
>> 
>> Best wishes,
>> 
>> Roger
>> 
>> --
>> Roger Bivand
>> Emeritus Professor
>> Norwegian School of Economics
>> Postboks 3490 Ytre Sandviken, 5045 Bergen, Norway
>> roger.biv...@nhh.no
>> ___
>> grass-dev mailing list
>> grass-dev@lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/grass-dev
> 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Maintainer for rgrass needed by end 2024

[Steven Pawley via grass-dev]

Hi Roger,

I would be happy to support the R rgrass package. I’m the author of another 
“bridge” package (Rsagacmd) on CRAN, and I’ve contributed a few add-ons to 
GRASS, including the scikit-learn / raster add-on r.learn.ml2.

I've also been working on some R-GRASS integration but bringing in reticulate 
so that the pygrass interface can be used from R, with the aim of being able to 
use common generics (particularly ‘predict’) that can work directly with GRASS 
RasterRow objects etc.

Irrespectively, happy to take a look at some of the open issues over the next 
few days.

Kind regards,

Steven

Alberta Geological Survey

> On Oct 3, 2024, at 5:21 AM, Roger Bivand via grass-dev 
>  wrote:
> 
> In connection with the transfer of the grass-dev list to Discourse, which I 
> am not prepared to follow as it will absorb much more time than scanning the 
> digested list headers, I will give up maintaining the R-GRASS interface 
> package at the end of 2024. 
> 
> The package lives in https://github.com/rsbivand/rgrass and the repo can be 
> moved to the preferred location of the new maintainer, and the maintainer 
> change notified to CRAN without difficulty. 
> 
> The arrival of Discourse precipitates a change that was started before a 
> workshop at FOSS4G in 2022 
> https://rsbivand.github.io/foss4g_2022/modernizing_220822.html, and several 
> GRASS users/developers then contributed to the workshop, and have helped 
> improve the package over many years.
> 
> Before a hand-over, this group of contributors can go through issues not yet 
> formally closed but mostly settled, to make sure that the new maintainer 
> ramps up smoothly. 
> 
> Best wishes,
> 
> Roger
> 
> --
> Roger Bivand
> Emeritus Professor
> Norwegian School of Economics
> Postboks 3490 Ytre Sandviken, 5045 Bergen, Norway
> roger.biv...@nhh.no
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Call for changes in the Python library

[Steven Pawley]

Hi Vaclav,

A couple of thoughts:

An equivalent of a RasterRow class for 3D rasters to make it easier to read 
stacks of GRASS raster datasets into multidimensional arrays.
Not really a Python addition as it mostly relates to the raster C library, but 
the ability to read windows/blocks/cubes of raster data, not just rows, IMHO 
would be a large improvement for raster data access into python. AFAIU, this is 
only currently possible by repeatedly moving the computational window, which I 
think was slower (when I tested this for reading chunks for r.learn.ml2) than 
the performance from windowed reading in other libraries (e.g. GDAL/Rasterio).

Steve

> On Sep 24, 2020, at 8:29 PM, Vaclav Petras  wrote:
> 
> Now would be a great time to do any large or small changes in the Python 
> library. Let's discuss these. For example, there was a talk about more clear 
> splitting of the ctypes dependent and independent code in grass.pygrass, 
> about moving the library to a different place, there is the new grass.grassdb 
> subpackage, PEP8/Black/Flake8, ... Is there anything *you* see as a priority 
> for 8.0? Is there anything you can do before 8.0? Let's coordinate here.
> 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Make terminal window optional?

[Steven Pawley]

Hi Vaclav,

Thanks for taking the time to provide this detailed information.

So, perhaps option 5 is starting to look too complicated, and you are right
that I would probably always start grass from the terminal anyway. So,
option 2 overall looks reasonable.

As a separate PR, I think having a "launch Jupyter" or "new notebook"
button or menu option would be great. Similar to the "simple python editor"
it could serve as a place to include tutorials etc. into python/jupyter
notebook scripting, a little like how Rstudio now has a "tutorial" tab that
launches interactive R markdown tutorials for various data processing tasks.

Steve

On Tue, Sep 8, 2020 at 9:39 PM Vaclav Petras  wrote:

> Hi Steven,
>
> You brought up several good points. I tried to cover all of them in my
> answer, but some may need more discussion. Hopefully, I was able to clarify
> some of my points.
>
> On Sun, Sep 6, 2020 at 10:56 AM Steven Pawley 
> wrote:
>
>>
>> My 2c is that the terminal should be made as optional because it can
>> definitely be confusing and intimidating/off-putting for new users.
>>
>
> Right, I think that's because people who don't want to use the command
> line, get it anyway. What I like about the options 2-5 is that they leave
> the choice to the user either automatically based on use of terminal
> (options 2 and 3) or explicit from command line parameters (options 4 and
> 5).
>
>
>> Apologies if I’ve confused some of the options but here are my thoughts
>> regarding the start-up options.
>>
>
> I have tried to clarify here and in the email to Paulo van Breugel. Let me
> know what parts are still unclear.
>
>
>> Unfortunately (in terms of complexity) option 5 would my preference. I
>> like what it inherits from option 4, in that “grass —text” would always
>> start with a terminal and just “grass” will *always* opens the full desktop
>> application/GUI.
>>
>
> I like that this makes GRASS GIS fall into clear categories of GUI app and
> shell one at a time, not both at the same time, but see my comments at the
> end of this email. When I'm typing "grass" in an interactive terminal, do I
> want to get only the GUI as a GRASS GIS user who is clearly also a command
> line user?
>
>
>> However, once in the GUI, ideally you would be able to launch a terminal
>> session from a menu option, e.g. a bit like Rstudio or VScode.
>>
>
> The PR which was adding or fixing "open terminal" for VS Code looked like
> a struggle, basically the same conclusion I made: you need user
> customization to make it robust enough. GRASS GIS has additional
> complexity. I assume that people expect to get a specific prompt and
> history like what you have in the terminal.
>
>
>> This is obviously how you want to launch an R or Jupyter session, and it
>> was be unfortunate to have to exit GRASS and restart a session with a
>> terminal just to do this.
>>
>
> If you are the type of user starting R and Jupyter sessions aren't you
> already in a terminal to run GRASS GIS from there? On the other hand,
> starting R, RStudio, or perhaps even Jupyter could be nicely supported by
> this "run custom application in the current session from GUI" feature
> (which is what makes options 3 and 5 a lot of work to implement).
>
>
>> Also, what happens when the GRASS GUI crashes and you are not running a
>> terminal? One of the aspects that I really like about GRASS is that even if
>> any particular component of the application, like the GUI crashes, the
>> session continues and modules/scripts keep on running, so everything is
>> usually recoverable.
>>
>
> Well, what happens now? What is running from the GUI should end with the
> GUI. If you are used to center your workflow around the terminal and you
> would start GRASS GIS from a terminal starting the (sub-)shell (--text,
> --shell, or options 2 and 3), nothing really changes. What you started from
> the terminal, still runs after a GUI crash. Your already written maps are
> fine. In the pure GUI scenario, you get to the mapset after breaking the
> lock.
>
> If you would like to have that recovery even in the pure GUI scenario,
> that would be possible. GUI is currently started from another process which
> could restart the GUI on failure as long as we can detect it. That's a
> related, yet separate feature.
>
>
>> If the implementation of option (5) is problematic, then I guess mixing
>> the startup options by  “grass —gui —shell” to open both the GUI and a
>> terminal (like currently) would be possible although it is a bit cumbersome 
>> for
>> all of 

Re: [GRASS-dev] Make terminal window optional?

[Steven Pawley]

Hi Vaclav,

Thanks for this. My 2c is that the terminal should be made as optional because 
it can definitely be confusing and intimidating/off-putting for new users. 
Apologies if I’ve confused some of the options but here are my thoughts 
regarding the start-up options.

Unfortunately (in terms of complexity) option 5 would my preference. I like 
what it inherits from option 4, in that “grass —text” would always start with a 
terminal and just “grass” will *always* opens the full desktop application/GUI.

However, once in the GUI, ideally you would be able to launch a terminal 
session from a menu option, e.g. a bit like Rstudio or VScode. This is 
obviously how you want to launch an R or Jupyter session, and it was be 
unfortunate to have to exit GRASS and restart a session with a terminal just to 
do this. Also, what happens when the GRASS GUI crashes and you are not running 
a terminal? One of the aspects that I really like about GRASS is that even if 
any particular component of the application, like the GUI crashes, the session 
continues and modules/scripts keep on running, so everything is usually 
recoverable.

If the implementation of option (5) is problematic, then I guess mixing the 
startup options by  “grass —gui —shell” to open both the GUI and a terminal 
(like currently) would be possible although it is a bit cumbersome for all of 
the people who routinely punch GRASS commands into the terminal etc.

Steve


> On Sep 5, 2020, at 9:05 PM, Vaclav Petras  wrote:
> 
> 4. Make the shell start only in the text mode (grass --text) or with a new 
> additional option (--shell), i.e., you get it, only when you actually ask for 
> it. In other words, with --text, GRASS GIS would behave more like R or 
> Octave, without that (with --gui), it would behave more like QGIS or any 
> other GUI application. (This includes the no terminal from desktop launchers 
> from option 2.)
> 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] g.extension, svn, and Mac

[Steven Pawley]

Hi Michael,

Yes, the 7.8.4.dev binary has the same issue in that it is also looking for the 
/Users/cmbarton/SDKs/MacOSX10.14.sdk SDK.

Steve

> On Aug 6, 2020, at 11:22 AM, Michael Barton  wrote:
> 
> Steven,
> 
> Can you try this with the GRASS 7.8.4dev binary I have on the Mac site? I'd 
> like to know if this affects all binaries or just 7.9
> 
> Thanks
> Michael
> 
> C. Michael Barton
> Director, Center for Social Dynamics & Complexity 
> Director, Network for Computational Modeling in Social & Ecological Sciences
> Professor of Anthropology, School of Human Evolution & Social Change
> Head, Graduate Faculty in Complex Adaptive Systems Science
> Arizona State University
> 
> voice:  480-965-6262 (SHESC), 480-965-8130/727-9746 (CSDC)
> fax: 480-965-7671 (SHESC),  480-727-0709 (CSDC)
> www: 'http://www.public.asu.edu/~cmbarton 
> <http://www.public.asu.edu/~cmbarton>, https://complexity.asu.edu/csdc' 
> <https://complexity.asu.edu/csdc'>
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>> On Aug 5, 2020, at 2:39 PM, Steven Pawley > <mailto:dr.stevenpaw...@gmail.com>> wrote:
>> 
>> Hi Michael,
>> 
>> Thanks for updating. I can still install python-based extensions without any 
>> error, however I still get the same error for compiled code:
>> 
>> g.extension extension=r.fill.gaps
>>
>> Fetching  from GRASS GIS Addons repository (be patient)...
>> Compiling...
>> clang-10: warning: no such sysroot directory:
>> '/Users/cmbarton/SDKs/MacOSX10.14.sdk' [-Wmissing-sysroot]
>> cell_funcs.c:21:10: fatal error: 'math.h' file not found
>> #include 
>>  ^~~~
>> 1 error generated.
>> make: *** [OBJ.x86_64-apple-darwin19.6.0/cell_funcs.o] Error
>> 1
>> ERROR: Compilation failed, sorry. Please check above error messages.
>> 
>> Steve
>> 
>>> On Aug 5, 2020, at 2:00 PM, Michael Barton >> <mailto:michael.bar...@asu.edu>> wrote:
>>> 
>>> Steve,
>>> 
>>> I just now posted a new updated GRASS 7.9dev that includes svn in the app. 
>>> It solved the g.extension problem for me. Can you test and see if it solves 
>>> it for you too? 
>>> 
>>> Michael
>>> 
>>> C. Michael Barton
>>> Director, Center for Social Dynamics & Complexity 
>>> Director, Network for Computational Modeling in Social & Ecological Sciences
>>> Professor of Anthropology, School of Human Evolution & Social Change
>>> Head, Graduate Faculty in Complex Adaptive Systems Science
>>> Arizona State University
>>> 
>>> voice:  480-965-6262 (SHESC), 480-965-8130/727-9746 (CSDC)
>>> fax: 480-965-7671 (SHESC),  480-727-0709 (CSDC)
>>> www: 'http://www.public.asu.edu/~cmbarton 
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.public.asu.edu_-7Ecmbarton&d=DwMFaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=lk-7X7CEOMDN8GaGVhiDsuO6gEp1wbG6nfT1XEEEtR0&m=2Gr42Gr6-mCS6vSIKkL1gNxGKhgP9akeCkWc5-eOGZ8&s=bm8GNsnlGrRMgk5Aw-PILxKCHgti6ue1ipDNLN5bS8o&e=>,
>>>  https://complexity.asu.edu/csdc' 
>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__complexity.asu.edu_csdc-27&d=DwMFaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=lk-7X7CEOMDN8GaGVhiDsuO6gEp1wbG6nfT1XEEEtR0&m=2Gr42Gr6-mCS6vSIKkL1gNxGKhgP9akeCkWc5-eOGZ8&s=h-op3BcIQNfbjiQKiuUA4VsQRB09LrvWPs0cx0vmmAc&e=>
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>>> On Aug 4, 2020, at 11:59 AM, Michael Barton >>> <mailto:michael.bar...@asu.edu>> wrote:
>>>> 
>>>> I'm pretty sure this happened when I installed developer tools 11.5 
>>>> 
>>>> I want to make sure that anyone using the binaries can do so without 
>>>> having to separately install any dependencies. So I'll see what happens if 
>>>> I just include svn in the app. I will need to test on a system that 
>>>> suffers from this lack. 
>>>> 
>>>> Michael
>>>> 
>>>> C. Michael Barton
>>>> Director, Center for Social Dynamics & Complexity 
>>>> Director, Network for Computa

Re: [GRASS-dev] g.extension, svn, and Mac

[Steven Pawley]

I’m on 10.15 (on a relatively new Mac) and officially Xcode only ships with the 
latest SDK. I think unofficially it might be possible to copy older SDKs into 
the Xcode app bundle but presumably not ideal.


> On Aug 5, 2020, at 3:46 PM, Michael Barton  wrote:
> 
> This error implies that you need the 10.14.sdk folder. Why would that be so? 
> That would be a problem. 
> 
> Michael
> 
> C. Michael Barton
> Director, Center for Social Dynamics & Complexity 
> Director, Network for Computational Modeling in Social & Ecological Sciences
> Professor of Anthropology, School of Human Evolution & Social Change
> Head, Graduate Faculty in Complex Adaptive Systems Science
> Arizona State University
> 
> voice:  480-965-6262 (SHESC), 480-965-8130/727-9746 (CSDC)
> fax: 480-965-7671 (SHESC),  480-727-0709 (CSDC)
> www: 'http://www.public.asu.edu/~cmbarton 
> <http://www.public.asu.edu/~cmbarton>, https://complexity.asu.edu/csdc' 
> <https://complexity.asu.edu/csdc'>
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>> On Aug 5, 2020, at 2:39 PM, Steven Pawley > <mailto:dr.stevenpaw...@gmail.com>> wrote:
>> 
>> Hi Michael,
>> 
>> Thanks for updating. I can still install python-based extensions without any 
>> error, however I still get the same error for compiled code:
>> 
>> g.extension extension=r.fill.gaps
>>
>> Fetching  from GRASS GIS Addons repository (be patient)...
>> Compiling...
>> clang-10: warning: no such sysroot directory:
>> '/Users/cmbarton/SDKs/MacOSX10.14.sdk' [-Wmissing-sysroot]
>> cell_funcs.c:21:10: fatal error: 'math.h' file not found
>> #include 
>>  ^~~~
>> 1 error generated.
>> make: *** [OBJ.x86_64-apple-darwin19.6.0/cell_funcs.o] Error
>> 1
>> ERROR: Compilation failed, sorry. Please check above error messages.
>> 
>> Steve
>> 
>>> On Aug 5, 2020, at 2:00 PM, Michael Barton >> <mailto:michael.bar...@asu.edu>> wrote:
>>> 
>>> Steve,
>>> 
>>> I just now posted a new updated GRASS 7.9dev that includes svn in the app. 
>>> It solved the g.extension problem for me. Can you test and see if it solves 
>>> it for you too? 
>>> 
>>> Michael
>>> 
>>> C. Michael Barton
>>> Director, Center for Social Dynamics & Complexity 
>>> Director, Network for Computational Modeling in Social & Ecological Sciences
>>> Professor of Anthropology, School of Human Evolution & Social Change
>>> Head, Graduate Faculty in Complex Adaptive Systems Science
>>> Arizona State University
>>> 
>>> voice:  480-965-6262 (SHESC), 480-965-8130/727-9746 (CSDC)
>>> fax: 480-965-7671 (SHESC),  480-727-0709 (CSDC)
>>> www: 'http://www.public.asu.edu/~cmbarton 
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.public.asu.edu_-7Ecmbarton&d=DwMFaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=lk-7X7CEOMDN8GaGVhiDsuO6gEp1wbG6nfT1XEEEtR0&m=2Gr42Gr6-mCS6vSIKkL1gNxGKhgP9akeCkWc5-eOGZ8&s=bm8GNsnlGrRMgk5Aw-PILxKCHgti6ue1ipDNLN5bS8o&e=>,
>>>  https://complexity.asu.edu/csdc' 
>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__complexity.asu.edu_csdc-27&d=DwMFaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=lk-7X7CEOMDN8GaGVhiDsuO6gEp1wbG6nfT1XEEEtR0&m=2Gr42Gr6-mCS6vSIKkL1gNxGKhgP9akeCkWc5-eOGZ8&s=h-op3BcIQNfbjiQKiuUA4VsQRB09LrvWPs0cx0vmmAc&e=>
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>>> On Aug 4, 2020, at 11:59 AM, Michael Barton >>> <mailto:michael.bar...@asu.edu>> wrote:
>>>> 
>>>> I'm pretty sure this happened when I installed developer tools 11.5 
>>>> 
>>>> I want to make sure that anyone using the binaries can do so without 
>>>> having to separately install any dependencies. So I'll see what happens if 
>>>> I just include svn in the app. I will need to test on a system that 
>>>> suffers from this lack. 
>>>> 
>>>> Michael
>>>> 
>>>> C. Michael Barton
>>>> Director, Center for Social Dynamics & Complexity 
>>>> Director, Network for Computational Modelin

Re: [GRASS-dev] g.extension, svn, and Mac

[Steven Pawley]

Hi Michael,

Thanks for updating. I can still install python-based extensions without any 
error, however I still get the same error for compiled code:

g.extension extension=r.fill.gaps   
Fetching  from GRASS GIS Addons repository (be patient)...
Compiling...
clang-10: warning: no such sysroot directory:
'/Users/cmbarton/SDKs/MacOSX10.14.sdk' [-Wmissing-sysroot]
cell_funcs.c:21:10: fatal error: 'math.h' file not found
#include 
 ^~~~
1 error generated.
make: *** [OBJ.x86_64-apple-darwin19.6.0/cell_funcs.o] Error
1
ERROR: Compilation failed, sorry. Please check above error messages.

Steve

> On Aug 5, 2020, at 2:00 PM, Michael Barton  wrote:
> 
> Steve,
> 
> I just now posted a new updated GRASS 7.9dev that includes svn in the app. It 
> solved the g.extension problem for me. Can you test and see if it solves it 
> for you too? 
> 
> Michael
> 
> C. Michael Barton
> Director, Center for Social Dynamics & Complexity 
> Director, Network for Computational Modeling in Social & Ecological Sciences
> Professor of Anthropology, School of Human Evolution & Social Change
> Head, Graduate Faculty in Complex Adaptive Systems Science
> Arizona State University
> 
> voice:  480-965-6262 (SHESC), 480-965-8130/727-9746 (CSDC)
> fax: 480-965-7671 (SHESC),  480-727-0709 (CSDC)
> www: 'http://www.public.asu.edu/~cmbarton 
> <http://www.public.asu.edu/~cmbarton>, https://complexity.asu.edu/csdc' 
> <https://complexity.asu.edu/csdc'>
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>> On Aug 4, 2020, at 11:59 AM, Michael Barton > <mailto:michael.bar...@asu.edu>> wrote:
>> 
>> I'm pretty sure this happened when I installed developer tools 11.5 
>> 
>> I want to make sure that anyone using the binaries can do so without having 
>> to separately install any dependencies. So I'll see what happens if I just 
>> include svn in the app. I will need to test on a system that suffers from 
>> this lack. 
>> 
>> Michael
>> 
>> C. Michael Barton
>> Director, Center for Social Dynamics & Complexity 
>> Director, Network for Computational Modeling in Social & Ecological Sciences
>> Professor of Anthropology, School of Human Evolution & Social Change
>> Head, Graduate Faculty in Complex Adaptive Systems Science
>> Arizona State University
>> 
>> voice:  480-965-6262 (SHESC), 480-965-8130/727-9746 (CSDC)
>> fax: 480-965-7671 (SHESC),  480-727-0709 (CSDC)
>> www: 'http://www.public.asu.edu/~cmbarton 
>> <http://www.public.asu.edu/~cmbarton>, https://complexity.asu.edu/csdc' 
>> <https://complexity.asu.edu/csdc'>
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>>> On Aug 4, 2020, at 11:56 AM, Steven Pawley >> <mailto:dr.stevenpaw...@gmail.com>> wrote:
>>> 
>>> Hi Michael,
>>> 
>>> Right, I do have svn installed via Homebrew (I get  'Type 'svn help' for 
>>> usage.’), so that probably explains why g.extension works for me overall, 
>>> apart from the issue with finding the MacOS SDK.
>>> 
>>> Steve
>>> 
>>>> On Aug 4, 2020, at 11:27 AM, Michael Barton >>> <mailto:michael.bar...@asu.edu>> wrote:
>>>> 
>>>> Steven,
>>>> 
>>>> What happens if you open a terminal window and type 'svn' followed by a 
>>>> return? 
>>>> 
>>>> Michael
>>>> 
>>>> C. Michael Barton
>>>> Director, Center for Social Dynamics & Complexity 
>>>> Director, Network for Computational Modeling in Social & Ecological 
>>>> Sciences
>>>> Professor of Anthropology, School of Human Evolution & Social Change
>>>> Head, Graduate Faculty in Complex Adaptive Systems Science
>>>> Arizona State University
>>>> 
>>>> voice:  480-965-6262 (SHESC), 480-965-8130/727-9746 (CSDC)
>>>> fax: 480-965-7671 (SHESC),  480-727-0709 (CSDC)
>>>> www: 'http://www.public.asu.edu/~cmbarton 
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.public.asu.edu_-7Ecmbarton&d=DwMFaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=lk-7X7CEOMDN8GaGVhiDsuO6gEp1wbG6nfT1XEEEtR0&m=3OtUJxnVDVfConnURZzc-TyoKzTbZWrxoF7zL8E2qZ0&s=6KybHpYJMo7V86xk_1V

Re: [GRASS-dev] g.extension, svn, and Mac

[Steven Pawley]

Perhaps a different issue,  but I can install extensions, but only pure 
python-based extensions. Anything that requires compilation of C code fails.

For example, when using the latest GRASS GIS binary (7.9, 25th July 2020 
build), this works with r.texture.tiled:

Fetching  from GRASS GIS Addons repository (be patient)...
Compiling...
Installing...
Updating extensions metadata file...
Updating extension modules metadata file...
Installation of  successfully finished

But for example, r.fill.gaps does not:

g.extension extension=r.fill.gaps   
Fetching  from GRASS GIS Addons repository (be patient)...
Compiling...
clang-10: warning: no such sysroot directory:
'/Users/cmbarton/SDKs/MacOSX10.14.sdk' [-Wmissing-sysroot]
cell_funcs.c:21:10: fatal error: 'math.h' file not found
#include 
 ^~~~
1 error generated.
make: *** [OBJ.x86_64-apple-darwin19.5.0/cell_funcs.o] Error
1
ERROR: Compilation failed, sorry. Please check above error messages.

The 7.9-dev build throws an error because it appears to be still looking for 
components based on your system, specifically.,The 7.8.4 binary (16th July 2020 
build) also throws the same error. I’m running the MacOS 10.15.6.

Steve 


> On Aug 2, 2020, at 4:44 PM, Michael Barton  wrote:
> 
> I just learned yesterday, when I tried to install something from g.extension 
> that:
> 
> 1. g.extension uses svn to download packages, and
> 2. Apple stopped including svn with the current OS (10.15.x) and its 
> developer tools
> 
> I don't know if anyone else has run into this yet, but it could be an 
> increasingly sticky issue going forward. Unless there are plans to change how 
> g.extension gets packages, I suppose I need to include it in the Mac app 
> bundle. 
> 
> Has anyone else hit this yet?
> 
> Michael 
> 
> C. Michael Barton
> Director, Center for Social Dynamics & Complexity 
> Director, Network for Computational Modeling in Social & Ecological Sciences
> Professor of Anthropology, School of Human Evolution & Social Change
> Head, Graduate Faculty in Complex Adaptive Systems Science
> Arizona State University
> 
> voice:  480-965-6262 (SHESC), 480-965-8130/727-9746 (CSDC)
> fax: 480-965-7671 (SHESC),  480-727-0709 (CSDC)
> www: 'http://www.public.asu.edu/~cmbarton 
> , https://complexity.asu.edu/csdc' 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] where to make compiling instructions available

[Steven Pawley]

Whether it goes in the macosx directory, or somewhere else, this will be really 
useful for us Mac users who would like to have GRASS within their own conda 
environments. I struggle to compile GRASS on macOS and thanks for putting this 
together.


> On May 27, 2020, at 2:59 PM, Michael Barton  wrote:
> 
> I've put together a step by step guide to compiling Mac binaries, using 
> Anaconda. Because it uses Anaconda, it probably does not belong in the source 
> code (though it might help someone create instructions for the source code). 
> However, it would probably be helpful to at least some people. Any 
> suggestions as to where I should put this?
> 
> Michael
> 
> C. Michael Barton
> Director, Center for Social Dynamics & Complexity 
> Director, Network for Computational Modeling in Social & Ecological Sciences
> Professor of Anthropology, School of Human Evolution & Social Change
> Head, Graduate Faculty in Complex Adaptive Systems Science
> Arizona State University
> 
> voice:  480-965-6262 (SHESC), 480-965-8130/727-9746 (CSDC)
> fax: 480-965-7671 (SHESC),  480-727-0709 (CSDC)
> www: 'http://www.public.asu.edu/~cmbarton 
> , https://complexity.asu.edu/csdc' 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] Fwd: Memory consumption using pygrass.utils.get_raster_for_points

[Steven Pawley]

As a follow-up to this, I tried tracking the memory usage using the python
Pympler package.

I ran the following code block:

from grass.pygrass.vector import VectorTopo
from grass.pygrass.raster import RasterRow
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.gis.region import Region
from grass.pygrass.utils import get_raster_for_points

# set region
reg = Region()
reg.from_rast("landclass96")
reg.write()
reg.set_raster_region()

# generate a large point dataset
r.random(input="landclass96", npoints=20, vector="landclass96_roi",
 overwrite=True)

# memory tracking
from pympler.tracker import SummaryTracker
tracker = SummaryTracker()

points = VectorTopo("landclass96_roi")
points.open("r")

# repeat spatial query of raster
for i in range(10):
print(i)
with RasterRow("lsat5_1987_10") as src:
arr = get_raster_for_points(points, src)
points.close()

tracker.print_diff()

The memory tracker results are:

   types |   # objects
|   total size
 | ===
| 
  
Date: Sat, Feb 9, 2019 at 9:48 AM
Subject: Memory consumption using pygrass.utils.get_raster_for_points
To: GRASS developers list 


Hello devs,

When running pygrass.utils.get_raster_for_points repeatedly, it appears
that the python memory allocation continuously increases until all ram is
consumed, even if the extracted values are not being collected (or are
overwriting the same variable).

I noticed this when extracting raster data at point locations, when using a
large point dataset, even though I had pre-allocated a numpy array to
receive the results.

Below is an example on the nc_spm_08_grass7 example data (in the landsat
mapset), repeating the operation say 50 times on the same point vector
dataset. I wouldn't have expected the memory consumption to continuously
increase for this operation, because I'm overwriting the 'arr' variable
each time. However, if you repeat this enough times, you will run out of
system memory and the allocated memory does not appear to be released, i.e.
even if you manually force garbage collection.

Any suggestions?


from grass.pygrass.vector import VectorTopo
from grass.pygrass.raster import RasterRow
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.gis.region import Region
from grass.pygrass.utils import get_raster_for_points

reg = Region()
reg.from_rast("landclass96")
reg.write()
reg.set_raster_region()

r.random(input="landclass96", npoints=20, vector="landclass96_roi",
 overwrite=True)

points = VectorTopo("landclass96_roi")
points.open("r")

# repeat spatial query of raster
for i in range(50):
with RasterRow("lsat5_1987_10") as src:
arr = get_raster_for_points(points, src)
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] Memory consumption using pygrass.utils.get_raster_for_points

[Steven Pawley]

Hello devs,

When running pygrass.utils.get_raster_for_points repeatedly, it appears
that the python memory allocation continuously increases until all ram is
consumed, even if the extracted values are not being collected (or are
overwriting the same variable).

I noticed this when extracting raster data at point locations, when using a
large point dataset, even though I had pre-allocated a numpy array to
receive the results.

Below is an example on the nc_spm_08_grass7 example data (in the landsat
mapset), repeating the operation say 50 times on the same point vector
dataset. I wouldn't have expected the memory consumption to continuously
increase for this operation, because I'm overwriting the 'arr' variable
each time. However, if you repeat this enough times, you will run out of
system memory and the allocated memory does not appear to be released, i.e.
even if you manually force garbage collection.

Any suggestions?


from grass.pygrass.vector import VectorTopo
from grass.pygrass.raster import RasterRow
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.gis.region import Region
from grass.pygrass.utils import get_raster_for_points

reg = Region()
reg.from_rast("landclass96")
reg.write()
reg.set_raster_region()

r.random(input="landclass96", npoints=20, vector="landclass96_roi",
 overwrite=True)

points = VectorTopo("landclass96_roi")
points.open("r")

# repeat spatial query of raster
for i in range(50):
with RasterRow("lsat5_1987_10") as src:
arr = get_raster_for_points(points, src)
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] Memory consumption using pygrass.utils.get_raster_for_points

[Steven Pawley]

Hello devs,

When running pygrass.utils.get_raster_for_points repeatedly, it appears 
that the python memory allocation continuously increases until all ram 
is consumed, even if the extracted values are not being collected (or 
are overwriting the same variable).


I noticed this when extracting raster data at point locations, when 
using a large point dataset, even though I had pre-allocated a numpy 
array to receive the results.


Below is an example on the nc_spm_08_grass7 example data (in the landsat 
mapset), repeating the operation say 50 times on the same point vector 
dataset. I wouldn't have expected the memory consumption to continuously 
increase for this operation, because I'm overwriting the 'arr' variable 
each time. However, if you repeat this enough times, you will run out of 
system memory and the allocated memory does not appear to be released, 
i.e. even if you manually force garbage collection.


Any suggestions?


from grass.pygrass.vector import VectorTopo
from grass.pygrass.raster import RasterRow
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.gis.region import Region
from grass.pygrass.utils import get_raster_for_points

reg = Region()
reg.from_rast("landclass96")
reg.write()
reg.set_raster_region()

r.random(input="landclass96", npoints=20, vector="landclass96_roi",
 overwrite=True)

points = VectorTopo("landclass96_roi")
points.open("r")

# repeat spatial query of raster
for i in range(50):
    with RasterRow("lsat5_1987_10") as src:
    arr = get_raster_for_points(points, src)

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3554: r.slope.aspect: add flag/functionality to not shrink

[Steven Pawley]

Thanks Markus for the update to r.slope.aspect, and for the additional info.

I just updated the mapcalc part of r.valley.bottom (72655)to deal with null
values as part of the mapcalc expression. I used the strategy of firstly
attempting to replace null pixels in the neighbourhood with their opposite
(i.e. mirroring), and if that also results in a null value (i.e. in
corners) then the value of the centre pixel is used.

Steve

On Sun, Apr 29, 2018 at 8:30 AM, Markus Metz 
wrote:

>
>
> On Sun, Apr 29, 2018 at 7:36 AM, Steven Pawley 
> wrote:
> >
> > Handling of nodata values at raster borders for modules which involve
> focal functions would be handy to have for several GRASS GIS modules, not
> just for r.slope.aspect. IMHO r.mapcalc, r.texture, and r.resamp.interp (if
> using bilinear or cubic resampling) would also benefit from this option
> (perhaps other tools as well).
>
> r.texture already has the -n flag: Allow NULL cells in a moving window
> r.slope.aspect has a new -e flag: Compute output at edges and near NULL
> values (same like gdaldem -compute_edges)
> r.mapcalc has the isnull() function and the special operators "&&&" and
> "|||" handling NULL values
> r.resamp.interp needs to be done for bilinear, bicubic, lanczos,
> implementation could be relatively easy in the raster library functions
>
> Markus M
>
> > Although some users may be concerned that filling nodata values at
> raster borders can create edge effects, in most cases the common approaches
> (such as mirroring, reflecting or wrapping) leave little in the way of an
> obvious edge effect, and also satisfy what I think are most user's
> expectations that the output raster will have the same dimensions as the
> input.
> >
> > Although a similar effect can be achieved by growing the input raster
> first, this can be inconvenient when used as part of a complex set of focal
> function operations. An example is that I have just completed working on an
> updated r.valley.bottom module, which required multiple r.grow operations
> as the algorithm repeatedly generalizes the input DEM and calculate
> flatness and lowness using r.slope.aspect and r.mapcalc focal functions. It
> also appears that most other tools, including SAGA-GIS, scipy (image
> filters), and gdaldem  provide a method to handle raster borders.
> >
> >
> > On Sat, Apr 28, 2018 at 2:55 PM GRASS GIS  wrote:
> >>
> >> #3554: r.slope.aspect: add flag/functionality to not shrink
> >> -+-
> >>  Reporter:  hellik   |  Owner:  grass-dev@…
> >>  Type:  enhancement  | Status:  new
> >>  Priority:  normal   |  Milestone:  8.0.0
> >> Component:  Raster   |Version:  svn-trunk
> >>  Keywords:   |CPU:  All
> >>  Platform:  All  |
> >> -+-
> >>  at the moment, the result maps of r.slope.aspect, e.g. slope etc, are
> >>  shrinked by 1 pixel.
> >>
> >>  adding a flag/functionality to not shrink the result maps e.g. by
> adding
> >>  the same value as the -1 off pixel, some interpolations or similar as
> an
> >>  enhancement.
> >>
> >> --
> >> Ticket URL: <https://trac.osgeo.org/grass/ticket/3554>
> >> GRASS GIS <https://grass.osgeo.org>
> >>
> >> ___
> >> grass-dev mailing list
> >> grass-dev@lists.osgeo.org
> >> https://lists.osgeo.org/mailman/listinfo/grass-dev
> >
> >
> > ___
> > grass-dev mailing list
> > grass-dev@lists.osgeo.org
> > https://lists.osgeo.org/mailman/listinfo/grass-dev
>
>
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3554: r.slope.aspect: add flag/functionality to not shrink

[Steven Pawley]

Handling of nodata values at raster borders for modules which involve focal
functions would be handy to have for several GRASS GIS modules, not just
for r.slope.aspect. IMHO r.mapcalc, r.texture, and r.resamp.interp (if
using bilinear or cubic resampling) would also benefit from this option
(perhaps other tools as well). Although some users may be concerned that
filling nodata values at raster borders can create edge effects, in most
cases the common approaches (such as mirroring, reflecting or wrapping)
leave little in the way of an obvious edge effect, and also satisfy what I
think are most user's expectations that the output raster will have the
same dimensions as the input.

Although a similar effect can be achieved by growing the input raster
first, this can be inconvenient when used as part of a complex set of focal
function operations. An example is that I have just completed working on an
updated r.valley.bottom module, which required multiple r.grow operations
as the algorithm repeatedly generalizes the input DEM and calculate
flatness and lowness using r.slope.aspect and r.mapcalc focal functions. It
also appears that most other tools, including SAGA-GIS, scipy (image
filters), and gdaldem  provide a method to handle raster borders.


On Sat, Apr 28, 2018 at 2:55 PM GRASS GIS  wrote:

> #3554: r.slope.aspect: add flag/functionality to not shrink
> -+-
>  Reporter:  hellik   |  Owner:  grass-dev@…
>  Type:  enhancement  | Status:  new
>  Priority:  normal   |  Milestone:  8.0.0
> Component:  Raster   |Version:  svn-trunk
>  Keywords:   |CPU:  All
>  Platform:  All  |
> -+-
>  at the moment, the result maps of r.slope.aspect, e.g. slope etc, are
>  shrinked by 1 pixel.
>
>  adding a flag/functionality to not shrink the result maps e.g. by adding
>  the same value as the -1 off pixel, some interpolations or similar as an
>  enhancement.
>
> --
> Ticket URL: 
> GRASS GIS 
>
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
>
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3401: r.learn.ml some notes and suggestions

[Steven Pawley]

Hello Mait,

Thank you, these notes and elaborations look useful. I'm away with field work 
but I'll take a closer look when I can.

Steve

Sent from my iPhone

> On Aug 18, 2017, at 6:59 AM, GRASS GIS  wrote:
> 
> #3401: r.learn.ml some notes and suggestions
> --+-
>  Reporter:  maitl|  Owner:  grass-dev@…
>  Type:  enhancement  | Status:  new
>  Priority:  minor|  Milestone:  7.2.2
> Component:  Addons   |Version:  7.2.1
> Resolution:   |   Keywords:  r.learn.ml
>   CPU:  Unspecified  |   Platform:  All
> --+-
> Changes (by maitl):
> 
> * Attachment "r.learn.ml.notes.18.08.2017" added.
> 
> The list of notes.
> 
> --
> Ticket URL: 
> GRASS GIS 
> 
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Inclusion of additional files with add-on

[Steven Pawley]

I'll open a ticket for r.tileset.

Sent from my iPhone

> On Aug 16, 2017, at 2:36 AM, Helmut Kudrnovsky  wrote:
> 
> Moritz Lennert wrote
>>> On 16/08/17 00:48, Steven Pawley wrote:
>>> This seems to be an issue on windows that is related to r.tileset not 
>>> r.terrain.texture. Infact on windows I can't get r.tileset to function 
>>> at all on a windows system, but it works on linux:
>>> 
>>> r.tileset sourceproj="+proj=tmerc +lat_0=0 +lon_0=-115 +k=0.9992 
>>> +x_0=50 +y_0=0 +no_defs +a=6378137 +rf=298.257222101 
>>> +towgs84=0.000,0.000,0.000 +to_meter=1"
>>> 
>>> Traceback (most recent call last):
>>>   File
>>> "C:\OSGEO4~1\apps\grass\grass-7.3.svn/scripts/m.proj.py
>>> <http://m.proj.py>";,
>>> line 308, in 
>> 
>>> main()
>>>   File
>>> "C:\OSGEO4~1\apps\grass\grass-7.3.svn/scripts/m.proj.py
>>> <http://m.proj.py>";,
>>> line 243, in main
>>> inf = file(infile)
>>> IOError: [Errno 13] Permission denied: 'c:\\users\\cb2dd\\ap
>>> pdata\\local\\temp\\grass7-cb2dd-10432\\tmpovmbzm'
>>> ERROR: cs2cs failed
>> 
>> IIUC, this means that the call to m.proj in r.tileset has failed:
>> 
>> Could someone try if m.proj works on Windows ?
> 
> C:\>v.out.ascii bridges | m.proj -o input=-
> 
> and
> 
> m.proj -o input=D:\wd\testepsg\coor.txt   
>   
> 81d37'10.19915"W|36d35'10.7323"N|1.000
> 81d37'47.06273"W|36d35'1.19866"N|2.000
> 81d33'45.90899"W|36d34'47.26688"N|3.000
> 
> works here with
> 
> System Info   
>   
> GRASS version: 7.3.svn
>   
> GRASS SVN revision: r71384
>   
> Build date: 2017-08-11
>   
> Build platform: x86_64-w64-mingw32
>   
> GDAL: 2.2.1   
>   
> PROJ.4: 4.9.3 
>   
> GEOS: 3.5.0   
>   
> SQLite: 3.17.0
>   
> Python: 2.7.5 
>   
> wxPython: 2.8.12.1
>   
> Platform: Windows-8-6.2.9200 (OSGeo4W) 
> 
>> IOError: [Errno 13] Permission denied: 'c:\\users\\cb2dd\\ap
>> pdata\\local\\temp\\grass7-cb2dd-10432\\tmpovmbzm'
> 
> maybe there is some issue with the temporary file as input for cs2cs, but I
> can't find where ...
> 
> 
> 
> -
> best regards
> Helmut
> --
> View this message in context: 
> http://osgeo-org.1560.x6.nabble.com/Inclusion-of-additional-files-with-add-on-tp5331342p5331585.html
> Sent from the Grass - Dev mailing list archive at Nabble.com.
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Inclusion of additional files with add-on

[Steven Pawley]

This seems to be an issue on windows that is related to r.tileset not
r.terrain.texture. Infact on windows I can't get r.tileset to function at
all on a windows system, but it works on linux:

r.tileset sourceproj="+proj=tmerc +lat_0=0 +lon_0=-115 +k=0.9992
+x_0=50 +y_0=0 +no_defs +a=6378137 +rf=298.257222101
+towgs84=0.000,0.000,0.000 +to_meter=1"

Traceback (most recent call last):
  File
"C:\OSGEO4~1\apps\grass\grass-7.3.svn/scripts/m.proj.py",
line 308, in 
main()
  File
"C:\OSGEO4~1\apps\grass\grass-7.3.svn/scripts/m.proj.py",
line 243, in main
inf = file(infile)
IOError: [Errno 13] Permission denied: 'c:\\users\\cb2dd\\ap
pdata\\local\\temp\\grass7-cb2dd-10432\\tmpovmbzm'
ERROR: cs2cs failed




On Tue, Aug 15, 2017 at 1:31 AM, Helmut Kudrnovsky  wrote:

> >I have gone with your's and Moritz's suggestion and have completed
> r.terrain.texture with the Iwahashi >and Pike nested-means terrain
> classification method in the add-ons.
>
> addon tested here with
>
> System Info
> GRASS version: 7.3.svn
> GRASS SVN revision: r71384
> Build date: 2017-08-11
> Build platform: x86_64-w64-mingw32
> GDAL: 2.2.1
> PROJ.4: 4.9.3
> GEOS: 3.5.0
> SQLite: 3.17.0
> Python: 2.7.5
> wxPython: 2.8.12.1
> Platform: Windows-8-6.2.9200 (OSGeo4W)
>
> and
>
> r.terrain.texture elevation=elevation@PERMANENT slope=myslope@user1
> classes=12 texture=terr_tex convexity=terr_convex concavity=terr_concav
> features=terr_class
>
> I get
>
> --
> r.terrain.texture elevation=elevation@PERMANENT slope=myslope@user1
> classes=12 texture=terr_tex convexity=terr_convex concavity=terr_concav
> features=terr_class
> Calculating terrain surface texture...
> 1. Smoothing input DEM with a 3x3 median filter...
> 2. Extracting pits and peaks with difference > thres...
> 3. Using resampling filter to create terrain texture...
> 4. Converting to percentage...
> Calculating terrain convexity and concavity...
> 1. Calculating terrain curvature using laplacian filter...
> 2. Extracting convexities and concavities...
> 3. Using resampling filter to create surface convexity/concavity...
> 4. Converting to percentages...
> Performing terrain surface classification...
> Traceback (most recent call last):
>   File
> "C:\OSGEO4~1\apps\grass\grass-7.3.svn/scripts/m.proj.py",
> line 308, in 
> main()
>   File
> "C:\OSGEO4~1\apps\grass\grass-7.3.svn/scripts/m.proj.py",
> line 243, in main
> inf = file(infile)
> IOError: [Errno 13] Permission denied: 'c:\\users\\xxx\\ap
> pdata\\local\\temp\\grass7-hkmyr-6804\\tmpl1pwdf'
> ERROR: cs2cs failed
> Deleting intermediate files...
> WARNING: No data base element files found
> Traceback (most recent call last):
>   File "C:\Users\xxx\AppData\Roaming\GRASS7\addons/scripts
> /r.terrain.texture.py", line 575, in 
> sys.exit(main())
>   File "C:\Users\xxx\AppData\Roaming\GRASS7\addons/scripts
> /r.terrain.texture.py", line 471, in main
> flags='g', quiet=True,
> stdout_=PIPE).outputs.stdout.split(os.linesep)[:-1]
>   File "C:\OSGEO4~1\apps\grass\grass-7.3.svn\etc\python\gras
> s\pygrass\modules\interface\module.py", line 648, in
> __call__
> return self.run()
>   File "C:\OSGEO4~1\apps\grass\grass-7.3.svn\etc\python\gras
> s\pygrass\modules\interface\module.py", line 769, in run
> self.wait()
>   File "C:\OSGEO4~1\apps\grass\grass-7.3.svn\etc\python\gras
> s\pygrass\modules\interface\module.py", line 790, in wait
> module=self.name, errors=stderr)
> grass.exceptions.CalledModuleError: Module run r.tileset
> r.tileset sourceproj=+proj=lcc +lat_1=36.16
> +lat_2=34.34 +lat_0=33.75 +lon_0=-79
> +x_0=609601.22 +y_0=0 +no_defs +a=6378137 +rf=298.257222101
> +towgs84=0.000,0.000,0.000 +to_meter=1 sourcescale=1
> maxcols=1500 maxrows=1352 overlap=0 separator=pipe -g --q
> ended with error
> Process ended with non-zero return code 1. See errors in the
> (error) output.
> --
>
> C:\>where cs2cs
> C:\OSGeo4W64\bin\cs2cs.exe
> C:\>cs2cs
> Rel. 4.9.3, 15 August 2016
> usage: cs2cs [ -eEfIlrstvwW [args] ] [ +opts[=arg] ]
>[+to [+opts[=arg] [ files ]
>
> cs2cs seems to be available.
>
>
>
> -
> best regards
> Helmut
> --
> View this message in context: http://osgeo-org.1560.x6.
> nabble.com/Inclusion-of-additional-files-with-add-on-
> tp5331342p5331497.html
> Sent from the Grass - Dev mailing list archive at Nabble.com.
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Inclusion of additional files with add-on

[Steven Pawley]

Thanks Helmut,

I have gone with your's and Moritz's suggestion and have completed 
r.terrain.texture with the Iwahashi and Pike nested-means terrain 
classification method in the add-ons.

Best regards,

Steve

Sent from my iPhone

On Aug 12, 2017, at 2:22 PM, Helmut Kudrnovsky  wrote:

>> (I also have three files representing color rules, and three category >files
> that are applied to the 8, 12 or 16 terrain classification >results)
> 
> https://trac.osgeo.org/grass/browser/grass-addons/grass7/raster/r.euro.ecosystem/r.euro.ecosystem.py#L72
> 
> an example where I've embeded color and category rules in the script. 
> 
> 
> 
> -
> best regards
> Helmut
> --
> View this message in context: 
> http://osgeo-org.1560.x6.nabble.com/Inclusion-of-additional-files-with-add-on-tp5331342p5331362.html
> Sent from the Grass - Dev mailing list archive at Nabble.com.
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Inclusion of additional files with add-on

[Steven Pawley]

Thanks Markus for the pointer, I will take a look.

Steve

On Sat, Aug 12, 2017 at 1:35 PM, Markus Neteler  wrote:

> On Aug 12, 2017 9:34 PM, "Markus Neteler"  wrote:
> >
> > On Aug 12, 2017 4:45 PM, "Steven Pawley" 
> wrote:
> > >
> > > Hello,
> > >
> > > I'm getting around to completing my TODO list for the add-on
> r.terrain.texture, which includes generating the additional terrain metrics
> (terrain surface concavity and convexity), and calculating the nested 8,12
> or 16 class terrain classification as per Iwahashi and Pike (2007).
> Everything is working but the script uses r.mfilter, which I supply a
> matrix filter to, stored as a text file.
> > >
> > > Is there a makefile command that can be used to bundle a text file
> with the add on, which can then be found using set_path?
> >
> > You may want to check how the color files are installed. Should work
> pretty much the same.
>
> I meant the Makefile in lib/gis/
>
> Markus
>
> > > Steve
> >
> > >
> > > ___
> > > grass-dev mailing list
> > > grass-dev@lists.osgeo.org
> > > https://lists.osgeo.org/mailman/listinfo/grass-dev
>
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Inclusion of additional files with add-on

[Steven Pawley]

Thanks Moritz, that's one option although I dropping large blobs of text
strings into the script might not be very 'clean' (I also have three files
representing color rules, and three category files that are applied to the
8, 12 or 16 terrain classification results). If there is no alternative
I'll go with that (or create some additional python files containing the
strings that are imported), but I was wondering if there is a way to
package the typical GRASS rule files in the add-on.

Steve

On Sat, Aug 12, 2017 at 9:00 AM, Moritz Lennert <
mlenn...@club.worldonline.be> wrote:

> On 12/08/17 16:45, Steven Pawley wrote:
>
>> Hello,
>>
>> I'm getting around to completing my TODO list for the add-on
>> r.terrain.texture, which includes generating the additional terrain metrics
>> (terrain surface concavity and convexity), and calculating the nested 8,12
>> or 16 class terrain classification as per Iwahashi and Pike (2007).
>> Everything is working but the script uses r.mfilter, which I supply a
>> matrix filter to, stored as a text file.
>>
>> Is there a makefile command that can be used to bundle a text file with
>> the add on, which can then be found using set_path?
>>
>
> Can't you include the filter as a string directly in the code ?
>
> Moritz
>
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] Inclusion of additional files with add-on

[Steven Pawley]

Hello,

I'm getting around to completing my TODO list for the add-on
r.terrain.texture, which includes generating the additional terrain metrics
(terrain surface concavity and convexity), and calculating the nested 8,12
or 16 class terrain classification as per Iwahashi and Pike (2007).
Everything is working but the script uses r.mfilter, which I supply a
matrix filter to, stored as a text file.

Is there a makefile command that can be used to bundle a text file with the
add on, which can then be found using set_path?

Steve
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] r.colors.matplotlib

[Steven Pawley]

Thanks Vaclav,

Will see if I can figure out.

Steve

On Thu, Jun 1, 2017 at 8:51 PM, Vaclav Petras  wrote:

> Hi Steve,
>
> On Tue, May 30, 2017 at 3:10 PM, Steven Pawley 
> wrote:
>
>>
>> Thanks for writing the r.colors.matplotlib tools because I use the
>> matplotlib colorramps very frequently both in GRASS and in python more
>> generally.
>>
>
> Glad to hear that.
>
>
>>
>> I seem to be having a problem of using the perceptually uniform ramps in
>> the r.colors.matplotlib add on, receiving the error: ERROR: Matplotlib
>> 2.0.2 does not contain color table 
>>
>> I'm on linux and matplotlib 2.0.2 has the perceptually uniform ramps, and
>> I can use them if I display data in python within a GRASS session directly
>> (i.e. plt.imshow(numpyraster, cmap='plasma').
>>
>> Is there a reason why r.colors.matplotlib can't see these ramps? It still
>> works with all of the traditional matplotlib ramps.
>
>
> The module does not check for specific color maps in mpl, but it uses
> general code to check the presence of a requested color map. It imports
> matplotlib.cm.datad dictionary. Perhaps, this dictionary does not contain
> the new color maps. The checking is not ideal, it should based on
> cm.get_map() function. I think the reason why I "ask for permission" here
> instead is that I was not able to figure out what is the error state from
> get_map() or how to transfer it to the user. But perhaps you will be more
> successful or 2.0 behaves differently. The relevant code is here:
>
> https://trac.osgeo.org/grass/browser/grass-addons/grass7/
> raster/r.colors.matplotlib/r.colors.matplotlib.py#L164
>
> Best,
> Vaclav
>
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] r.colors.matplotlib

[Steven Pawley]

Hello Vaclav,

Thanks for writing the r.colors.matplotlib tools because I use the
matplotlib colorramps very frequently both in GRASS and in python more
generally.

I seem to be having a problem of using the perceptually uniform ramps in
the r.colors.matplotlib add on, receiving the error: ERROR: Matplotlib
2.0.2 does not contain color table 

I'm on linux and matplotlib 2.0.2 has the perceptually uniform ramps, and I
can use them if I display data in python within a GRASS session directly
(i.e. plt.imshow(numpyraster, cmap='plasma').

Is there a reason why r.colors.matplotlib can't see these ramps? It still
works with all of the traditional matplotlib ramps.

Steve
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3352: Behavior of v.sort.points

[Steven Pawley]

Thanks Moritz for explaining that.

I suspected that there was something incorrect with my expectations, but
I'm glad that (1) was caught.

Steve



On Sat, May 27, 2017 at 10:04 AM, GRASS GIS  wrote:

> #3352: Behavior of v.sort.points
> --+-
>   Reporter:  spawley  |  Owner:  grass-dev@…
>   Type:  defect   | Status:  closed
>   Priority:  normal   |  Milestone:  7.4.0
>  Component:  Addons   |Version:  unspecified
> Resolution:  fixed|   Keywords:
>CPU:  Unspecified  |   Platform:  Linux
> --+-
>
> Comment (by mlennert):
>
>  Replying to [comment:2 mlennert]:
>  > Replying to [ticket:3352 spawley]:
>  >
>  > > (2) even if 'cat' is the first column, sorting by an integer column
>  does not appear to be sorting point datasets in numeric order. Example
>  from the nc_spm location:
>  > >
>  > > v.sort.points input=firestations@PERMANENT output=firestations_sorted
>  column=ID
>  > >
>  > > The order of the points based on opening the attribute table, and the
>  relationship between the 'ID' column and 'cat' appears to be the same as
>  the original dataset, where 'ID' was not in ascending order.
>  >
>  > This is not a bug: cat values are conserved, i.e. the same points have
>  the same cat value in the input and the output. However, points are in a
>  different order in the file after v.sort.points. Category values of
>  objects do not define order.
>
>  Try
>
>
>  {{{
>  d.vect map=firestations icon=basic/circle size=0.5 size_column=ID
>  d.vect map=firestations_sorted icon=basic/circle size=0.5 size_column=ID
>  }}}
>
>  and compare the two maps to see the difference.
>
>  Moritz
>
> --
> Ticket URL: 
> GRASS GIS 
>
>
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] Behavior of v.sort.points

[Steven Pawley]

Hello devs,

I appear to be having some problems with the add on v.sort.points. Two
issues, potentially bugs:

(1) the module fails if the 'cat' column is not the first attribute in the
table (i.e. if points have been generated from a database table), returning
the error:

'Error in sqlite3_prepare(): duplicate column name: cat'

(2) even if 'cat' is the first column, sorting by an integer column does
not appear to be sorting point datasets in numeric order. Example from the
nc_spm location:

v.sort.points input=firestations@PERMANENT output=firestations_sorted
column=ID

The order of the points based on opening the attribute table, and the
relationship between the 'ID' column and 'cat' appears to be the same as
the original dataset, where 'ID' was not in ascending order.

Either I'm doing something wrong or the add on requires some tweaks.
Otherwise this would be a useful tool.

Steve
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS-SVN] r70999 - grass-addons/grass7/raster/r.learn.ml

[Steven Pawley]

Thanks! Made a slip-up there.

I'm working on extending multiprocessing to the permutation feature importances 
so I'll make the fix tonight

Steve

Sent from my iPhone

> On May 2, 2017, at 5:07 PM, Vaclav Petras  wrote:
> 
> 
>> On Tue, May 2, 2017 at 12:45 PM,  wrote:
>> Author: spawley
>> Date: 2017-05-02 09:45:12 -0700 (Tue, 02 May 2017)
>> New Revision: 70999
>> 
>> Modified:
>>grass-addons/grass7/raster/r.learn.ml/r.learn.ml.py
>>grass-addons/grass7/raster/r.learn.ml/r_learn_utils.py
>> Log:
>> r.learn.ml using multiprocessing for all cross-validations
>> 
>> Modified: grass-addons/grass7/raster/r.learn.ml/r.learn.ml.py
>> ===
>> --- grass-addons/grass7/raster/r.learn.ml/r.learn.ml.py 2017-05-02 13:41:15 
>> UTC (rev 70998)
>> +++ grass-addons/grass7/raster/r.learn.ml/r.learn.ml.py 2017-05-02 16:45:12 
>> UTC (rev 70999)
>> @@ -249,13 +249,15 @@
>> ...
>> @@ -176,9 +215,9 @@
>>  try:
>>  list(scoring_methods.keys()).index(i)
>>  except:
>> -print('Scoring ' + i + ' is not a valid scoring method')
>> -print('Valid methods are:')
>> -print(scoring_methods.keys())
>> +gscript.fatal('Scoring ' + i + ' is not a valid scoring method')
>> +gscript.message('Valid methods are:')
>> +gscript.message(scoring_methods.keys())
> 
> Hi Steven,
> 
> the fatal call will end the execution. What you can do is to put there a 
> multiline string.
> 
> Vaclav
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] installing r.learn.ml addon - compilation error

[Steven Pawley]

Hi Paulo,

Try it now, it works for on linux and windows without being previously
installed.

Steve

On Fri, Apr 21, 2017 at 2:03 PM, Paulo van Breugel 
wrote:

> Hi Steven
>
> I am trying to update the r.learn.ml addon, but I am getting the
> following error:
>
> GRASS 7.3.svn (nc_spm_08_grass7):~ > g.extension extension=r.learn.ml
> WARNING: Extension  already installed. Re-installing...
> Fetching  from GRASS GIS Addons repository (be patient)...
> Compiling...
> Traceback (most recent call last):
>   File "/tmp/grass7-paulo-31834/tmpa1dITz/r.learn.ml/scripts/r.learn.ml",
> line 364, in 
> from raster_learning import (
> ImportError: cannot import name predict
> make: *** [r.learn.ml.tmp.html] Error 1
> ERROR: Compilation failed, sorry. Please check above error messages.
> [Raster MASK present]
>
> Any idea?
>
> Cheers,
>
> Paulo
>
>
>
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3269: r.roughness.vector: bug in rescale formula

[Steven Pawley]

Hi Stefan,

+1 for that suggestion. There might be a few variants / other
directional measures that could be included with a 'method' argument like
real surface area.

Cheers,

Steve

On Wed, Jan 25, 2017 at 3:32 PM, Blumentrath, Stefan <
stefan.blumentr...@nina.no> wrote:

> Hi Steven,
>
>
>
> And many thanks for clarification and contrasting results in detail…
>
> And sorry for throwing the question at the list without having checked
> that (I just scanned the module code).
>
>
>
> I noticed the difference in the references referred to in both modules.
> From scanning the code they seemed to do  the same thing, but now I see
> that the main difference is probably that r.vector.ruggedness uses slope
> directly for weighting vector strength, while r.roughness.vector uses the
> “inverted” slope / colatitude angle (90 – slope). Apart from that both
> modules look quite similar (as are the names) in terms of what they do to
> the DEM.
>
>
>
> So, there might still be a point of consolidating them into one module
> which offers both the “Sappington et al. 2007 metric” and the “Hobson 1972
> metric”…
>
>
>
> Just a thought…
>
>
>
> Kind regards,
>
> Stefan
>
>
>
>
>
> *From:* Steven Pawley [mailto:dr.stevenpaw...@gmail.com]
> *Sent:* onsdag 25. januar 2017 21.18
> *To:* Luca Delucchi 
> *Cc:* Blumentrath, Stefan ;
> carlos.grohm...@gmail.com; grass-dev@lists.osgeo.org
> *Subject:* Re: [GRASS-dev] [GRASS GIS] #3269: r.roughness.vector: bug in
> rescale formula
>
>
>
> Hi,
>
>
>
> There seems to be some difference in the calculation between these two
> add-ons. Apologies for any duplication, but in the way of explanation I
> wrote the parallelized r.vector.ruggedness because I needed to calculate
> the VRM measure in a hurry and was confused about the results from
> r.roughness.vector.
>
>
>
> The result from r.vector.ruggedness is identical (apart from how slope and
> aspect are calculated) to the result from the Sappington et al. 2007 paper
> and the Sappington-authored script in ArcGIS (see attached derived from the
> nc_spm_08 grass dataset). The form of the calculation is slightly different
> (but the end result is the same) because I average rather than sum the x,y,
> and z rasters to avoid a strong edge effect because AFAIK r.mapcalc doesn't
> have an automated method of dealing with bordering nulls (hence the
> averaging workaround). The r.vector.ruggedness results are also the same as
> how SAGA GIS calculates this metric, and it is this version of the 'VRM'
> metric that has been used extensively in the literature over the past few
> years.
>
>
>
> r.roughness.vector appears to produce a very different result (see
> attached). Perhaps this represents an alternative implementation, but the
> main difference lies in how the DEM is decomposed into its x,y, and z
> components.
>
>
>
> So if the add-ons are to be merged then perhaps this needs to be resolved?
>
>
>
> Steve
>
>
>
> On Tue, Jan 24, 2017 at 3:43 PM, Luca Delucchi 
> wrote:
>
> On 24 January 2017 at 09:55, Blumentrath, Stefan
>  wrote:
> > Hi,
> >
>
> Hi,
>
> > Another related question: There are now two AddOns for this purpose:
> >
> > r.roughness.vector (last changed 2 years ago)
> >
> > and from 2016: r.vector.ruggedness
> >
> > Both basically calculate the same metric(s), though they have their
> differences and both have their pros:
> > r.roughness.vector offers more advanced options (esp. useful for multi
> scale application)
> > r.vector.ruggedness uses parallelization
> >
> > Having two modules for one task / metric, which are not conceptually
> different is a bit confusing...
> > IMHO both should be merged into one, keeping the strength of each of
> them...
> >
>
> +1 for merging... this could be a work for the next code sprint
>
> > Other thoughts?
> >
> > Cheers
> > Stefan
> >
>
> --
> ciao
> Luca
>
> www.lucadelu.org
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
>
>
>
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Addons: Python error when checking if package is installed - r.randomforest

[Steven Pawley]

Thank you Markus and Vaclav for all of the suggestions and I’m glad that it 
appears to be working now.

Unfortunately I did not think far enough ahead when I named ‘r.randomforest’. 
Once I have a relatively stable template for the machine learning 
classification/regression, I was planning to implement some other methods in 
the scikit-learn toolbox that are commonly applied in the geoscientific, realm, 
e.g., logistic regression, svm, boosted regression trees, nearest neighbor, 
calling these r.scikit.randomforest, r.scikit.svn, r.scikit.nn, etc. 

Alternatively I could incorporate them within the same add on so that there are 
several different tabs for the parameters that are associated with each method, 
and have a ‘classifier’ option in the required section. I’m guessing that this 
is the best option although its might still necessitate a name change to 
something like ‘r.scikit.learn’.

Steve

> On Jul 8, 2016, at 3:04 AM, Markus Neteler  wrote:
> 
> FWIW

___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Addons: Python error when checking if package is installed - r.randomforest

[Steven Pawley]

Thanks for this Vaclav. I'm confused as to why sklearn is not being found
by testing for the presence of the module, but I can make an update to use
the try-except method to get around this.

On Mon, Jun 27, 2016 at 8:00 AM, Vaclav Petras  wrote:

>
> On Mon, Jun 27, 2016 at 7:18 AM, Markus Neteler  wrote:
>
>> (I noted that various GRASS scripts use different ways for this check,
>> the test for 3rd party packages should be standardized.)
>>
>
>
> I so far promoted lazy import* as a solution to these dependencies. The
> important part is that import happens after calling grass.script.parser(),
> so for manual and GUI, you don't need to have the dependencies at all. For
> simple modules (with dependency usage limited to main() function), the
> import** is done in main() after parser() call and try-except with
> ImportError is used to catch the missing dependency and report an error to
> the user (example: r.colors.cubehelix***). More complicated modules (with
> explicit dependency usage outside of main() function) must be solved on a
> case-to-case basis (examples: v.class.ml and v.class.mlpy).
>
> Older discussions:
>
> https://lists.osgeo.org/pipermail/grass-dev/2015-February/073734.html
> https://lists.osgeo.org/pipermail/grass-dev/2016-March/079610.html
>
> Examples:
>
>
> https://trac.osgeo.org/grass/browser/grass-addons/grass7/raster/r.colors.cubehelix/r.colors.cubehelix.py#L186
>
> https://trac.osgeo.org/grass/browser/grass-addons/grass7/raster/r.flexure/r.flexure.py?rev=64452
> https://trac.osgeo.org/grass/changeset/66482/
>
> Vaclav
>
>
> * By lazy import I mean an import which is not done at the beginning of
> the file, but somewhere in some function only right before it is actually
> needed.
>
> ** I think that import with try-except is just more straightforward than
> testing the presence of the module/package before the import. It also
> follows "it is easier to ask for forgiveness than for permission" (EAFP),
> so one can say that it is more Pythonic.
>
> *** r.colors.cubehelix is actually not the basic example, because it
> provides fallback when the dependency is not available and shows just
> warning and an error but without exiting (uses warning() and error() and it
> does not use not fatal()).
>
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] RandomForest classifier for imagery groups add-on

[Steven Pawley]

Hello Paulo,
Many thanks for this. I updated the mode last night to include the ability to 
force regression mode, as well as including some more error checking for valid 
combinations of input parameters. Classification mode also checks that the 
input labelled pixels are CELL type. I'm not outputting all of the appropriate 
uncertainty measures like RSQ yet for regression mode yet, but I'll add those 
in.
That is interesting that you had better performance when using regression. I 
will have to check that for my application using scikit learn. In R using the 
randomforest package, the results were pretty much identical but my classes 
were balanced already, which I think is one factor that can lead to significant 
differences between binary classification probabilities vs regression.

Yes definitely will use this as a template to include other methods. I Only 
recently switched my work from R to Python but am just submitting a paper based 
on R which uses a range of classifiers like randomforest, GLM, GAM, and MARS 
which it was useful to evaluate the differences.
Steve


_
From: Paulo van Breugel 
Sent: Sunday, March 27, 2016 3:11 AM
Subject: Re: [GRASS-dev] RandomForest classifier for imagery groups add-on
To: Vaclav Petras , Steven Pawley 

Cc:  


   Hi Steve  
   
 Yes, your user case will not differ methodologically from species modeling 
based on presence/absence. One reason I was asking for the regression 
randomForest is that in one article (can't remember the title, will look it up) 
it was found that the regression approach yielded better results, even though 
the response variable is binary. One your help page, you write that 
r.randomforest performs random forest classification and regression, and the 
regression mode can be used by setting the mode to the regression option. But I 
am not seeing that option?  
   
 Great you are planning other methods as well. Giving model uncertainties 
(quite an issue in species distribution modeling), having multiple methods is 
really a plus, especially as it allows one to build consensus models [1] and 
combine them to create uncertainty maps.  
   
 Cheers,  
   
 Paulo  
   
 [1]Marmion, M., Parviainen, M., Luoto, M., Heikkinen, R.K., & Thuiller, W. 
2009. Evaluation of consensus methods in predictive species distribution 
modelling.   Diversity and Distributions 15: 59–69.  
   

  On 27-03-16 00:47, Steven Pawley wrote:   
Hi Vaclaw and Paulo,  
   Thanks for those pointers re. lazy technique and documentation. 
I have a RandomForest diagram to explain the process, as well as some examples, 
so I'll update documentation next week.  
   Paulo thanks for running a few tests. It looks there is an error 
with the class_weight parameter, I'll check into that.  
   In terms of species distribution modelling, I have been using 
the tool for landslide susceptibility modelling, which I believe is 
methodologically similar to SDM in terms of having a binary response variable. 
I have been doing this for the area of Alberta, using an 8000 x 14000 pixel and 
17 band stack of predictors. In the case of a binary response variable, the 
usual approach is to run random forest in classification mode, i.e. with fully 
grown trees, but use the class probabilities to represent the 'species' or 
'landslide' index.  
   I am planning to implement other methods in the scikit learn 
package, which represents a trivial change to the module once he bugs are 
ironed out. I will probably look to create modules for SVM and logistic 
regression, and maybe  nearest neighbours classification. Certainly open to any 
suggestions.  
   Steve_
 From: Vaclav Petras <wenzesl...@gmail.com>
 Sent: Saturday, March 26, 2016 11:21 AM
 Subject: Re: [GRASS-dev] RandomForest classifier for imagery groups add-on
 To: Steven Pawley <dr.stevenpaw...@gmail.com>
 Cc: <grass-dev@lists.osgeo.org>
 
 
      
   On Sat, Mar 26, 2016 at 12:40 PM, Steven Pawley
 wrote:
 I would like to draw your attention to a new GRASS add-on, 
r.randomforest, which uses the scikit-learn and pandas Python packages to 
classify GRASS rasters.  
   Thanks, this looks good. Please consider adding an image to 
the documentation to better promote the module [1] and also an example which 
would work with the NC SPM dataset [2]. For the addon to generate documentation 
on the server and work well at few other special occasions, it is advantageous 
to employ lazy import technique for the non-standard dependencies, see for 
example   v.class.ml and v.class.mlpy [3].   
   
  

Re: [GRASS-dev] RandomForest classifier for imagery groups add-on

[Steven Pawley]

Hi Vaclaw and Paulo,
Thanks for those pointers re. lazy technique and documentation. I have a 
RandomForest diagram to explain the process, as well as some examples, so I'll 
update documentation next week.
Paulo thanks for running a few tests. It looks there is an error with the 
class_weight parameter, I'll check into that.
In terms of species distribution modelling, I have been using the tool for 
landslide susceptibility modelling, which I believe is methodologically similar 
to SDM in terms of having a binary response variable. I have been doing this 
for the area of Alberta, using an 8000 x 14000 pixel and 17 band stack of 
predictors. In the case of a binary response variable, the usual approach is to 
run random forest in classification mode, i.e. with fully grown trees, but use 
the class probabilities to represent the 'species' or 'landslide' index.
I am planning to implement other methods in the scikit learn package, which 
represents a trivial change to the module once he bugs are ironed out. I will 
probably look to create modules for SVM and logistic regression, and maybe  
nearest neighbours classification. Certainly open to any suggestions.
Steve
_
From: Vaclav Petras 
Sent: Saturday, March 26, 2016 11:21 AM
Subject: Re: [GRASS-dev] RandomForest classifier for imagery groups add-on
To: Steven Pawley 
Cc:  


   
 On Sat, Mar 26, 2016 at 12:40 PM, Steven Pawley  
 wrote: 
   I would like to draw your attention to a new GRASS add-on, 
r.randomforest, which uses the scikit-learn and pandas Python packages to 
classify GRASS rasters. 
  Thanks, this looks good. Please consider adding an image to the 
documentation to better promote the module [1] and also an example which would 
work with the NC SPM dataset [2]. For the addon to generate documentation on 
the server and work well at few other special occasions, it is advantageous to 
employ lazy import technique for the non-standard dependencies, see for example 
v.class.ml and v.class.mlpy [3].

  Vaclav
  
[1] https://trac.osgeo.org/grass/wiki/Submitting/Docs#Images
[2] https://grass.osgeo.org/download/sample-data/
[3] https://trac.osgeo.org/grass/changeset/66482/
   


  ___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] RandomForest classifier for imagery groups add-on

[Steven Pawley]

Hello developers,
I would like to draw your attention to a new GRASS add-on, r.randomforest, 
which uses the scikit-learn and pandas Python packages to classify GRASS 
rasters. Similar to existing GRASS classification methods, it uses an imagery 
group and a raster of labelled pixels as the inputs for the classification. It 
also reads the rasters row-by-row, and then bundles these rows based on a user 
specified row increment to the classifier to keep memory requirements low, but 
also allow efficient classification because the scikit-learn implementation is 
multithreaded by default, and row-by-row results in too much stop-start 
behaviour. The feature importance scores and out-of-bag error are displayed in 
the command window.
I would appreciate testing - you need to have scikit-learn and pandas installed 
in your Python environment which is easy on Linux and OS X, and instructions 
are provided in the tool for Windows.
I have another add-on that I will upload soon, r.roc, which generates ROC and 
AUROC for prediction models.
Steve
Sent from Outlook Mobile
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] R and GRASS integration

[Steven Pawley]

Thank you Moritz,

Yes I have also had difficulties with Rpy2 apart from on Linux. Also, Rpy2 is 
quite onerous in terms of effort required to integrate R scripts into Python. 
Your solution certainly works,  but as you mentioned it makes the R script 
harder to maintain. PypeR is another alternative and is straightforward to 
install and is simpler from a user perspective. 

I would also be interested in hearing opinions from 'true' developers who have 
much greater expertise than myself in this area.

Kind regards,

Steve

> On Jan 4, 2016, at 2:31 AM, Moritz Lennert  
> wrote:
> 
>> On 04/01/16 10:28, Moritz Lennert wrote:
>>> On 03/01/16 23:54, Steven Pawley wrote:
>>> Like many R-GRASS users, I have a collection of R scripts that
>>> interact with GRASS to perform various workflows. I have debated
>>> about converting these to Python using Rpy2, although this package
>>> can be a difficult to install on all platforms and depends on
>>> specific versions of R and Python. I noticed that Moritz Lennert
>>> recently developed a GRASS add on which consists of simply writing
>>> out the R commands to a temporary script for R to run.
>> 
>> [...]
>>> 
>>> Does this represent a desirable or even acceptable approach for
>>> embedding R scripts into grass add ons, or is Rpy2 the 'official'
>>> approach.
>> 
>> I wouldn't consider my approach in any way official, but AFAIK, rpy2
>> does not have any "official" status in GRASS either. In my particular
>> case (v.class.mlR) this was a quick and dirty hack for a course I had to
>> teach. The difficulty of getting rpy2 installed on the lab machines on
>> short notice was one of the motivations not to use it. I also agree that
>> dependency on rpy2 can be a nuisance and has caused me some headaches
>> with other modules, before. However, the approach I used (and others
>> have used before) is a bit unwieldy and makes maintaining such modules a
>> bit of a pain.
>> 
>> So, I'm curious to hear the opinions of others...
> 
> See [1] for a related issue.
> 
> Moritz
> 
> [1] https://trac.osgeo.org/grass/ticket/1290
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] R and GRASS integration

[Steven Pawley]

Dear GRASS developers,

I'm wondering what represents the most desirable approach to integrate R 
scripts into GRASS so that they are part of the FOSS community. It seems like 
the integration with R represents a fairly important component of GRASS, which 
relies on R for many geostatistical operations and machine learning algorithms. 
There are numerous publications that describe workflows using GRASS and R, but 
rarely are these publicly available. 

Like many R-GRASS users, I have a collection of R scripts that interact with 
GRASS to perform various workflows. I have debated about converting these to 
Python using Rpy2, although this package can be a difficult to install on all 
platforms and depends on specific versions of R and Python. I noticed that 
Moritz Lennert recently developed a GRASS add on which consists of simply 
writing out the R commands to a temporary script for R to run. Based on this, I 
have converted some of my scripts which include random forest classification on 
both raster and vector data, random forest presence prediction, and spectral 
separability so that they can be accessed directly from the GRASS GUI.

Does this represent a desirable or even acceptable approach for embedding R 
scripts into grass add ons, or is Rpy2 the 'official' approach. Also, there are 
other alternatives to Rpy2 like PypeR which have the both conveniences and 
inconveniences, so should these also be considered? Or should we look at having 
a feature like in QGIS processing where R scripts can be launched directly with 
GUI options?

Steve
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev