Re: [GRASS-dev] R and GRASS integration
On 06-01-16 18:24, Rainer M Krug wrote: Paulo van Breugelwrites: On Wed, Jan 6, 2016 at 10:27 AM, Rainer M Krug wrote: Paulo van Breugel writes: > I by any stretch of imagination a developer, but I did use the > combination of shell or pythons script with R, basically following the > approach you described, having a python or shell script write a R > script to a text file and run it. I think it can work well, and not > that much harder to maintain. But I also would be very interested to > learn how to do this better. I also would be interested to see the > randomForest scripts you mentioned Steven, are you already sharing it > somewhere? > > As you mentioned, there are probably many people using / writing R > scripts that interact with GRASS. For some it will be easier, or it > may be more logical for them, to turn these into R packages rather > than writing a GRASS addon. I am one of those. I have thought about making a GRASS (or QGIS) addon / plugin, but I stayed with the R package. I have a complete simulation written in R which uses spatial data from GRASS, does simulations, and returns the results to GRASS. To run the simulations in itself is a three liner in R. > It would be nice if there would be some kind of repository where > people share such code (github perhaps?). I am sure there are existing > ones on e.g., github, so perhaps just a GRASS-wiki page listing > existing repositories would be enough. I know there is > https://grasswiki.osgeo.org/wiki/R_statistics, but I don't think there > is an place on the GRASS website, wiki or trac to share/list R code? > Would this be of interest to create such page (on the Wiki perhaps?). Different people use different repos (I use e.g. github and gitlab) so creating another place where I should publish my code would be not really an option. What would be an idea to make it easy (probably even easier even than editing a wiki page?) to add a repo to a list of projects which integrate R in GRASS or GRASS in R, and which could indicate the last commit to aser if the repos are current or just archives from e.g. papers or finished projects. +1 That would be something quick to implement. However, what form did you have in mind, if not a wiki page? I wouldn't mind creating such page, but perhaps first some further ideas on the best form from others as well. I thought about a field, where I just add the link to the repository and some info (Author, short description, possibly License type, ...) and this is automatically added to the page including information from the repo like last updated, link to the README, ... But this might be getting to complicated. But I know it from my side - editing a wiki page is often a "will do later" thing. Also a system where there is an form where one fills in the info and it is emailed to somebody to add it to the wiki would work. Creating an automated form, etc wouldn't be something I could do, but I wouldn't mind maintaining a a list on the GRASS wiki based on information from a user form. Perhaps a Google form? Any ideas / opinions on this from the devs? My repo is private at the moment, but I plan to make it open in the next few weeks. A brief presentation about a very similar simulation model can be found at https://github.com/rkrug/INTECOL_2013_Optimizing Thanks for sharing, interesting. Looking forwards seeing some further code Thanks - I will keep you posted. Coming back to the initial question: What we would need in the meantime is possibly a discussion, on how R functions could be used in an automated way - i.e. something along the lines of an R package, which contains a set of defined functions like: getFunctionNames() :: which returns the function names which can be called from GRASS getFunctionInterface(functionName) :: which returns the arguments of the function functionName (similar to the interfaceDescription (I think it is called differently) in GRASS commands) would make it possible to just 1) load any R package which has these functions 2) and get all function names and arguments needed 3) and just call the R function in an easy way like g.call.R package=PACKAGENAME function=FUNCTIONNAME arguments="arguments for the R function) The function could than be executed by using either a relatively simple python or direct Rscript approach. Also, one could even dynamically load an R package and construct all the calls including help, so they could be called by using g.call.R.PACKAGENAME.FUNCTIONNAME arg1=ARG1 arg2=ARG2 From the R side, this would be relatively
Re: [GRASS-dev] R and GRASS integration
Paulo van Breugelwrites: > I by any stretch of imagination a developer, but I did use the > combination of shell or pythons script with R, basically following the > approach you described, having a python or shell script write a R > script to a text file and run it. I think it can work well, and not > that much harder to maintain. But I also would be very interested to > learn how to do this better. I also would be interested to see the > randomForest scripts you mentioned Steven, are you already sharing it > somewhere? > > As you mentioned, there are probably many people using / writing R > scripts that interact with GRASS. For some it will be easier, or it > may be more logical for them, to turn these into R packages rather > than writing a GRASS addon. I am one of those. I have thought about making a GRASS (or QGIS) addon / plugin, but I stayed with the R package. I have a complete simulation written in R which uses spatial data from GRASS, does simulations, and returns the results to GRASS. To run the simulations in itself is a three liner in R. > It would be nice if there would be some kind of repository where > people share such code (github perhaps?). I am sure there are existing > ones on e.g., github, so perhaps just a GRASS-wiki page listing > existing repositories would be enough. I know there is > https://grasswiki.osgeo.org/wiki/R_statistics, but I don't think there > is an place on the GRASS website, wiki or trac to share/list R code? > Would this be of interest to create such page (on the Wiki perhaps?). Different people use different repos (I use e.g. github and gitlab) so creating another place where I should publish my code would be not really an option. What would be an idea to make it easy (probably even easier even than editing a wiki page?) to add a repo to a list of projects which integrate R in GRASS or GRASS in R, and which could indicate the last commit to aser if the repos are current or just archives from e.g. papers or finished projects. My repo is private at the moment, but I plan to make it open in the next few weeks. A brief presentation about a very similar simulation model can be found at https://github.com/rkrug/INTECOL_2013_Optimizing My gitlab repos (most or all private at the moment) are at https://gitlab.com/rkrug/asm and the simulation is at (also still private but it will change soon) https://gitlab.com/rkrug/asm For reference, my github page is at https://github.com/rkrug Cheers, Rainer > > Paulo > > > On 04-01-16 16:14, Steven Pawley wrote: >> Thank you Moritz, >> >> Yes I have also had difficulties with Rpy2 apart from on >> Linux. Also, Rpy2 is quite onerous in terms of effort required to >> integrate R scripts into Python. Your solution certainly works, but >> as you mentioned it makes the R script harder to maintain. PypeR is >> another alternative and is straightforward to install and is simpler >> from a user perspective. >> >> I would also be interested in hearing opinions from 'true' >> developers who have much greater expertise than myself in this area. >> >> Kind regards, >> >> Steve >> >>> On Jan 4, 2016, at 2:31 AM, Moritz Lennert >>> wrote: >>> On 04/01/16 10:28, Moritz Lennert wrote: > On 03/01/16 23:54, Steven Pawley wrote: > Like many R-GRASS users, I have a collection of R scripts that > interact with GRASS to perform various workflows. I have debated > about converting these to Python using Rpy2, although this package > can be a difficult to install on all platforms and depends on > specific versions of R and Python. I noticed that Moritz Lennert > recently developed a GRASS add on which consists of simply writing > out the R commands to a temporary script for R to run. [...] > Does this represent a desirable or even acceptable approach for > embedding R scripts into grass add ons, or is Rpy2 the 'official' > approach. I wouldn't consider my approach in any way official, but AFAIK, rpy2 does not have any "official" status in GRASS either. In my particular case (v.class.mlR) this was a quick and dirty hack for a course I had to teach. The difficulty of getting rpy2 installed on the lab machines on short notice was one of the motivations not to use it. I also agree that dependency on rpy2 can be a nuisance and has caused me some headaches with other modules, before. However, the approach I used (and others have used before) is a bit unwieldy and makes maintaining such modules a bit of a pain. So, I'm curious to hear the opinions of others... >>> See [1] for a related issue. >>> >>> Moritz >>> >>> [1] https://trac.osgeo.org/grass/ticket/1290 >> ___ >> grass-dev mailing list >> grass-dev@lists.osgeo.org >> http://lists.osgeo.org/mailman/listinfo/grass-dev > > ___ > grass-dev
Re: [GRASS-dev] R and GRASS integration
On Wed, Jan 6, 2016 at 10:27 AM, Rainer M Krugwrote: > Paulo van Breugel writes: > > > I by any stretch of imagination a developer, but I did use the > > combination of shell or pythons script with R, basically following the > > approach you described, having a python or shell script write a R > > script to a text file and run it. I think it can work well, and not > > that much harder to maintain. But I also would be very interested to > > learn how to do this better. I also would be interested to see the > > randomForest scripts you mentioned Steven, are you already sharing it > > somewhere? > > > > As you mentioned, there are probably many people using / writing R > > scripts that interact with GRASS. For some it will be easier, or it > > may be more logical for them, to turn these into R packages rather > > than writing a GRASS addon. > > I am one of those. I have thought about making a GRASS (or QGIS) addon / > plugin, but I stayed with the R package. I have a complete simulation > written in R which uses spatial data from GRASS, does simulations, and > returns the results to GRASS. > > To run the simulations in itself is a three liner in R. > > > It would be nice if there would be some kind of repository where > > people share such code (github perhaps?). I am sure there are existing > > ones on e.g., github, so perhaps just a GRASS-wiki page listing > > existing repositories would be enough. I know there is > > https://grasswiki.osgeo.org/wiki/R_statistics, but I don't think there > > is an place on the GRASS website, wiki or trac to share/list R code? > > Would this be of interest to create such page (on the Wiki perhaps?). > > Different people use different repos (I use e.g. github and gitlab) so > creating another place where I should publish my code would be not > really an option. What would be an idea to make it easy (probably even > easier even than editing a wiki page?) to add a repo to a list of > projects which integrate R in GRASS or GRASS in R, and which could > indicate the last commit to aser if the repos are current or just > archives from e.g. papers or finished projects. > +1 That would be something quick to implement. However, what form did you have in mind, if not a wiki page? I wouldn't mind creating such page, but perhaps first some further ideas on the best form from others as well. > > My repo is private at the moment, but I plan to make it open in the next > few weeks. > > A brief presentation about a very similar simulation model can be > found at > > https://github.com/rkrug/INTECOL_2013_Optimizing > > > Thanks for sharing, interesting. Looking forwards seeing some further code > > My gitlab repos (most or all private at the moment) are at > > https://gitlab.com/rkrug/asm > > and the simulation is at (also still private but it will change soon) > > https://gitlab.com/rkrug/asm > > For reference, my github page is at > > https://github.com/rkrug > > Cheers, > > Rainer > > > > > > Paulo > > > > > > On 04-01-16 16:14, Steven Pawley wrote: > >> Thank you Moritz, > >> > >> Yes I have also had difficulties with Rpy2 apart from on > >> Linux. Also, Rpy2 is quite onerous in terms of effort required to > >> integrate R scripts into Python. Your solution certainly works, but > >> as you mentioned it makes the R script harder to maintain. PypeR is > >> another alternative and is straightforward to install and is simpler > >> from a user perspective. > >> > >> I would also be interested in hearing opinions from 'true' > >> developers who have much greater expertise than myself in this area. > >> > >> Kind regards, > >> > >> Steve > >> > >>> On Jan 4, 2016, at 2:31 AM, Moritz Lennert < > mlenn...@club.worldonline.be> wrote: > >>> > On 04/01/16 10:28, Moritz Lennert wrote: > > On 03/01/16 23:54, Steven Pawley wrote: > > Like many R-GRASS users, I have a collection of R scripts that > > interact with GRASS to perform various workflows. I have debated > > about converting these to Python using Rpy2, although this package > > can be a difficult to install on all platforms and depends on > > specific versions of R and Python. I noticed that Moritz Lennert > > recently developed a GRASS add on which consists of simply writing > > out the R commands to a temporary script for R to run. > [...] > > Does this represent a desirable or even acceptable approach for > > embedding R scripts into grass add ons, or is Rpy2 the 'official' > > approach. > I wouldn't consider my approach in any way official, but AFAIK, rpy2 > does not have any "official" status in GRASS either. In my particular > case (v.class.mlR) this was a quick and dirty hack for a course I had > to > teach. The difficulty of getting rpy2 installed on the lab machines on > short notice was one of the motivations not to use it. I also agree > that > dependency on rpy2 can be a nuisance and has caused me
Re: [GRASS-dev] R and GRASS integration
Paulo van Breugelwrites: > On Wed, Jan 6, 2016 at 10:27 AM, Rainer M Krug > wrote: > > Paulo van Breugel writes: > > > I by any stretch of imagination a developer, but I did use the > > combination of shell or pythons script with R, basically > following the > > approach you described, having a python or shell script write a > R > > script to a text file and run it. I think it can work well, and > not > > that much harder to maintain. But I also would be very > interested to > > learn how to do this better. I also would be interested to see > the > > randomForest scripts you mentioned Steven, are you already > sharing it > > somewhere? > > > > As you mentioned, there are probably many people using / writing > R > > scripts that interact with GRASS. For some it will be easier, or > it > > may be more logical for them, to turn these into R packages > rather > > than writing a GRASS addon. > > I am one of those. I have thought about making a GRASS (or QGIS) > addon / > plugin, but I stayed with the R package. I have a complete > simulation > written in R which uses spatial data from GRASS, does simulations, > and > returns the results to GRASS. > > > To run the simulations in itself is a three liner in R. > > > It would be nice if there would be some kind of repository where > > people share such code (github perhaps?). I am sure there are > existing > > ones on e.g., github, so perhaps just a GRASS-wiki page listing > > existing repositories would be enough. I know there is > > https://grasswiki.osgeo.org/wiki/R_statistics, but I don't think > there > > is an place on the GRASS website, wiki or trac to share/list R > code? > > Would this be of interest to create such page (on the Wiki > perhaps?). > > Different people use different repos (I use e.g. github and > gitlab) so > creating another place where I should publish my code would be not > really an option. What would be an idea to make it easy (probably > even > easier even than editing a wiki page?) to add a repo to a list of > projects which integrate R in GRASS or GRASS in R, and which could > indicate the last commit to aser if the repos are current or just > archives from e.g. papers or finished projects. > > > +1 That would be something quick to implement. However, what form did > you have in mind, if not a wiki page? I wouldn't mind creating such > page, but perhaps first some further ideas on the best form from > others as well. I thought about a field, where I just add the link to the repository and some info (Author, short description, possibly License type, ...) and this is automatically added to the page including information from the repo like last updated, link to the README, ... But this might be getting to complicated. But I know it from my side - editing a wiki page is often a "will do later" thing. Also a system where there is an form where one fills in the info and it is emailed to somebody to add it to the wiki would work. > > My repo is private at the moment, but I plan to make it open in > the next > few weeks. > > A brief presentation about a very similar simulation model can be > found at > > https://github.com/rkrug/INTECOL_2013_Optimizing > > > > > Thanks for sharing, interesting. Looking forwards seeing some further > code Thanks - I will keep you posted. Coming back to the initial question: What we would need in the meantime is possibly a discussion, on how R functions could be used in an automated way - i.e. something along the lines of an R package, which contains a set of defined functions like: getFunctionNames() :: which returns the function names which can be called from GRASS getFunctionInterface(functionName) :: which returns the arguments of the function functionName (similar to the interfaceDescription (I think it is called differently) in GRASS commands) would make it possible to just 1) load any R package which has these functions 2) and get all function names and arguments needed 3) and just call the R function in an easy way like g.call.R package=PACKAGENAME function=FUNCTIONNAME arguments="arguments for the R function) The function could than be executed by using either a relatively simple python or direct Rscript approach. Also, one could even dynamically load an R package and construct all the calls including help, so they could be called by using g.call.R.PACKAGENAME.FUNCTIONNAME arg1=ARG1 arg2=ARG2 From the R side, this would be relatively easy to include, (there are the packages rgrass7 and spgrass6 to interact with GRASS 7 and 6) and it I think that it should not be to difficult to implement on the GRASS side. This would than be very
Re: [GRASS-dev] R and GRASS integration
I by any stretch of imagination a developer, but I did use the combination of shell or pythons script with R, basically following the approach you described, having a python or shell script write a R script to a text file and run it. I think it can work well, and not that much harder to maintain. But I also would be very interested to learn how to do this better. I also would be interested to see the randomForest scripts you mentioned Steven, are you already sharing it somewhere? As you mentioned, there are probably many people using / writing R scripts that interact with GRASS. For some it will be easier, or it may be more logical for them, to turn these into R packages rather than writing a GRASS addon. It would be nice if there would be some kind of repository where people share such code (github perhaps?). I am sure there are existing ones on e.g., github, so perhaps just a GRASS-wiki page listing existing repositories would be enough. I know there is https://grasswiki.osgeo.org/wiki/R_statistics, but I don't think there is an place on the GRASS website, wiki or trac to share/list R code? Would this be of interest to create such page (on the Wiki perhaps?). Paulo On 04-01-16 16:14, Steven Pawley wrote: Thank you Moritz, Yes I have also had difficulties with Rpy2 apart from on Linux. Also, Rpy2 is quite onerous in terms of effort required to integrate R scripts into Python. Your solution certainly works, but as you mentioned it makes the R script harder to maintain. PypeR is another alternative and is straightforward to install and is simpler from a user perspective. I would also be interested in hearing opinions from 'true' developers who have much greater expertise than myself in this area. Kind regards, Steve On Jan 4, 2016, at 2:31 AM, Moritz Lennertwrote: On 04/01/16 10:28, Moritz Lennert wrote: On 03/01/16 23:54, Steven Pawley wrote: Like many R-GRASS users, I have a collection of R scripts that interact with GRASS to perform various workflows. I have debated about converting these to Python using Rpy2, although this package can be a difficult to install on all platforms and depends on specific versions of R and Python. I noticed that Moritz Lennert recently developed a GRASS add on which consists of simply writing out the R commands to a temporary script for R to run. [...] Does this represent a desirable or even acceptable approach for embedding R scripts into grass add ons, or is Rpy2 the 'official' approach. I wouldn't consider my approach in any way official, but AFAIK, rpy2 does not have any "official" status in GRASS either. In my particular case (v.class.mlR) this was a quick and dirty hack for a course I had to teach. The difficulty of getting rpy2 installed on the lab machines on short notice was one of the motivations not to use it. I also agree that dependency on rpy2 can be a nuisance and has caused me some headaches with other modules, before. However, the approach I used (and others have used before) is a bit unwieldy and makes maintaining such modules a bit of a pain. So, I'm curious to hear the opinions of others... See [1] for a related issue. Moritz [1] https://trac.osgeo.org/grass/ticket/1290 ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] R and GRASS integration
On 04/01/16 10:28, Moritz Lennert wrote: On 03/01/16 23:54, Steven Pawley wrote: Like many R-GRASS users, I have a collection of R scripts that interact with GRASS to perform various workflows. I have debated about converting these to Python using Rpy2, although this package can be a difficult to install on all platforms and depends on specific versions of R and Python. I noticed that Moritz Lennert recently developed a GRASS add on which consists of simply writing out the R commands to a temporary script for R to run. [...] Does this represent a desirable or even acceptable approach for embedding R scripts into grass add ons, or is Rpy2 the 'official' approach. I wouldn't consider my approach in any way official, but AFAIK, rpy2 does not have any "official" status in GRASS either. In my particular case (v.class.mlR) this was a quick and dirty hack for a course I had to teach. The difficulty of getting rpy2 installed on the lab machines on short notice was one of the motivations not to use it. I also agree that dependency on rpy2 can be a nuisance and has caused me some headaches with other modules, before. However, the approach I used (and others have used before) is a bit unwieldy and makes maintaining such modules a bit of a pain. So, I'm curious to hear the opinions of others... See [1] for a related issue. Moritz [1] https://trac.osgeo.org/grass/ticket/1290 ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] R and GRASS integration
On 03/01/16 23:54, Steven Pawley wrote: Like many R-GRASS users, I have a collection of R scripts that interact with GRASS to perform various workflows. I have debated about converting these to Python using Rpy2, although this package can be a difficult to install on all platforms and depends on specific versions of R and Python. I noticed that Moritz Lennert recently developed a GRASS add on which consists of simply writing out the R commands to a temporary script for R to run. [...] Does this represent a desirable or even acceptable approach for embedding R scripts into grass add ons, or is Rpy2 the 'official' approach. I wouldn't consider my approach in any way official, but AFAIK, rpy2 does not have any "official" status in GRASS either. In my particular case (v.class.mlR) this was a quick and dirty hack for a course I had to teach. The difficulty of getting rpy2 installed on the lab machines on short notice was one of the motivations not to use it. I also agree that dependency on rpy2 can be a nuisance and has caused me some headaches with other modules, before. However, the approach I used (and others have used before) is a bit unwieldy and makes maintaining such modules a bit of a pain. So, I'm curious to hear the opinions of others... Moritz ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] R and GRASS integration
Thank you Moritz, Yes I have also had difficulties with Rpy2 apart from on Linux. Also, Rpy2 is quite onerous in terms of effort required to integrate R scripts into Python. Your solution certainly works, but as you mentioned it makes the R script harder to maintain. PypeR is another alternative and is straightforward to install and is simpler from a user perspective. I would also be interested in hearing opinions from 'true' developers who have much greater expertise than myself in this area. Kind regards, Steve > On Jan 4, 2016, at 2:31 AM, Moritz Lennert> wrote: > >> On 04/01/16 10:28, Moritz Lennert wrote: >>> On 03/01/16 23:54, Steven Pawley wrote: >>> Like many R-GRASS users, I have a collection of R scripts that >>> interact with GRASS to perform various workflows. I have debated >>> about converting these to Python using Rpy2, although this package >>> can be a difficult to install on all platforms and depends on >>> specific versions of R and Python. I noticed that Moritz Lennert >>> recently developed a GRASS add on which consists of simply writing >>> out the R commands to a temporary script for R to run. >> >> [...] >>> >>> Does this represent a desirable or even acceptable approach for >>> embedding R scripts into grass add ons, or is Rpy2 the 'official' >>> approach. >> >> I wouldn't consider my approach in any way official, but AFAIK, rpy2 >> does not have any "official" status in GRASS either. In my particular >> case (v.class.mlR) this was a quick and dirty hack for a course I had to >> teach. The difficulty of getting rpy2 installed on the lab machines on >> short notice was one of the motivations not to use it. I also agree that >> dependency on rpy2 can be a nuisance and has caused me some headaches >> with other modules, before. However, the approach I used (and others >> have used before) is a bit unwieldy and makes maintaining such modules a >> bit of a pain. >> >> So, I'm curious to hear the opinions of others... > > See [1] for a related issue. > > Moritz > > [1] https://trac.osgeo.org/grass/ticket/1290 ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] R and GRASS integration
Dear GRASS developers, I'm wondering what represents the most desirable approach to integrate R scripts into GRASS so that they are part of the FOSS community. It seems like the integration with R represents a fairly important component of GRASS, which relies on R for many geostatistical operations and machine learning algorithms. There are numerous publications that describe workflows using GRASS and R, but rarely are these publicly available. Like many R-GRASS users, I have a collection of R scripts that interact with GRASS to perform various workflows. I have debated about converting these to Python using Rpy2, although this package can be a difficult to install on all platforms and depends on specific versions of R and Python. I noticed that Moritz Lennert recently developed a GRASS add on which consists of simply writing out the R commands to a temporary script for R to run. Based on this, I have converted some of my scripts which include random forest classification on both raster and vector data, random forest presence prediction, and spectral separability so that they can be accessed directly from the GRASS GUI. Does this represent a desirable or even acceptable approach for embedding R scripts into grass add ons, or is Rpy2 the 'official' approach. Also, there are other alternatives to Rpy2 like PypeR which have the both conveniences and inconveniences, so should these also be considered? Or should we look at having a feature like in QGIS processing where R scripts can be launched directly with GUI options? Steve ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev