Re: [R-sig-Geo] speed problem with %over% function

2012-03-01 Thread Edzer Pebesma
Bastien, your report is very helpful. I believe I managed to get overlay()'s efficiency back into the over() methods. Because of the way the code is set up for over(), the efficiency gain will propagate to all over() methods where the second argument has attributes. Committed to svn on r-forge; wi

Re: [R-sig-Geo] speed problem with %over% function

2012-02-29 Thread Colin Rundel
For this example at least you are severely handicapping rgeos as gIntersect needs to create all new points that result from the intersections which then need to be translated back to R from the geos representation. If you use something like the following you should see much better results: > sy

Re: [R-sig-Geo] speed problem with %over% function

2012-02-29 Thread Bastien.Ferland-Raymond
Thanks to Francis and Edzer for their responses. I been working on my problem for the last 2 days and I managed to get some very interesting information. I also prepared a reproducible example so people can try it. I've found 3 functions that do the thing I want: 1) %over% 2) overlay() (which

Re: [R-sig-Geo] speed problem with %over% function (Francis Markham)

2012-02-29 Thread gregor . hochschild
I made the same experience as Francis Markham. I recently used the over function to check in which polygon from a shapefile a large number of points fall, and it crashed because of memory requirements. I was running the function in the cloud with 12GB of RAM... Here are some details and below i

Re: [R-sig-Geo] speed problem with %over% function

2012-02-28 Thread Francis Markham
I've had examples in the past where using %over% from the sp package takes all available RAM (7GB) and several hours, while ArcGIS takes about 300MB and 5 minutes, so I would agree that there is plenty of room for improvement here. I'll try to give a reproducable example in the coming weeks when I

Re: [R-sig-Geo] speed problem with %over% function

2012-02-27 Thread Edzer Pebesma
Dear Bastien, the %over% method was primary written to work, and it seems that it did. It is hard to tell why ArcGIS is so much faster without access to its source code, and your data. Chances are that it uses a spatial index, and R does not. Talking about spatial indexes, you might want to try

[R-sig-Geo] speed problem with %over% function

2012-02-27 Thread Bastien.Ferland-Raymond
Dear geo-list, I'm just starting using R and the rgdal package to manage and create some shapes and I am hitting some speed issues while using the %over% function. R is significantly slower than ArcGIS. Here is what I'm doing (I've shorten the code to make it easier to follow, I don't expect i