Dear R-sig-geo users, I’m using the raster package (mainly the calc function) to calculate new indices for a large MODIS raster that I am using. The code works perfectly when run over small rasters (e.g. 100 MB), but I now want to run the process on a 60 GB raster. My computer has 20 GB of RAM and about 500 GB of storage space.
What is the most efficient and easiest way to run raster calc functions on extremely large files? I’ve been reading about how to get past memory and storage limitations in R, but I would love advice on what the easiest and fastest option might be based on your experience. Some possibilities based on my reading are: 1) Somehow use the ff package with the code I’ve written using the raster package on the full 60 GB file 2) Using a form of tiling where I break the 60 GB file into smaller sub-files and run the raster code on each tile using parallels or possibly multiple computers. I then will piece back together all of these sub-files to form one large file when they are all done running. Here is an example of the code that I need to run over the large file (this is only 1 of the 15 total steps I will eventually run) and a small example raster is downloadable using the dropbox link below. *Example code:* library(raster) library(sp) library(rgdal) saevi<-brick('D:/Meha_Files/SAFull/SAFull_0013_Final') spl60fun<-function(x){ x[x==255]<-NA if (sum(is.na(x))>20) {spline<-rep(NA,302) return(spline)} else if (is.na(x[1])) {spline<-rep(NA,302) return(spline)} else if (is.na(x[302])) {spline<-rep(NA,302) return(spline)} else {x1<-c(1:length(x)) y1<-x intx<-approx(x1,y1,xout=1:length(x1))$y spline<-smooth.spline(intx,df=60)$y return(spline)} } sasp<-calc(saevi,spl60fun) writeRaster(sasp,"sasp",format="GTiff") *Example raster:* https://www.dropbox.com/s/hh5cl4r02g7havo/Example_Raster.zip Thanks! Meha -- View this message in context: http://r-sig-geo.2731867.n2.nabble.com/Analyzing-Large-Raster-Files-in-R-tp7584531.html Sent from the R-sig-geo mailing list archive at Nabble.com. _______________________________________________ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo