You could stick to the native raster format (grd), in which case calc()
writes to your final file directly.
Of course that doesn't change the file size issue, but saves you the
translation step to geotiff.
For the file size you could consider restricting the datatype to integer
(see ?dataType).
If I have reflectance data [0,1] for example; I scale them by a factor
10000 and then save as "INT4U" or "INT2U" (depending on your maximally
expected value range), or "INT4S" or "INT2S" if you have negative
values. That can bring down the filesize quite a bit, while retaining
most of the relevant precission (beware: remaining decimal places will
be cut-off,)
e.g. calc(..., function(x) { yourcalculations(x) * 10000 }, datatype =
"INT4S")
pro tip: the argument in calc() (or more precisely in writeRaster()) is
called datatype, not to be confused with the stand-alone function
dataType() with a capital T. That one has bitten me many times, because
due to the "..." argument there will be no warning if you mistype it ;-)
On 06.04.2017 19:27, Gregovich, Dave P (DFG) wrote:
Hi,
I am performing a math operation on a stack of large rasters. The code below
uses smaller files for illustration and reproducibility.
Any alternative way of performing this task that does not create huge temporary
files, and perhaps cuts down on processing time, would be greatly appreciated.
The 'calc' process creates a couple of temporary raster files that are huge.
The first one is 142 GB, and I don't have hard drive space for that one and the
second one that begins writing during the process.
Thanks kindly for any advice!
Dave.
#create raster stack and coefficients...
library(raster)
mod.coefs <- rnorm(10)
s <- stack()
r <- raster(nrow = 100, ncol = 100)
#actual rasters I am working with are about 40000, pixels square, with each
GeoTiff raster in the stack taking about 2.5 GB on disk
for(i in 1:10){
r[] <- rnorm(10000)
s <- addLayer(s, r)
}
#attempt to perform raster math...
out.file <- 'C:/ out.rast.tif'
out.rast <- calc(s, function(x){exp(sum(mod.coefs * x))}, filename = out.file)
#at this point, the temporary files in the
\AppData\Local\Temp\RtmpqQYzfS\raster folder eventually become quite large,
#with one .GRI file reaching 142 GB, and another now growing to 8 GB before I
ended the process
#the file 'out.file' has not been created at that point.
#_____________end____________________________________________________________________
[[alternative HTML version deleted]]
_______________________________________________
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
--
Benjamin Leutner M.Sc.
PhD candidate
Department of Remote Sensing
University of Wuerzburg
Campus Hubland Nord 86
97074 Wuerzburg, Germany
Tel: +49-(0)931-31 89594
Email: benjamin.leut...@uni-wuerzburg.de
Web: http://www.fernerkundung.uni-wuerzburg.de
_______________________________________________
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
