[R-sig-Geo] Raster memory problem
Dear list, I am using large raster bricks for my analyses and I am running out of memory. I get the following message: Error: cannot allocate vector of size 231.0 Mb and this whether I do all my brick manipulations by loading the brick in R's memory or not (i.e., whether I use the option filename=name of my file). The strangest characteristic of the problem I have is that I get an error message when I handle smaller brick and not when handle larger brick. Here is a simple example: library(raster) b0 - brick(nrows=79, ncols=90, nl=1000) b0[] - rnorm(ncell(b0)*nlayers(b0)) writeRaster(b0, filename=b0,overwrite=TRUE) s1 - stack(b0,b0,b0,b0) # inMemory(s1) s2 - stack(b0,b0,b0,b0,b0) b1 - brick(s1, filename=b1, overwrite=TRUE) # b1 gives an error of the type: # Error: cannot allocate vector of size 217.0 Mb b2 - brick(s2, filename=b2, overwrite=TRUE) # b2 does not give me an error # And this even if b1 is smaller than b2 Platform info: R version 2.15.2 (2012-10-26) -- Trick or Treat Copyright (C) 2012 The R Foundation for Statistical Computing ISBN 3-900051-07-0 Platform: i386-w64-mingw32/i386 (32-bit) On windows 32-bit I have clean up my computer, defragmented it, increase my virtual memory, etc, and I still get an error. Any help will be greatly appreciated, Marie [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Raster memory problem
2012/12/12 Marie Auger-Methe marie.augerme...@gmail.com Dear list, I am using large raster bricks for my analyses and I am running out of memory. I get the following message: Error: cannot allocate vector of size 231.0 Mb and this whether I do all my brick manipulations by loading the brick in R's memory or not (i.e., whether I use the option filename=name of my file). The strangest characteristic of the problem I have is that I get an error message when I handle smaller brick and not when handle larger brick. I have clean up my computer, defragmented it, increase my virtual memory, etc, and I still get an error. Any help will be greatly appreciated, Marie Welcome to the band called Three of us (can't handle such big rasters) ;-) -raffaele [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Raster memory problem
Take a look at setOptions to force disk to disk. raster will guess if it can do the operation in memory but occasionally underestimates the memory available or required. Sent from my iPhone On 12/12/2012, at 15:20, Raffaele Morelli raffaele.more...@gmail.com wrote: 2012/12/12 Marie Auger-Methe marie.augerme...@gmail.com Dear list, I am using large raster bricks for my analyses and I am running out of memory. I get the following message: Error: cannot allocate vector of size 231.0 Mb and this whether I do all my brick manipulations by loading the brick in R's memory or not (i.e., whether I use the option filename=name of my file). The strangest characteristic of the problem I have is that I get an error message when I handle smaller brick and not when handle larger brick. I have clean up my computer, defragmented it, increase my virtual memory, etc, and I still get an error. Any help will be greatly appreciated, Marie Welcome to the band called Three of us (can't handle such big rasters) ;-) -raffaele [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Raster memory problem
Marie, It seems you are manipulating large amounts of data in RAM with 32-bit R --- use 64 bit if you can. As Jacob indicated, you can use options like rasterOptions(maxmemory=1e+07, chunksize=1e+06). However, you might also have been able to avoid memory problems by using the more direct b1 - writeRaster(s1, filename=b1, overwrite=TRUE) instead of: b1 - brick(s1, filename=b1, overwrite=TRUE) It would probably also be better to do: b0 - writeRaster(b0, filename=b0,overwrite=TRUE) (values of b0 now on disk) instead of writeRaster(b0, filename=b0,overwrite=TRUE) (R object b0 did not change; all values still in RAM) Best, Robert On Wed, Dec 12, 2012 at 1:14 PM, Marie Auger-Methe marie.augerme...@gmail.com wrote: Thanks for the replies. changing the raster options as suggested by Jacob fixed the problem. I lowered the maxmemory: rasterOptions(maxmemory=**9000) rather than changing the todisk option. It worked like a charm. Thanks! Marie On 12/12/2012 8:24 PM, Jacob van Etten wrote: Take a look at setOptions to force disk to disk. raster will guess if it can do the operation in memory but occasionally underestimates the memory available or required. Sent from my iPhone On 12/12/2012, at 15:20, Raffaele Morelli raffaele.more...@gmail.com wrote: 2012/12/12 Marie Auger-Methe marie.augerme...@gmail.com Dear list, I am using large raster bricks for my analyses and I am running out of memory. I get the following message: Error: cannot allocate vector of size 231.0 Mb and this whether I do all my brick manipulations by loading the brick in R's memory or not (i.e., whether I use the option filename=name of my file). The strangest characteristic of the problem I have is that I get an error message when I handle smaller brick and not when handle larger brick. I have clean up my computer, defragmented it, increase my virtual memory, etc, and I still get an error. Any help will be greatly appreciated, Marie Welcome to the band called Three of us (can't handle such big rasters) ;-) -raffaele [[alternative HTML version deleted]] __**_ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/**listinfo/r-sig-geohttps://stat.ethz.ch/mailman/listinfo/r-sig-geo __**_ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/**listinfo/r-sig-geohttps://stat.ethz.ch/mailman/listinfo/r-sig-geo __**_ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/**listinfo/r-sig-geohttps://stat.ethz.ch/mailman/listinfo/r-sig-geo [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
