These days in GIS on may have to manipulate big datasets or arrays. What is the best way of tuning the R memory ? or how can we not worry about the memory usage i.e. could it that if it is needed it will swap on the disk (the system I suppose) That means declaring even more RAM than existing ...
Here I am on WINDOWS I have a 4Gb my aim was to have an array of dim 298249 12 10 22 but that's 2.9Gb Here is some tries to find the size I could have : First of all I did: > memory.limit(size=3500) NULL > memory.limit(size=NA)/2^20 [1] 3500 > memory.size(max=TRUE)/2^20 [1] 341.125 >ZZ=array(rep(0,round(298249/16)*12*10*22),c(round(298249/16),12,10,22)) > gc() used (Mb) gc trigger (Mb) max used (Mb) Ncells 835490 22.4 1368491 36.6 954139 25.5 Vcells 90663806 691.8 154550742 1179.2 139876061 1067.2 > > 298249 *12*10*22 [1] 787377360 > dd=as.vector(rep(0,298249 *12*10*22)) Error: cannot allocate vector of size 2.9 Gb In addition: Warning messages: 1: Reached total allocation of 3500Mb: see help(memory.size) > dd=as.vector(rep(0,round(298249/4) *12*10*22)) Error: cannot allocate vector of size 750.9 Mb > dd=as.vector(rep(0,round(298249/2) *12*10*22)) Error: cannot allocate vector of size 1.5 Gb > dd=as.vector(rep(0,round(298249/9) *12*10*22)) Error: cannot allocate vector of size 333.7 Mb > dd=as.vector(rep(0,round(298249/10) *12*10*22)) Error: cannot allocate vector of size 300.4 Mb > object.size(ZZ) [1] 393698040 > object.size(ZZ)/2^20 [1] 375.4597 > gc() used (Mb) gc trigger (Mb) max used (Mb) Ncells 835535 22.4 1368491 36.6 954139 25.5 Vcells 90663830 691.8 162358279 1238.7 139876061 1067.2 > rm(ZZ) > gc() used (Mb) gc trigger (Mb) max used (Mb) Ncells 835622 22.4 1368491 36.6 954139 25.5 Vcells 41451603 316.3 129886623 991.0 139876061 1067.2 > dd=as.vector(rep(0,round(298249/10) *12*10*22)) Error: cannot allocate vector of size 600.7 Mb > It is also strange that once a dd needed 300.4Mb and then 600.7Mb (?) as also I made some room in removing ZZ? I had already modiied the starting of windows by putting [operating systems] multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows Server 2003, Standard" /noexecute=optout /fastdetect /redirect /5GB in the boot.ini which I don't really know if it took into account as the limit is greater than the physical RAM of 4GB. ...? would it be easier using Linux ? Thanks -- Dr Didier Leibovici http://www.nottingham.ac.uk/cgs/leibovici.shtml Centre for Geospatial Science Sir Clive Granger Building University of Nottingham, University Park Nottingham NG7 2RD, UK Tel: +44 - (0)115 84 66058 Fax: +44 (0)115 95 15249 This message has been checked for viruses but the contents of an attachment may still contain software viruses, which could damage your computer system: you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation. _______________________________________________ R-sig-Geo mailing list R-sig-Geo@stat.math.ethz.ch https://stat.ethz.ch/mailman/listinfo/r-sig-geo