On Sat, 6 Dec 2003, Prof Brian Ripley wrote: > I think you misunderstand how R uses memory. gc() does not free up all > the memory used for the objects it frees, and repeated calls will free > more. Don't speculate about how memory management works: do your > homework!
Are you saying that consecutive calls to gc() will free more memory than a single call, or am I misunderstanding? Reading ?gc and ?Memory I don't see anything about this mentioned. Where should I be looking to find more comprehensive info on R's memory management?? I'm not writing any packages, just would like to have a better handle on efficiently using memory as it is usually the limiting factor with R. FYI, I'm running R1.8.1 and RedHat9 on a P4 with 2GB of RAM in case there is any platform specific info that may be applicable. Thanks, Doug Grove Statistical Research Associate Fred Hutchinson Cancer Research Center > In any case, you are using an outdated version of R, and your first > course of action should be to compile up R-devel and try that, as there > has been improvements to memory management under Windows. You could also > try compiling using the native malloc (and that *is* described in the > INSTALL file) as that has different compromises. > > > On Sat, 6 Dec 2003, Richard Pugh wrote: > > > Hi all, > > > > I am currently building an application based on R 1.7.1 (+ compiled > > C/C++ code + MySql + VB). I am building this application to work on 2 > > different platforms (Windows XP Professional (500mb memory) and Windows > > NT 4.0 with service pack 6 (1gb memory)). This is a very memory > > intensive application performing sophisticated operations on "large" > > matrices (typically 5000x1500 matrices). > > > > I have run into some issues regarding the way R handles its memory, > > especially on NT. In particular, R does not seem able to recollect some > > of the memory used following the creation and manipulation of large data > > objects. For example, I have a function which receives a (large) > > numeric matrix, matches against more data (maybe imported from MySql) > > and returns a large list structure for further analysis. A typical call > > may look like this . > > > > > myInputData <- matrix(sample(1:100, 7500000, T), nrow=5000) > > > myPortfolio <- createPortfolio(myInputData) > > > > It seems I can only repeat this code process 2/3 times before I have to > > restart R (to get the memory back). I use the same object names > > (myInputData and myPortfolio) each time, so I am not create more large > > objects .. > > > > I think the problems I have are illustrated with the following example > > from a small R session . > > > > > # Memory usage for Rui process = 19,800 > > > testData <- matrix(rnorm(10000000), 1000) # Create big matrix > > > # Memory usage for Rgui process = 254,550k > > > rm(testData) > > > # Memory usage for Rgui process = 254,550k > > > gc() > > used (Mb) gc trigger (Mb) > > Ncells 369277 9.9 667722 17.9 > > Vcells 87650 0.7 24286664 185.3 > > > # Memory usage for Rgui process = 20,200k > > > > In the above code, R cannot recollect all memory used, so the memory > > usage increases from 19.8k to 20.2. However, the following example is > > more typical of the environments I use . > > > > > # Memory 128,100k > > > myTestData <- matrix(rnorm(10000000), 1000) > > > # Memory 357,272k > > > rm(myTestData) > > > # Memory 357,272k > > > gc() > > used (Mb) gc trigger (Mb) > > Ncells 478197 12.8 818163 21.9 > > Vcells 9309525 71.1 31670210 241.7 > > > # Memory 279,152k > > > > Here, the memory usage increases from 128.1k to 279.1k > > > > Could anyone point out what I could do to rectify this (if anything), or > > generally what strategy I could take to improve this? > > > > Many thanks, > > Rich. > > > > Mango Solutions > > Tel : (01628) 418134 > > Mob : (07967) 808091 > > > > > > [[alternative HTML version deleted]] > > > > ______________________________________________ > > [EMAIL PROTECTED] mailing list > > https://www.stat.math.ethz.ch/mailman/listinfo/r-help > > > > > > -- > Brian D. Ripley, [EMAIL PROTECTED] > Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ > University of Oxford, Tel: +44 1865 272861 (self) > 1 South Parks Road, +44 1865 272866 (PA) > Oxford OX1 3TG, UK Fax: +44 1865 272595 > > ______________________________________________ > [EMAIL PROTECTED] mailing list > https://www.stat.math.ethz.ch/mailman/listinfo/r-help > ______________________________________________ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help