I think you misunderstand how R uses memory. gc() does not free up all the memory used for the objects it frees, and repeated calls will free more. Don't speculate about how memory management works: do your homework!
In any case, you are using an outdated version of R, and your first course of action should be to compile up R-devel and try that, as there has been improvements to memory management under Windows. You could also try compiling using the native malloc (and that *is* described in the INSTALL file) as that has different compromises. On Sat, 6 Dec 2003, Richard Pugh wrote: > Hi all, > > I am currently building an application based on R 1.7.1 (+ compiled > C/C++ code + MySql + VB). I am building this application to work on 2 > different platforms (Windows XP Professional (500mb memory) and Windows > NT 4.0 with service pack 6 (1gb memory)). This is a very memory > intensive application performing sophisticated operations on "large" > matrices (typically 5000x1500 matrices). > > I have run into some issues regarding the way R handles its memory, > especially on NT. In particular, R does not seem able to recollect some > of the memory used following the creation and manipulation of large data > objects. For example, I have a function which receives a (large) > numeric matrix, matches against more data (maybe imported from MySql) > and returns a large list structure for further analysis. A typical call > may look like this . > > > myInputData <- matrix(sample(1:100, 7500000, T), nrow=5000) > > myPortfolio <- createPortfolio(myInputData) > > It seems I can only repeat this code process 2/3 times before I have to > restart R (to get the memory back). I use the same object names > (myInputData and myPortfolio) each time, so I am not create more large > objects .. > > I think the problems I have are illustrated with the following example > from a small R session . > > > # Memory usage for Rui process = 19,800 > > testData <- matrix(rnorm(10000000), 1000) # Create big matrix > > # Memory usage for Rgui process = 254,550k > > rm(testData) > > # Memory usage for Rgui process = 254,550k > > gc() > used (Mb) gc trigger (Mb) > Ncells 369277 9.9 667722 17.9 > Vcells 87650 0.7 24286664 185.3 > > # Memory usage for Rgui process = 20,200k > > In the above code, R cannot recollect all memory used, so the memory > usage increases from 19.8k to 20.2. However, the following example is > more typical of the environments I use . > > > # Memory 128,100k > > myTestData <- matrix(rnorm(10000000), 1000) > > # Memory 357,272k > > rm(myTestData) > > # Memory 357,272k > > gc() > used (Mb) gc trigger (Mb) > Ncells 478197 12.8 818163 21.9 > Vcells 9309525 71.1 31670210 241.7 > > # Memory 279,152k > > Here, the memory usage increases from 128.1k to 279.1k > > Could anyone point out what I could do to rectify this (if anything), or > generally what strategy I could take to improve this? > > Many thanks, > Rich. > > Mango Solutions > Tel : (01628) 418134 > Mob : (07967) 808091 > > > [[alternative HTML version deleted]] > > ______________________________________________ > [EMAIL PROTECTED] mailing list > https://www.stat.math.ethz.ch/mailman/listinfo/r-help > > -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595 ______________________________________________ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help