Hi all,
I am currently building an application based on R 1.7.1 (+ compiled
C/C++ code + MySql + VB). I am building this application to work on 2
different platforms (Windows XP Professional (500mb memory) and Windows
NT 4.0 with service pack 6 (1gb memory)). This is a very memory
intensive application performing sophisticated operations on "large"
matrices (typically 5000x1500 matrices).
I have run into some issues regarding the way R handles its memory,
especially on NT. In particular, R does not seem able to recollect some
of the memory used following the creation and manipulation of large data
objects. For example, I have a function which receives a (large)
numeric matrix, matches against more data (maybe imported from MySql)
and returns a large list structure for further analysis. A typical call
may look like this .
> myInputData <- matrix(sample(1:100, 7500000, T), nrow=5000)
> myPortfolio <- createPortfolio(myInputData)
It seems I can only repeat this code process 2/3 times before I have to
restart R (to get the memory back). I use the same object names
(myInputData and myPortfolio) each time, so I am not create more large
objects ..
I think the problems I have are illustrated with the following example
from a small R session .
> # Memory usage for Rui process = 19,800
> testData <- matrix(rnorm(10000000), 1000) # Create big matrix
> # Memory usage for Rgui process = 254,550k
> rm(testData)
> # Memory usage for Rgui process = 254,550k
> gc()
used (Mb) gc trigger (Mb)
Ncells 369277 9.9 667722 17.9
Vcells 87650 0.7 24286664 185.3
> # Memory usage for Rgui process = 20,200k
In the above code, R cannot recollect all memory used, so the memory
usage increases from 19.8k to 20.2. However, the following example is
more typical of the environments I use .
> # Memory 128,100k
> myTestData <- matrix(rnorm(10000000), 1000)
> # Memory 357,272k
> rm(myTestData)
> # Memory 357,272k
> gc()
used (Mb) gc trigger (Mb)
Ncells 478197 12.8 818163 21.9
Vcells 9309525 71.1 31670210 241.7
> # Memory 279,152k
Here, the memory usage increases from 128.1k to 279.1k
Could anyone point out what I could do to rectify this (if anything), or
generally what strategy I could take to improve this?
Many thanks,
Rich.
Mango Solutions
Tel : (01628) 418134
Mob : (07967) 808091
[[alternative HTML version deleted]]
______________________________________________
[EMAIL PROTECTED] mailing list
https://www.stat.math.ethz.ch/mailman/listinfo/r-help
