On Fri, 10 Aug 2007, Monica Pisica wrote:
Thanks! I will look into ...
I have 4 GB RAM, and i was monitoring the memory with Windows task
manager so i was looking how R "gets" more and more memory allocation
from less than 100Mb to .... 1500Mb .....
Then you are almost certainly fragmenting the address space.
We still don't know your OS and whether you have enabled the /3GB switch
(if relevant to that version of Windows). Most versions of Windows have
a 2Gb address space, but some can be as high as 4Gb (Vista 64 which I use
is one: the details are in the rw-FAQ for the latest versions of R, e.g.
R-patched and R-devel). That factor of 2 can make a big difference.
My initial tables are between 30 to 80 Mb and the resulting tables that
incorporate the initial tables plus PCA and kmeans results are inbetween
50 to 200MB or thereabouts!
And yes, i don't really care about memory allocation in detail - what i
want is to free that memory after every cycle ;-)
Although, after i didn't do anything in R and it was idle for more than
30 min. the memory allocation according to Task manager dropped to 15 Mb
..... which is good - but i cannot wait inbetween cycles half an hour
though .....
Calling gc() will reduce the memory allocation, but that is not the point.
You can have 15Mb allocated and still not a 50Mb hole in the address
space (although that would be extremely unlucky, not having several 200Mb
holes is quite likely).
Again thanks,
Monica> Date: Fri, 10 Aug 2007 18:28:07 +0100> From:
[EMAIL PROTECTED]> To: [EMAIL PROTECTED]> CC:
r-help@stat.math.ethz.ch> Subject: Re: [R] Cleaning up the memory> > On
Fri, 10 Aug 2007, Monica Pisica wrote:> > >> > Hi,> >> > I have 4 huge
tables on which i want to do a PCA analysis and a kmean > > clustering.
If i run each table individually i have no problems, but if > > i want
to run it in a for loop i exceed the memory alocation after the > >
second table, even if i save the results as a csv table and i clean up >
> all the big objects with rm command. To me it seems that even if i
don't > > have the objects anymore, the memory these objects used to
occupy is not > > cleared. Is there any way to clear up the memory as
well? I don't want > > to close R and start it up again. Also i am
running R under Windows.> > See ?gc, which does the clearing.> >
However, unless you study the memory allocation in detail (which you >
cannot do from R code), you don't actually know that this is the
problem. > More likely is that you have fragmentation of your 32-bit
address space: > see ?"Memory-limits".> > Without any idea what memory
you have and what 'huge' means, we can only > make wild guesses. It
might be worth raising the memory limit (the > --max-mem-size flag).> >
>> > thanks,> >> > Monica> >
_________________________________________________________________> >
[[trailing spam removed]]> >> > [[alternative HTML version deleted]]> >>
> ______________________________________________> >
R-help@stat.math.ethz.ch mailing list> >
https://stat.ethz.ch/mailman/listinfo/r-help> > PLEASE do read the
posting guide http://www.R-project.org/posting-guide.html> > and provide
commented, minimal, self-contained, reproducible code.> >> > -- > Brian
D. Ripley, [EMAIL PROTECTED]> Professor of Applied Statistics,
http://www.stats.ox.ac.uk/~ripley/> University of Oxford, Tel: +44 1865
272861 (self)> 1 South Parks Road, +44 1865 272866 (PA)> Oxford OX1 3TG,
UK Fax: +44 1865 272595
_________________________________________________________________
Messenger Café ? open for fun 24/7. Hot games, cool activities served
daily. Visit now. http://cafemessenger.com?ocid=TXT_TAGLM_AugWLtagline
--
Brian D. Ripley, [EMAIL PROTECTED]
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UK Fax: +44 1865 272595
______________________________________________
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
