On 08/12/2013 10:18 AM, Stackpole, Chris wrote:
Greetings, I have a user who is running an R program on two different
Linux systems. For the most part, they are very similar in terms of
hardware and 64bit OS. However, they perform significantly different.
Under one box the program uses upwards of 20GB of ram but fluctuates
around 15GB of ram and the job runs for a few hours. The second box
has even more memory available to it, however, the exact same program
with the exact same data set peaks at 7GB of ram but runs around 5GB
of ram and takes 3x longer to run the job!
I did some research, and from what I can tell R should just use as
much memory as it needs on Linux. So a lot of the "help" I found
online has been windows related information (eg: --max-mem-size ) and
not very useful to me. I looked at the ulimits and everything looks
like it should be correct (or at least it is comparable to the
ulimits on the system that is working correctly). I have also checked
other tidbits here and there but nothing seems to be of use. I also
checked that a single user can allocate large quantities of memory
(eg: Matlab and SAS both were able to allocate 20GB+ of memory) so I
don't think it is a user-restriction placed by the OS.
The only differences I have found between the two boxes that really
stands out is that the system that works runs RHEL proper and has R
compiled but the one that doesn't allocate all of the memory was
installed via EPEL RPM on CentOS. Compiling R on the CentOS system is
on the try-this list, but before I spend that time trying to compile
I thought I would ask a few questions.
1) Anyone know why I might be seeing this strange behavior? 5-7GB of
ram is clearly over any 32bit limitation so I don't think it has
anything to do with that. It could be a RHEL vs CentOS thing, but
that seems very strange to me.
2) When I compile from source to test this, is there a specific
option I should pass to ensure max usage?
Thank you.
Chris Stackpole
What does "ulimit -a" report on both of these machines?
--
Kevin E. Thorpe
Head of Biostatistics, Applied Health Research Centre (AHRC)
Li Ka Shing Knowledge Institute of St. Michael's
Assistant Professor, Dalla Lana School of Public Health
University of Toronto
email: kevin.tho...@utoronto.ca Tel: 416.864.5776 Fax: 416.864.3016
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
