On Mon, 2009-06-15 at 07:35 -0700, Michael wrote: > Yet what's the benefit of using Design Package?
Given that glm apparently can't operate within the memory limits of your PC setup on your particular data set/model, perhaps the question should "what do you have to loose in trying Design?" If lrm() also won't work, you might try glm.fit supplied with R and used as the work horse function behind glm. The reason I mention this is that the problem you might be seeing could be coming from the part of glm that builds the model matrices and the like rather than from the fitting part of glm.fit. You need to provide the design matrix yourself of course. 60000 x 20 doesn't sound like a such a large design matrix - perhaps there is something you are not telling us about the variables in C? Are some/all of the 19 predictors factors, with lots of levels? Maybe they shouldn't be but have ended up that way after being read in to R? You could do a traceback() straight after the error to see where the failure was, and maybe also debug glm debug(glm) to step through the code as you run your model to see where the code is bailing out. That will guide you to whether the failure is in the fitting part or the bit before glm.fit is called which is where the model (design) matrices are computed etc. Another alternative is to try the biglm package which can fit GLMs on huge data sets. HTH G > > Thanks! > > On Sun, Jun 14, 2009 at 8:04 PM, Frank E Harrell > Jr<f.harr...@vanderbilt.edu> wrote: > > Also it would be useful to compare glm with the lrm function in the Design > > package, for speed and memory use. > > > > Frank > > > > > > David Winsemius wrote: > >> > >> On Jun 14, 2009, at 9:06 PM, Michael wrote: > >> > >>> Hi all, > >>> > >>> I am getting the following error message: > >>> > >>>> mymodel = glm(response ~ . , family=binomial, data=C); > >>> > >>> Error: cannot allocate vector of size 734.2 Mb > >>> In addition: Warning messages: > >>> 1: In array(0, c(n, n), list(levs, levs)) : > >>> Reached total allocation of 1535Mb: see help(memory.size) > >>> 2: In array(0, c(n, n), list(levs, levs)) : > >>> Reached total allocation of 1535Mb: see help(memory.size) > >>> 3: In array(0, c(n, n), list(levs, levs)) : > >>> Reached total allocation of 1535Mb: see help(memory.size) > >>> 4: In array(0, c(n, n), list(levs, levs)) : > >>> Reached total allocation of 1535Mb: see help(memory.size) > >>> > >>> ----------- > >>> > >>> The data frame is 60000 x 20, > >>> > >>> is it too large for R? > >>> > >>> What shall I do? Will close all other softwares/applications help? My > >>> PC is Vista with 4GB memory. Thank you. > >> > >> It's certainly not too large for R. Have you looked at the R Windows FAQ > >> on the topic? > >> > >> > >> http://cran.r-project.org/bin/windows/base/rw-FAQ.html#There-seems-to-be-a-limit-on-the-memory-it-uses_0021 > >> > >> ... and perhaps: > >> > >> http://finzi.psych.upenn.edu/Rhelp08/2008-August/171649.html > >> > >> > >> David Winsemius, MD > >> Heritage Laboratories > >> West Hartford, CT > >> > >> ______________________________________________ > >> R-help@r-project.org mailing list > >> https://stat.ethz.ch/mailman/listinfo/r-help > >> PLEASE do read the posting guide > >> http://www.R-project.org/posting-guide.html > >> and provide commented, minimal, self-contained, reproducible code. > >> > > > > > > -- > > Frank E Harrell Jr Professor and Chair School of Medicine > > Department of Biostatistics Vanderbilt University > > > > ______________________________________________ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. -- %~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~% Dr. Gavin Simpson [t] +44 (0)20 7679 0522 ECRC, UCL Geography, [f] +44 (0)20 7679 0565 Pearson Building, [e] gavin.simpsonATNOSPAMucl.ac.uk Gower Street, London [w] http://www.ucl.ac.uk/~ucfagls/ UK. WC1E 6BT. [w] http://www.freshwaters.org.uk %~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~% ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.