Hello R List, I'm trying to run a linear mixed effects analysis on fairly large datasets with lmer (from the lme4 package) on a 32-bit Windows machine running XP with 3 GB of RAM. It's not working. (details below)
I've researched the ff and bigmemory packages, but it appears they won't handle the mixed mode dataset I'm analyzing. It has some character fields for the categorical variables. It's also not clear a linear mixed effect regression can be run with those packages. Can anyone point me to a lme solution on larger datasets that can address the 32-bit memory limitations? The smallest dataframe I'm trying to run is about 120,000 observations and 7 variables, but I'd rather run a for loop script on 620K+ observations. I've seen several recommendations (Kabacoff in 'R in Action' and others) that when possible, run R in a 64-bit build. Problem is I'm on a deadline, and procuring a new computer takes time, and approvals up the food chain. Suggestions? Is 64-bit my only option? Here is the information about what I'm running: R version 2.14.0 (2011-10-31) Platform: i386-pc-mingw32/i386 (32-bit) locale: [1] LC_COLLATE=English_United States.1252 [2] LC_CTYPE=English_United States.1252 [3] LC_MONETARY=English_United States.1252 [4] LC_NUMERIC=C [5] LC_TIME=English_United States.1252 attached base packages: [1] stats graphics grDevices utils datasets methods [7] base other attached packages: [1] lme4_0.999999-0 Matrix_1.0-5 lattice_0.20-10 [4] RODBC_1.3-6 loaded via a namespace (and not attached): [1] grid_2.14.0 nlme_3.1-104 stats4_2.14.0 tools_2.14.0 I run this code: > install.packages("lme4") > library(lme4) > math07g4 <- sqlQuery(conn, "select ssid, ss_chg, campus2, district_id, > pblack, pfreelnch, pmob FROM codemob0607ma WHERE grade2 = 4") > attach(math07g4) > fit07ma4 <- lmer(ss_chg ~ 1 + factor(campus2) + factor(district_id) + pblack > + pfreelnch + pmob + (1 | campus2) + (1 | district_id), data=math07g4) And I get this: Error: cannot allocate vector of size 2.5 Gb In addition: Warning messages: 1: In model.matrix.default(mt, mf, contrasts) : Reached total allocation of 2187Mb: see help(memory.size) 2: In model.matrix.default(mt, mf, contrasts) : Reached total allocation of 2187Mb: see help(memory.size) 3: In model.matrix.default(mt, mf, contrasts) : Reached total allocation of 2187Mb: see help(memory.size) 4: In model.matrix.default(mt, mf, contrasts) : Reached total allocation of 2187Mb: see help(memory.size) Thanks in advance, Ken Ken McCall | Database reporter Dayton Daily News<http://www.daytondailynews.com/> | 1611 S. Main St. Dayton, Ohio 45409 Ph: 937-000-000 Cell: 937-225-2393 Email: ken.mcc...@coxinc.com<mailto:ken.mcc...@coxinc.com> COMPLETE. IN-DEPTH. DEPENDABLE. [[alternative HTML version deleted]] ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.