Are the two models (f1 and f2) actually nested? Aside from that, it is strange that the output is exactly the same after you used REML=FALSE. The log likelihoods should have changed.
Best, -- Wolfgang Viechtbauer Department of Methodology and Statistics School for Public Health and Primary Care University of Maastricht, The Netherlands http://www.wvbauer.com/ ----Original Message---- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Matt Killingsworth Sent: Friday, September 04, 2009 22:29 To: Bert Gunter Cc: r-help@r-project.org Subject: Re: [R] Using anova(f1, f2) to compare lmer models yields seemingly erroneous Chisq = 0, p = 1 > Hi Bert, > > Thank you for your note! I tried changing the REML default, and it > still produces the same result (see below). Is that what you meant > for me to try? > > Incidentally, I am using lmer() not lme() > > ### ORIGINAL ### >> f1 <- (lmer(outcome ~ predictor.1 + (1 | person), data=i)) >> f2 <- (lmer(outcome ~ predictor.2 + (1 | person), data=i)) anova(f1, >> f2) > > Data: i > Models: > f1: outcome ~ predictor.1 + (1 | person) > f2: outcome ~ predictor.2 + (1 | person) > Df AIC BIC logLik Chisq Chi Df Pr(>Chisq) > f1 6 45443 45489 -22715 > f2 25 47317 47511 -23633 0 19 1 > > ### DO NOT USE REML ### >> f1 <- (lmer(outcome ~ predictor.1 + (1 | person), data=i, REML = >> FALSE)) f2 <- (lmer(outcome ~ predictor.2 + (1 | person), data=i, >> REML = FALSE)) anova(f1, f2) > > Data: i > Models: > f1: outcome ~ predictor.1 + (1 | person) > f2: outcome ~ predictor.2 + (1 | person) > Df AIC BIC logLik Chisq Chi Df Pr(>Chisq) > f1 6 45443 45489 -22715 > f2 25 47317 47511 -23633 0 19 1 > > > On Fri, Sep 4, 2009 at 4:18 PM, Bert Gunter <gunter.ber...@gene.com> > wrote: > >> My guess would be: >> >> "Likelihood comparisons are not meaningful for objects fit using >> restricted maximum likelihood and with different fixed effects. " >> >> (from ?anova.lme in the nlme package). >> >> Are you using the REML = TRUE default? >> >> Bert Gunter >> Genentech Nonclinical Statistics >> >> -----Original Message----- >> From: r-help-boun...@r-project.org >> [mailto:r-help-boun...@r-project.org] >> On >> Behalf Of rapton >> Sent: Friday, September 04, 2009 9:10 AM >> To: r-help@r-project.org >> Subject: [R] Using anova(f1, f2) to compare lmer models yields >> seemingly erroneous Chisq = 0, p = 1 >> >> >> Hello, >> >> I am using R to analyze a large multilevel data set, using >> lmer() to model my data, and using anova() to compare the fit of >> various models. When I run two models, the output of each model is >> generated correctly as far as I can tell (e.g. summary(f1) and >> summary(f2) for the multilevel model output look perfectly >> reasonable), and in this case (see >> below) predictor.1 explains vastly more variance in outcome than >> predictor.2 (R2 = 15% vs. 5% in OLS regression, with very large N). >> What I am utterly puzzled by is that when I run an anova comparing >> the two multilevel model fits, the Chisq comes back as 0, with p = >> 1. I am pretty sure that fit #1 (f1) is a much better predictor of >> the outcome than f2, which is reflected in the AIC, BIC , and logLik >> values. Why might anova be giving me this curious output? How can >> I fix it? I am sure I am making a dumb error somewhere, but I >> cannot figure out what it is. Any help or suggestions would be >> greatly appreciated! >> >> -Matt >> >> >>> f1 <- (lmer(outcome ~ predictor.1 + (1 | person), data=i)) f2 <- >>> (lmer(outcome ~ predictor.2 + (1 | person), data=i)) anova(f1, f2) >> >> Data: i >> Models: >> f1: outcome ~ predictor.1 + (1 | person) >> f2: outcome ~ predictor.2 + (1 | person) >> Df AIC BIC logLik Chisq Chi Df Pr(>Chisq) >> f1 6 45443 45489 -22715 >> f2 25 47317 47511 -23633 0 19 1 >> -- ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.