Hi Peter, searched old mail archive and found this topic had been discussed before. The previous discussion was around a situation where there was a very large sample size involved so even a small effect still showed up as significant even with low R square of the model.
In my case, the sample size is 72, the significance of "group" effect is due to large effect relative to its standard error: obj<-lm(y~age+sex+school+group,dat) summary(obj) Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 169.8634 13.4678 12.613 <2e-16 age -0.3737 0.2762 -1.353 0.1805 sexM 2.1137 8.6585 0.244 0.8079 schoolS2 4.1711 8.1811 0.510 0.6118 groupG2 -20.8944 10.2807 -2.032 0.0461 Residual standard error: 32.13 on 67 degrees of freedom Multiple R-squared: 0.1732, Adjusted R-squared: 0.1238 F-statistic: 3.509 on 4 and 67 DF, p-value: 0.01163 So R-squared is quite low (0.17), what's your opinion on the argument that the significant coefficient for group is not trustworthy because the model variance was not sufficiently accounted for, and if additional factors could be identified and included in the model, that might changed the effect of group from significant to insignificant. Many thanks for sharing your thoughts. John ________________________________ To: peter dalgaard <pda...@gmail.com> Cc: "r-help@r-project.org" <r-help@r-project.org> Sent: Tuesday, May 8, 2012 1:45 PM Subject: Re: [R] low R square value from ANCOVA model Thanks again Peter. What about the argument that because low R square (e.g. R^2=0.2) indicated the model variance was not sufficiently explained by the factors in the model, there might be additional factors that should be identified and included in the model. And If these additional factors were indeed included, it might change the significance for the factor of interest that previously showed significant coefficient. In other word, if R square is low, the significant coefficient observed is not trustworthy. What's your opinion on this argument? Many thanks! John ________________________________ From: peter dalgaard <pda...@gmail.com> Cc: "r-help@r-project.org" <r-help@r-project.org> Sent: Monday, May 7, 2012 11:43 PM Subject: Re: [R] low R square value from ANCOVA model On May 8, 2012, at 08:34 , array chip wrote: > Thank you Peter, so if I observe a significant coefficient, that significance > still holds because the standard error of the coefficient has taken the > residual error (which is large because large R square) into account, am I > correct? In essence, yes. One might quibble over the use of "large because", but it's not important for the main point. -pd > John > From: peter dalgaard <pda...@gmail.com> > Cc: "r-help@r-project.org" <r-help@r-project.org> > Sent: Monday, May 7, 2012 11:07 PM > Subject: Re: [R] low R square value from ANCOVA model > > > On May 8, 2012, at 05:10 , array chip wrote: > > > Hi, what does a low R-square value from an ANCOVA model mean? For example, > > if the R square from the model is about 0.2, does this mean the results > > should NOT be trusted? I checked the residuals of the model, it looked > > fine... > > It just means that your model has low predictive power (at the individual > level). I.e. the noise (error) part of the model is large relative to the > signal (systematic part). Statistical inferences are not compromised by that, > except of course that large error variation is reflected in large standard > errors of estimated regression coefficients. > > > > > Thanks for any suggestion. > > > > John > > [[alternative HTML version deleted]] > > > > ______________________________________________ > > R-help@r-project.org mailing list > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > > and provide commented, minimal, self-contained, reproducible code. > > -- > Peter Dalgaard, Professor, > Center for Statistics, Copenhagen Business School > Solbjerg Plads 3, 2000 Frederiksberg, Denmark > Phone: (+45)38153501 > Email: pd....@cbs.dk Priv: pda...@gmail.com > > > > > > > > > > -- Peter Dalgaard, Professor Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Email: pd....@cbs.dk Priv: pda...@gmail.com [[alternative HTML version deleted]]
______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
