Re: [R-sig-eco] ICC confidence intervals and power analysis for random effects in lmer?
On 04/12/2012 03:50 AM, Bradley Carlson wrote: I'm performing an analysis of behavioral variation among individual tadpoles, using individual ID as a random effect and time as a continuous fixed covariate in the lmer function in lmer4 package. I'm really interested in making inferences about the random effect (i.e. the extent of variation among individuals). I'd like to do two things that I can't seem to find straightforward answers about and I'm hoping someone can help or point me to a good resource. 1) The intraclass correlation coefficient is of particular interest to me, as it describes the proportion of variation that occurs among individuals. Ideally I'd like to report a confidence interval of the ICC but I can't find any way to calculate one, other than a function in the psychometric package that appears to only work when there are no covariates in the model (random effect only). MCMC has already been mentioned and lme4 still has its mcmcsamp() function. Failing that, you could try a parametric bootstrap, which requires a little bit of coding but simulate() makes it much easier. 2) A reviewer requested a power analysis of the ability to detect a significant random effect. Any tips on how to approach that? Report the random effect and confidence intervals. Retrospective power analyses are pretty pointless (e.g. see http://beheco.oxfordjournals.org/content/14/3/446.full), unless you're planning to repeat the experiment. Bob -- Bob O'Hara Biodiversity and Climate Research Centre Senckenberganlage 25 D-60325 Frankfurt am Main, Germany Tel: +49 69 798 40226 Mobile: +49 1515 888 5440 WWW: http://www.bik-f.de/root/index.php?page_id=219 Blog: http://blogs.nature.com/boboh Journal of Negative Results - EEB: www.jnr-eeb.org ___ R-sig-ecology mailing list R-sig-ecology@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
Re: [R-sig-eco] ICC confidence intervals and power analysis for random effects in lmer?
2) A reviewer requested a power analysis of the ability to detect a significant random effect. Any tips on how to approach that? The reviewer was requesting a post-hoc power analysis, either an 'observed power' or 'detectable effect size' analysis. Hoenig and Heisey (2001) provide the definitive paper showing that the use of post hoc power analysis is fallacious, and that confidence intervals provide a more informative way to evaluate inferences about not rejecting the null statistical hypotheses. There are other good recent papers making the same point, but Hoenig and Heisey (2001) is definitive in my opinion. See: Hoenig, J. M., and D. M. Heisey. 2001. The abuse of power: The pervasive fallacy of power calculations for data analysis. Am. Stat. 55: 1-6. Note that this only applies to post hoc power analysis. Use of power analysis as an aid to planning experiments is an essential tool for experimental design. Tom Langen Associate Professor Departments of Biology Psychology Clarkson University Box 5805, Clarkson U., Potsdam NY 13699-5805 Phone: 315 268 7933, Fax: 315 268 7118 www.clarkson.edu/~tlangen -Original Message- From: r-sig-ecology-boun...@r-project.org [mailto:r-sig-ecology-boun...@r-project.org] On Behalf Of Bradley Carlson Sent: Wednesday, April 11, 2012 9:51 PM To: r-sig-ecology Subject: [R-sig-eco] ICC confidence intervals and power analysis for random effects in lmer? I'm performing an analysis of behavioral variation among individual tadpoles, using individual ID as a random effect and time as a continuous fixed covariate in the lmer function in lmer4 package. I'm really interested in making inferences about the random effect (i.e. the extent of variation among individuals). I'd like to do two things that I can't seem to find straightforward answers about and I'm hoping someone can help or point me to a good resource. 1) The intraclass correlation coefficient is of particular interest to me, as it describes the proportion of variation that occurs among individuals. Ideally I'd like to report a confidence interval of the ICC but I can't find any way to calculate one, other than a function in the psychometric package that appears to only work when there are no covariates in the model (random effect only). 2) A reviewer requested a power analysis of the ability to detect a significant random effect. Any tips on how to approach that? Thanks for any help, Brad -- Bradley Evan Carlson PhD Candidate Intercollege Graduate Degree Program in Ecology The Pennsylvania State University University Park, PA 16802 Email: carb...@gmail.com http://homes.bio.psu.edu/people/faculty/langkilde/index_files/carlson.htm https://sites.google.com/site/bradleyecarlson/home [[alternative HTML version deleted]] ___ R-sig-ecology mailing list R-sig-ecology@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-ecology ___ R-sig-ecology mailing list R-sig-ecology@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
Re: [R-sig-eco] ICC confidence intervals and power analysis for random effects in lmer?
In addition to Bob O'Hara's suggestion, here is another citation you can give the reviewer/editor, as to why retrospective power analyses are a waste of time. Hoenig, J. M. and D. M. Heisey (2001). The abuse of power: the pervasive fallacy of power calculations for data analysis. American Statistician 55(1): 19 - 24. Last year the ESA updated its author guidelines for reporting statistics, and removed a suggestion to report power analyses that had been inserted in the 1980s. -Brian Inouye Florida State University Chair, statistical ecology section of the ESA On 4/12/2012 6:00 AM, r-sig-ecology-requ...@r-project.org wrote: 2) A reviewer requested a power analysis of the ability to detect a significant random effect. Any tips on how to approach that? Report the random effect and confidence intervals. Retrospective power analyses are pretty pointless (e.g. see http://beheco.oxfordjournals.org/content/14/3/446.full), unless you're planning to repeat the experiment. Bob ___ R-sig-ecology mailing list R-sig-ecology@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
Re: [R-sig-eco] ICC confidence intervals and power analysis for random effects in lmer?
Thanks for the tips everyone - I'll look into MCMC sampling for the CI. As far as the power analysis goes, I'm somewhat familiar with the criticisms regarding power analysis. I think this reviewer was curious about it because there was a small sample size (small number of levels of the random effect) and the ICC point estimate was not so low as to be biologically insignificant. It would be nice to state in the paper how much larger of a sample size would have enabled us to detect an effect given the observed variation, as though this were a pilot study for planning a bigger experiment. I'll certainly bring up the suggested citations, but I would still be interested to know if there is a method available for performing a power analysis for an LRT of a random effect. Thanks again, Brad On Thu, Apr 12, 2012 at 12:18 PM, Brian Inouye bdino...@bio.fsu.edu wrote: In addition to Bob O'Hara's suggestion, here is another citation you can give the reviewer/editor, as to why retrospective power analyses are a waste of time. Hoenig, J. M. and D. M. Heisey (2001). The abuse of power: the pervasive fallacy of power calculations for data analysis. American Statistician 55(1): 19 - 24. Last year the ESA updated its author guidelines for reporting statistics, and removed a suggestion to report power analyses that had been inserted in the 1980s. -Brian Inouye Florida State University Chair, statistical ecology section of the ESA On 4/12/2012 6:00 AM, r-sig-ecology-request@r-**project.orgr-sig-ecology-requ...@r-project.orgwrote: 2) A reviewer requested a power analysis of the ability to detect a significant random effect. Any tips on how to approach that? Report the random effect and confidence intervals. Retrospective power analyses are pretty pointless (e.g. see http://beheco.oxfordjournals.** org/content/14/3/446.fullhttp://beheco.oxfordjournals.org/content/14/3/446.full), unless you're planning to repeat the experiment. Bob __**_ R-sig-ecology mailing list R-sig-ecology@r-project.org https://stat.ethz.ch/mailman/**listinfo/r-sig-ecologyhttps://stat.ethz.ch/mailman/listinfo/r-sig-ecology -- Bradley Evan Carlson PhD Candidate Intercollege Graduate Degree Program in Ecology The Pennsylvania State University University Park, PA 16802 Email: carb...@gmail.com http://homes.bio.psu.edu/people/faculty/langkilde/index_files/carlson.htm https://sites.google.com/site/bradleyecarlson/home [[alternative HTML version deleted]] ___ R-sig-ecology mailing list R-sig-ecology@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
[R-sig-eco] Which function wiould fit these data?
Hi list members.
Apologies if this is not too specifically R related, but I am looking fit a
model to some simulated data. X is distance to sample point, y is binary
outcome (present/absent). I was hoping someone can suggest a (presumably) non
linear function that might meet the following criteria
A. 0 y 1 (actual responses are binary)
B. xmax = $B!g(B
C. xmin 0
D. Where x = 0 then y 0
E. Where x = $B!g(B then y = 1
F. As x approaches $B!g(B from xmin, the function$B!G(Bs slope declines
monotonically toward 0.
Thanks in advance for any suggestions
Dr Terry Beutel
Agri-Science Queensland
DISCLAIMER**...{{dropped:15}}
___
R-sig-ecology mailing list
R-sig-ecology@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
