I am assuming this was an independent samples t test where some participants heard the "mother nature" language and others didn't. Using the d of .53 they obtained as my estimate of what effect size they would be interested in obtaining (or that they think would be worthwhile to note), it appears that, with a df of 50, they had less than a 50/50 chance of finding a significant result of that size if one existed in the population. As others have pointed out, you need to determine before the study begins, what effect size you are interested in obtaining. For example, you may believe that even a .05 effect size (1/20th of a standard deviation difference between the two means) could be meaningful given the question. If so, you are going to need a very large sample size to have a high probability of finding a significant result if such a small difference exists in the population. By my calculations*, if you wanted to have at least an 80 percent chance of detecting an effect size of at least .50 (half a standard deviation difference between the means) with an independent sample t test, you would need to have 128 participants in the study (64 in each group). If you wanted to have an 80% chance of detecting a .05 (5 percent) effect size in such a case, you would need 12560 participants (6280 in each group).
*My power calculations came from http://homepage.stat.uiowa.edu/~rlenth/Power/. The author has a nice discussion of power and why retrospective power analysis is worthless under the Advice section on that page. Rick Dr. Rick Froman, Chair Division of Humanities and Social Sciences Box 3519 x7295 rfro...@jbu.edu<mailto:rfro...@jbu.edu> http://bit.ly/DrFroman Proverbs 14:15 "A simple man believes anything, but a prudent man gives thought to his steps." From: Michael Britt [mailto:mich...@thepsychfiles.com] Sent: Tuesday, August 27, 2013 9:00 AM To: Teaching in the Psychological Sciences (TIPS) Subject: Re: [tips] Sample Size: How to Determine it? Also helpful. So, to answer my own previous question, based on what they found in the correlational study and what one might guess from previous research, I'm going to assume that the effect size here, if it exists, is probably small. So I used .3 in G*Power. The result? G*Power suggests that I get 242 subjects per group. These researchers had 26 subjects in each group. So: if you were the reviewer what would you conclude? The researchers found: "...the results revealed that participants in the anthropomorphism condition were tendentially less willing to help the victims of the natural disaster (M = 4.39, SD = 1.02) than participants in the control condition (M = 4.89, SD = 0.87), t(50) = -1.91, p = .06, d = 0.53. Would you recommend that they get more subjects? Michael Michael A. Britt, Ph.D. mich...@thepsychfiles.com<mailto:mich...@thepsychfiles.com> http://www.ThePsychFiles.com Twitter: @mbritt On Aug 27, 2013, at 8:59 AM, Stuart McKelvie <smcke...@ubishops.ca<mailto:smcke...@ubishops.ca>> wrote: Dear Tipsters, There are various ways to plan sample size. When teaching this in research methods, I divide the issues into two parts: 1. Estimation of population values. Here, more is better but there are diminishing returns. Think of the fact that we rarely see more than 1500 people in national polls and surveys. The formula is based on minimizing standard error. Of course, sampling is critical. 2. Conducting studies with variables: experimental, subject or correlational. There are four interconnected concepts: effect size, alpha, power and sample size. When any three are known, the fourth is determined. You can decide where to set alpha and power. For effect size (d), you can be guided by Cohen's guidelines for small, medium and large (.3, .5, .8) and choose the value you are looking for. This may come from past research or, in its absence, what you think is interesting theoretically or practically. Cohen's book on power analysis gives tables where you can look up the sample size needed after specifying the values you choose. There is also this webiste: http://homepage.stat.uiowa.edu/~rlenth/Power/ Sincerely, Stuart _____________________________________________________ Sent via Web Access "Floreat Labore" "Recti cultus pectora roborant" Stuart J. McKelvie, Ph.D., Phone: 819 822 9600 x 2402 Department of Psychology, Fax: 819 822 9661 Bishop's University, 2600 rue College, Sherbrooke, Québec J1M 1Z7, Canada. E-mail: stuart.mckel...@ubishops.ca<mailto:stuart.mckel...@ubishops.ca> (or smcke...@ubishops.ca<mailto:smcke...@ubishops.ca>) Bishop's University Psychology Department Web Page: http://www.ubishops.ca/ccc/div/soc/psy " Floreat Labore" _______________________________________________________ ________________________________ From: Paul C Bernhardt [pcbernha...@frostburg.edu<mailto:pcbernha...@frostburg.edu>] Sent: 27 August 2013 08:41 To: Teaching in the Psychological Sciences (TIPS) Subject: Re: [tips] Sample Size: How to Determine it? There is software to determine this. One excellent and free app is G*Power. http://www.psycho.uni-duesseldorf.de/abteilungen/aap/gpower3/ I would use the correlational study to give me an estimate of effect size. As you describe, I would use that in the software to estimate my number of participants to attain the desired power. Practicality constraints on number of available participants usually limits things. I did such an estimate using G*Power a few weeks ago for a study we are planning. We will need to collect data over two semesters because the anticipated number of participants available from one semester's worth of students would only give us power of about .66, whereas two semester's worth would bump us up over .90. Paul On Aug 27, 2013, at 8:18 AM, Michael Britt wrote: I'm reading an interesting piece of research on anthropomorphism which essentially states after a natural disaster if we use the term "mother nature" when describing it, people will be less willing to contribute to relief efforts ("Humanizing nature could help the perceiver to conceive natural events as imbued with intentionality and significance rather than considering them merely random and meaningless phenomena"). They did two studies. Here's the issue/question: * Study 1 was correlational and involved 96 students. The results were supportive at <.001 * Study 2 was an experiment (no need to go into the details) involving 56 students. The results were, in the authors words, "tangentially" supportive with p<.06 I think the study was well conducted so I don't mean to slight the researchers. My guess is that if they used more subjects they probably would have reached p<.05 - but would that have been an example of "selective stopping"? I assume it would be. So how exactly does a researcher determine beforehand - as we are suggesting they do - the number of subjects they ought to try to get for the study? I'm just not familiar with the process. Does one look at the effect sizes of previous related studies to determine if the effect is large or small and then make a decision? But let's say the effect is assumed to be small, so do you use 100 subjects? 500? How is this number determined? Appreciate the insight in this. Michael Michael A. Britt, Ph.D. mich...@thepsychfiles.com<mailto:mich...@thepsychfiles.com> http://www.ThePsychFiles.com Twitter: @mbritt --- You are currently subscribed to tips as: pcbernha...@frostburg.edu<mailto:pcbernha...@frostburg.edu>. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13441.4e79e96ebb5671bdb50111f18f263003&n=T&l=tips&o=27372 (It may be necessary to cut and paste the above URL if the line is broken) or send a blank email to leave-27372-13441.4e79e96ebb5671bdb50111f18f263...@fsulist.frostburg.edu<mailto:leave-27372-13441.4e79e96ebb5671bdb50111f18f263...@fsulist.frostburg.edu> --- You are currently subscribed to tips as: smcke...@ubishops.ca<mailto:smcke...@ubishops.ca>. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13510.2cc18398df2e6692fffc29a610cb72e3&n=T&l=tips&o=27373 (It may be necessary to cut and paste the above URL if the line is broken) or send a blank email to leave-27373-13510.2cc18398df2e6692fffc29a610cb7...@fsulist.frostburg.edu<mailto:leave-27373-13510.2cc18398df2e6692fffc29a610cb7...@fsulist.frostburg.edu> --- You are currently subscribed to tips as: michael.br...@thepsychfiles.com<mailto:michael.br...@thepsychfiles.com>. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13405.0125141592fa9ededc665c55d9958f69&n=T&l=tips&o=27374 (It may be necessary to cut and paste the above URL if the line is broken) or send a blank email to leave-27374-13405.0125141592fa9ededc665c55d9958...@fsulist.frostburg.edu<mailto:leave-27374-13405.0125141592fa9ededc665c55d9958...@fsulist.frostburg.edu> --- You are currently subscribed to tips as: rfro...@jbu.edu<mailto:rfro...@jbu.edu>. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13039.37a56d458b5e856d05bcfb3322db5f8a&n=T&l=tips&o=27379 (It may be necessary to cut and paste the above URL if the line is broken) or send a blank email to leave-27379-13039.37a56d458b5e856d05bcfb3322db5...@fsulist.frostburg.edu<mailto:leave-27379-13039.37a56d458b5e856d05bcfb3322db5...@fsulist.frostburg.edu> --- You are currently subscribed to tips as: arch...@jab.org. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=27387 or send a blank email to leave-27387-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
