Re: Post-hoc comparisons
On 2 Mar 2001 07:27:16 -0800, [EMAIL PROTECTED] (Esa M. Rantanen) wrote: [ snip, detail ] > contingency table. I have used a Chi-Sq. analysis to determine if there is > a statisitcally significant difference between the (treatment) groups (all > 4!), and indeed there is. I assume, however, that I cannot simply do > pairwise comparisons between the groups using Chi-Sq. and 2 x 2 matrices > without inflating the probability of Type 1 error, (1-alpha)^4 in this > case. As far as I know, there are no equivalents to Duncan's or Tukey's > tests for the type of data (binary) I have to deal with. Well, if you want to do the ANOVA on the dichotomous variable, I won't complain. My reaction is, you are assuming that, somewhere, great precision matters. But being precise in your thinking will gain you most, so that you do and report just ONE important test, that you figured out beforehand, instead of trying to cope with 6 tests that happen to fall into your lap. I would probably (a) Let the Overall test justify all my followup testing, where the followup testing is descriptive, among categories of equal N and equivalent importance; or (b) Do a few specified tests with Bonferroni correction, and report those tests. -- Rich Ulrich, [EMAIL PROTECTED] http://www.pitt.edu/~wpilib/index.html = Instructions for joining and leaving this list and remarks about the problem of INAPPROPRIATE MESSAGES are available at http://jse.stat.ncsu.edu/ =
Re: Post-hoc comparisons
Hi, Esa! You've had a couple of responses; here's another. You state "pairwise comparisons"; but it strikes me as at least possible that you might want (or might _also_ want) to consider more complex comparisons if any such comparisons seemed to offer a more parsimonious (or, perhaps, more theory-related?) explanation of the differences among the four conditions. (E.g., conditions A & B vs. conditions C & D; or, condition B vs. conditions A & C & D; or, condition A vs. conditions B & D and condition C vs. conditions B & D.) I would ordinarily think of using the Scheffe' method (or the Tukey method, if the sample sizes were equal in each condition and one's interest really were _only_ in pairwise comparisons): its experimentwise Type I error rate means no need for Bonferroni or similar calculations; just convert your binary response to a proportion passed (or proportion failed, if that be easier to interpret) and do a one-way ANOVA on that proportion in the four treatments. -- Don. On Fri, 2 Mar 2001, Esa M. Rantanen wrote: > I have a question concerning pairwise comparisons between four > treatment conditions. I have a single factor experiment with > four levels of the factor (treatment conditions) and a discrete > dependent measure (pass/fail), resulting in a 2 x 4 contingency table. > ... Chi-Sq. analysis [has found] a statistically significant difference > between the (treatment) groups (all 4!). > > I would appreciate [it] if anyone would confirm my reasoning above and > offer any advice on how to proceed with the analysis of pairwise > differences in the case of categorical (dichotomous) data. References > to relevant literature would also be welcome! -- Donald F. Burrill[EMAIL PROTECTED] 348 Hyde Hall, Plymouth State College, [EMAIL PROTECTED] MSC #29, Plymouth, NH 03264 (603) 535-2597 Department of Mathematics, Boston University[EMAIL PROTECTED] 111 Cummington Street, room 261, Boston, MA 02215 (617) 353-5288 184 Nashua Road, Bedford, NH 03110 (603) 471-7128 = Instructions for joining and leaving this list and remarks about the problem of INAPPROPRIATE MESSAGES are available at http://jse.stat.ncsu.edu/ =
Re: Post-hoc comparisons
At 08:43 AM 3/2/01 -0600, Esa M. Rantanen wrote: >Dear All: > >I have a question concerning pairwise comparisons between four treatment >conditions. My experience is mostly with ANOVA, and (I think!) I can >understand the reasoning for the use of multiple comparison procedures >(e.g., Duncan's, Tukey's, or LSD) instead of individual t-tests between >conditions. You should forget Duncan's or Neuman-Keuls procedures as beeing too liberal. > >I assume the case is the same with my current problem: I have a single >-factor experiment with four levels of the factor (treatment conditions) >and a discrete dependent measure (pass/fail), resulting in a 2 x 4 >contingency table. I have used a Chi-Sq. analysis to determine if there is >a statisitcally significant difference between the (treatment) groups (all >4!), and indeed there is. I assume, however, that I cannot simply do >pairwise comparisons between the groups using Chi-Sq. and 2 x 2 matrices >without inflating the probability of Type 1 error, (1-alpha)^4 in this >case. As far as I know, there are no equivalents to Duncan's or Tukey's >tests for the type of data (binary) I have to deal with. > >I would appreciate if anyone would confirm my reasoning above and offer any >advice on how to proceed with the analysis of pairwise differences in the >case of categorical (dichotomous) data. References to relevant literature >would also be welcome! You are correct. Unless you have a priori, non-redundant hypotheses where contrasts would be feasible, you should correct the probability of you pairwise chi-squares using Bonferroni or Sidak corrections. If you do all pairwise comparisons there would be 6. Thus, the comparisonwise error rate would be .0084 by Bonferroni and .0085 by Sidak. > >Best, > >Esa > >Esa M. Rantanen, Ph.D. >Assistant Professor >University of Illinois at Urbana-Champaign >Institute of Aviation, Aviation Human Factors Division >Aviation Research Laboratory, Q5, MC-394 >One Airport Road, Willard Airport >Savoy, IL 61874 >Tel. 217-244-8657 (ARL) >Tel. 217-244-7397 (Psych.) >Tel. 217-373-8276 (Home) >Fax 217-244-8647 >e-mail: [EMAIL PROTECTED] >url: http://www.aviation.uiuc.edu > > > > > >= >Instructions for joining and leaving this list and remarks about >the problem of INAPPROPRIATE MESSAGES are available at > http://jse.stat.ncsu.edu/ >= > Paul R. Swank, PhD. Professor & Advanced Quantitative Methodologist UT-Houston School of Nursing Center for Nursing Research Phone (713)500-2031 Fax (713) 500-2033 = Instructions for joining and leaving this list and remarks about the problem of INAPPROPRIATE MESSAGES are available at http://jse.stat.ncsu.edu/ =
Re: Post-hoc comparisons
It sounds to me like are are dealing with a comparison of four proportions... Why can't you follow up the initial test with the six pairwise tests of proportions, using some type of Bonferroni correction... There's the Holm modification and the FDR procedure, both of which give adequate protection with greater power than a "pure" Bonferroni approach... WBW __ William B. Ware, Professor and Chair Educational Psychology, CB# 3500 Measurement, and Evaluation University of North Carolina PHONE (919)-962-7848 Chapel Hill, NC 27599-3500 FAX: (919)-962-1533 http://www.unc.edu/~wbware/ EMAIL: [EMAIL PROTECTED] __ On Fri, 2 Mar 2001, Esa M. Rantanen wrote: > Dear All: > > I have a question concerning pairwise comparisons between four treatment > conditions. My experience is mostly with ANOVA, and (I think!) I can > understand the reasoning for the use of multiple comparison procedures > (e.g., Duncan's, Tukey's, or LSD) instead of individual t-tests between > conditions. > > I assume the case is the same with my current problem: I have a single > -factor experiment with four levels of the factor (treatment conditions) > and a discrete dependent measure (pass/fail), resulting in a 2 x 4 > contingency table. I have used a Chi-Sq. analysis to determine if there is > a statisitcally significant difference between the (treatment) groups (all > 4!), and indeed there is. I assume, however, that I cannot simply do > pairwise comparisons between the groups using Chi-Sq. and 2 x 2 matrices > without inflating the probability of Type 1 error, (1-alpha)^4 in this > case. As far as I know, there are no equivalents to Duncan's or Tukey's > tests for the type of data (binary) I have to deal with. > > I would appreciate if anyone would confirm my reasoning above and offer any > advice on how to proceed with the analysis of pairwise differences in the > case of categorical (dichotomous) data. References to relevant literature > would also be welcome! > > Best, > > Esa > > Esa M. Rantanen, Ph.D. > Assistant Professor > University of Illinois at Urbana-Champaign > Institute of Aviation, Aviation Human Factors Division > Aviation Research Laboratory, Q5, MC-394 > One Airport Road, Willard Airport > Savoy, IL 61874 > Tel. 217-244-8657 (ARL) > Tel. 217-244-7397 (Psych.) > Tel. 217-373-8276 (Home) > Fax 217-244-8647 > e-mail: [EMAIL PROTECTED] > url: http://www.aviation.uiuc.edu > > > > > > = > Instructions for joining and leaving this list and remarks about > the problem of INAPPROPRIATE MESSAGES are available at > http://jse.stat.ncsu.edu/ > = > = Instructions for joining and leaving this list and remarks about the problem of INAPPROPRIATE MESSAGES are available at http://jse.stat.ncsu.edu/ =
