If you want to bend the assumptions of statistics to this degree (and I'm not saying it is wrong, because there is plenty of evidence to support the robustness of t-test and ANOVA to all kinds of violations of assumptions, though I'm not sure about this particular choice, I'd want validating study to back me up), why not go for two-way within-subjects ANOVA? One variable is pre-post and the other is question number.
Paul ________________________________________ From: Mike Wiliams [jmicha5...@aol.com] Sent: Thursday, January 17, 2013 12:19 AM To: Teaching in the Psychological Sciences (TIPS) Subject: Re:[tips] my crummy knowledge of stats You can use a conventional paired t test. Although you have dichotomous scores that does not mean they are categorical. Correct/incorrect is a ratio scale of 1 unit. Green/Red, Accountant/Psychologist are the type of categorical dichotomies that bring in the nonparametric procedures like Chi-square or ranking tests. Just calculate a mean difference and variance for each item and analyze them the usual way. You might also try some of the test reliability stats that are now in SPSS, such as coefficient alpha. Alpha is a general index of how well the items intercorrelate or "hang together". Mike Williams > ----- Original Message ----- > > From: "Annette Taylor"<tay...@sandiego.edu> > To: "Teaching in the Psychological Sciences > (TIPS)"<tips@fsulist.frostburg.edu> > Sent: Tuesday, January 15, 2013 6:21:42 PM > Subject: [tips] my crummy knowledge of stats > > I know this is a basic question but here goes: > > I have categorical data, 0,1 which stands for incorrect (0) or correct (1) on > a test item. > > I have 25 items and I have a pretest and a posttest and I want to know on > which items students improved significantly, and not just by chance. Just > eyeballing the data I can tell that there are some on which the improved > quite a bit, some not at all and some are someplace in the middle and I can't > make a guess at all. That is why we have statistics. Yeah! .... > hmmmm....bleh..... > > As far as I know, the best thing to do is a chi-square test for each of 25 > items; but of course that will mean that with a .05 sig level I will have at > least one false positive, maybe more, but most assuredly at least one. This > seems to be a risk. At any rate I can use SPSS and the crosstabs command > allow for calculation of the chi-square. > > I know that when I do planned comparisons with multiple t-tests, I can do a > Simes' correction in which I can rank order my final, obtained alphas, and > adjust for the number of comparisons and reject from the point from which the > obtained alpha failed to exceed the corrected-for-number-of-comps alpha. But > as far as I know, I cannot do that with 25 chi square tests. There is > probably some reason why I can no more do that, that relates to the reason > for why I cannot do 25 t-tests in this situation with categorical data. > > Is there a better way to answer my research question? I need a major > professor! Oh wait, that's me... drat! I need to hire a statistician. Oh > wait, I'd need $$ for that and I don't have any. So I hope tipsters can stand > in as a quasi-hired-statistician and help me out. > > Oh, I get the digest. I don't mind waiting until tomorrow or the next day for > a response, but a backchannel is fine.tay...@sandiego.edu > > I will be at APS this year. Any other tipsters planning to be there? Let's > have a party! I'd love to put personalities to names. > > Thanks > > Annette > > Annette Kujawski Taylor, Ph. D. > Professor, Psychological Sciences > University of San Diego > 5998 Alcala Park > San Diego, CA 92110 > tay...@sandiego.edu --- You are currently subscribed to tips as: pcbernha...@frostburg.edu. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13441.4e79e96ebb5671bdb50111f18f263003&n=T&l=tips&o=23097 or send a blank email to leave-23097-13441.4e79e96ebb5671bdb50111f18f263...@fsulist.frostburg.edu --- You are currently subscribed to tips as: arch...@jab.org. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=23102 or send a blank email to leave-23102-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
