RE: Re:[tips] my crummy knowledge of stats

2013-01-20 Thread Wuensch, Karl L
From: Mike Wiliams [jmicha5...@aol.com] Sent: Thursday, January 17, 2013 12:19 AM To: Teaching in the Psychological Sciences (TIPS) Subject: Re:[tips] my crummy knowledge of stats You can use a conventional paired t test. Although you have dichotomous scores that does not

Re:[tips] my crummy knowledge of stats

2013-01-17 Thread Mike Wiliams
om: Mike Wiliams [jmicha5...@aol.com] Sent: Thursday, January 17, 2013 12:19 AM To: Teaching in the Psychological Sciences (TIPS) Subject: Re:[tips] my crummy knowledge of stats You can use a conventional paired t test. Although you have dichotomous scores that does not mean they are categorical. Cor

Re: [tips] my crummy knowledge of stats

2013-01-17 Thread John Kulig
quot; To: "Teaching in the Psychological Sciences (TIPS)" Sent: Thursday, January 17, 2013 1:41:34 PM Subject: Re: Re:[tips] my crummy knowledge of stats I think step 1 would still be an answer to the question "Was the change large enough to be considered "real" or stati

Re: [tips] my crummy knowledge of stats

2013-01-17 Thread John Kulig
Plymouth NH 03264 == - Original Message - From: "Claudia Stanny" To: "Teaching in the Psychological Sciences (TIPS)" Sent: Thursday, January 17, 2013 1:41:34 PM Subject: Re: Re:[tips] my crummy knowledge of stats I think step 1 woul

Re: [tips] my crummy knowledge of stats

2013-01-17 Thread John Kulig
Sent: Thursday, January 17, 2013 1:28:53 PM Subject: RE: Re:[tips] my crummy knowledge of stats My understanding of the intent of the analysis was to find items which were most affected, not a test for an omnibus effect across items. > - Original Message - > > From: "

Re: Re:[tips] my crummy knowledge of stats

2013-01-17 Thread Claudia Stanny
I think step 1 would still be an answer to the question "Was the change large enough to be considered "real" or statistically reliable." Step 2 would be to ask which of the statistically reliable changes were largest . . . perhaps an estimate of effect size and rank ordering of the questions. Out

RE: Re:[tips] my crummy knowledge of stats

2013-01-17 Thread Wuensch, Karl L
My understanding of the intent of the analysis was to find items which were most affected, not a test for an omnibus effect across items. > - Original Message - > > From: "Annette Taylor" > To: "Teaching in the Psychological Sciences > (TIPS)" > Sent: Tuesday, January 15, 2013 6:2

RE: Re:[tips] my crummy knowledge of stats

2013-01-17 Thread Paul C Bernhardt
ogical Sciences (TIPS) Subject: Re:[tips] my crummy knowledge of stats You can use a conventional paired t test. Although you have dichotomous scores that does not mean they are categorical. Correct/incorrect is a ratio scale of 1 unit. Green/Red, Accountant/Psychologist are the type of categori

Re:[tips] my crummy knowledge of stats

2013-01-16 Thread Mike Wiliams
You can use a conventional paired t test. Although you have dichotomous scores that does not mean they are categorical. Correct/incorrect is a ratio scale of 1 unit. Green/Red, Accountant/Psychologist are the type of categorical dichotomies that bring in the nonparametric procedures like C

Re: [tips] my crummy knowledge of stats

2013-01-16 Thread Jim Clark
Hi I would consider an alternative approach. For each ITEM, calculate the percentage of students who passed that item. Then do a paired-difference test of significance for pre vs post with items as the random factor (i.e., "subjects"). This will tell you whether there was an overall change.

Re: [tips] my crummy knowledge of stats

2013-01-16 Thread Paul C Bernhardt
The correct statistical test is called McNemar's Test: http://www.medcalc.org/manual/mcnemar_test.php It is specifically for dichotomous outcome data which is paired (repeated or matched). A modified Bonferroni more or less as you describe looks easy enough to do manually, after extracting the p

Re: [tips] my crummy knowledge of stats

2013-01-16 Thread John Kulig
Hi Annette Perhaps McNemar's Test for significance of changes, for dichotomous data. For each item, set up a table that looks like a 2*2 chi square but has "pretest" and "post-test" as variables (in texts its usually labelled "before" and "after") . Posttest - + + A B Pretest - C D So e