Not sure I understand the situation thoroughly, so will restate it; if my restatement of what I think you probably did agrees with what you think you did, we may have enough common ground for a conversation.
In an experiment, 24 persons (subjects, Ss) undertook 3 trials, and they attacked each trial 4 times, each time under a different condition. (Aside: are the three trials equal or equivalent in some sense, or were they trials of three quite different things altogether? It may be necessary for you to describe the experiment in more detail in order for anyone to assist you intelligently.) For each trial a "best" condition (one out of the four, presumably) was identified. This identification appears to have been based on the time taken to complete the task involved with each trial: minimum time, I would presume. (Aside: No indication of whether there was ever any ambiguity in deciding which condition was "best": whether there were ties for "best" time, or whether the differences in time were even statistically significant for any Ss.) "A chi square goodness of fit test" was carried out. I do not perceive what this will have been: there are lots of chi-square tests, many of which are not incompatible with the epithet "goodness of fit". What was the model you were trying to fit, and what precisely was the test? You are concerned about an "assumption that all data come from independent sources", which you feel the data violated. It is not clear (a) why you feel the data violated the assumption; (b) whether such an assumption is in fact necessary or implied. (Aside: Interesting phrase "violate an assumption". I know about "violations" in the context of "violating a law", e.g.; but an assumption? I take it you are trying to say that the assumption (whatever it is, when expressed in explicit detail) was not met by your data, or by the conditions of your experiment.) You haven't said what you're trying to find out in your analysis/es. You might be interested in whether the "best" time, regardless of condition (or averaged over all conditions), differs systematically between trials; but I don't think this is what you intended us to understand between the lines. Or you may be interested mostly in the four conditions: either in which conditions turn out to be "best" most frequently, or in what the "best" time is for each condition, or in what Ss have the "best" time. Hard to advise on statistical analyses without knowing the intended purpose(s) of the enquiry. So let's consider your data. You have 72 cases, 3 for each of 24 Ss. Each case contains, at a minimum, these variables: subject ID (a number between 1 and 24) trial ID (a number between 1 and 3) best condition ID (a number between 1 and 4) best condition time (in minutes? seconds? milliseconds? fortnights?) There may also be subject-level variables you haven't mentioned yet: sex, age, whether subjected to a certain experimental treatment or not, etc. Obvious analyses that could be performed include: 1. the 3x4 contingency table (trials by conditions), asking whether the distribution of conditions differs across trials. If the overall chi square value (with 6 df) is significant, you'd want to pursue follow-up enquiries of one kind or another: a possible topic of conversation. 2. the 3x24 trials-by-subjects repeated-measure ANOVA of "best" times, to see whether "best" time differs systematically by trial, or by any design variables (e.g., sex) within which Ss are nested. 3. the ANOVA of (2), but carried out in a multiple-regression module, to permit inclusion of the "condition" category (with 3 df) as a set of predictors. On 30 Jul 2003, Joe wrote: > I performed a test where 24 subjects performed 3 trials with 4 > conditions. So each person has three results of "best" conditions, > and I have a total of 72 "best" conditions. I did a chi square > goodness of fit test using all 72 trial results but realized I > violated the assumption that all data come from independent sources. > Are there any other tests that allow me to use all 72 trials, or can I > only perform the chi square GoF test on the results of a single trial > (for example, trial 3 with n=24)? > > The "best" measurement refers to task time, but each user did not > perform the same task so I can't do a parametric Anova. This does not necessarily follow. The obvious ANOVA (#2 above) would tell you whether the "best" time differs interestingly among trials. It wouldn't tell you anything about conditions; but the regression analyses hinted at (#3 above) could possibly do so. Doubtless with some ambiguity, since "best" condition ID may very well be associated (in a simple way if you're lucky, in a complicated way more likely) with "best" time. > I want to be able to perform a statistical analysis of the best > performing condition across all subjects, so I'm looking at > frequency of best condition as opposed to using the time > measurement. To say "a statistical analysis" is to be frustratingly vague. What RESEARCH QUESTION(s) had you in mind to attempt to attack? > For example: the frequecies of best condition are as follows (for all > 72 trials) > condition 1:28, condition 2:16, condition 3:17, condition 4: 11 > the chi square test shows significant difference when I compare all > conditions, and also when I compare condition 1 to all three of the > other conditions combined. Would this be a test of the form <chi-square value> = SUM[(O-E)^2/E] where the observed frequencies "O" are 28, 16, 17, and 11, and the expected frequencies "E" are all equal (to 72/4 = 18)? The second analysis would be of the same form with 1 d.f, "O" = 28 and 44, "E" = 18 and 54 respectively? ----------------------------------------------------------------------- Donald F. Burrill [EMAIL PROTECTED] 56 Sebbins Pond Drive, Bedford, NH 03110 (603) 626-0816 . . ================================================================= Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at: . http://jse.stat.ncsu.edu/ . =================================================================
