One of the nice things about this type of discussion is that, given a
suitable 'settling time' between postings, I can be reminded how far an
intro stat approach is from what _should_ be done.
A proper full blown consultation for a problem would not jump in as I did,
but would spend a lot of time exploring the presumptions that led to the
structure of the data in the first place. that the analyst (researcher
with the numbers) is unaware of the linkages and implications of the
clarification questions only adds to the difficulty. -- "Action" oriented
people see the consulting statistician as an "abstract thinker."
If an advisory suggestion is suited to the level of understanding of the
analyst, is this acceptable? I fear not. Yet simultaneously, adding in
the (possible) ramifications and implications can tie up said analyst in
total knots, unable to move intellectually. How to get out of this? Rank
the advice in ascending order of difficulty? But then the order of
assumptions required (simplifying assumptions) would be in descending
order.
Somewhere between the 'simplest' analysis and the most complete analysis
will lie the one which the analyst can live with, can understand, and draw
interesting/useful information from. I hope. How can one (or we) find a
way to provide that level of analysis?
Jay
Donald Burrill wrote:
> Hi, Jay.
>
> You are of course correct, in that the data from Pingu's (aka DN's)
> experiment can be analyzed by a one-way ANOVA (aka a completely
> randomized design). The thing is, there MAY be noticeable effects on
> performance time due to practice -- that is, due to the "repetitions"
> factor. Should that be the case, the completely randomized design may
> be less sensitive to differences among the eight conditions (which is
> what DN really wants to know about, or so I gather) than the randomized-
> block design. (It would be entirely appropriate to run both analyses,
> and then compute their relative efficiency: see, e.g., Wm.C. Guenther,
> "Analysis of Variance", Prentice-Hall, 1964, section 3-11. But I rather
> thought this might be more technical than DN actually wanted.)
>
> Whether the putative "less sensitive" is actually true, and whether the
> effects of the conditions are subtle enough that this might be a
> problem, depend on the data (or the state of nature the data are thought
> to reflect, I suppose).
>
> As I wrote earlier, any further comment would entail speculation, in the
> absence of more concrete information about the actual enterprise. So
> let's speculate a bit. (You can hit the <DELETE> key now, if you like.)
>
> The experiment involves several different conditions under which a task
> (nature unspecified, so far) is performed. The measure of interest is
> response time -- which may mean "time taken to complete the task",
> presumably measured from start to finish; or it may mean "the time
> taken to get started" on the task, presumably measured from the time the
> task is presented to the person until the time the person responds to
> that stimulus; or, imaginably, something else entirely.
>
> That DN is interested in seeing "whether any of the conditions lead to
> significantly faster response times than the others" leads me to suppose
> that faster response times are desirable, and that he has chosen 10 as a
> reasonable number of repetitions suggests (to me, anyway) that DN thinks
> that by the time one has done this task ten times, one's speed is about
> as fast as it's likely to get. I suspect DN envisions a <response-time>
> vs. <repetition-number> graph that declines toward a horizontal
> asymptote, which is why I wrote earlier that the decline was probably
> not linear.
> (I DID say I was speculating! :)
>
> Hence my advice to carry out a two-way ANOVA. In SPSS, or any other of
> the usual packages, it's as easy to do a two-way as a one-way; and the
> two-way results will include the one-way results, and will also have
> partitioned the between-conditions SS (with 72 degrees of freedom) into
> two parts with respectively 9 (between repetitions) and 63 ("error")
> d.f. If the error MS with 63 d.f. is notably smaller than the between-
> groups MS with 72 d.f., the two-way analysis will be more sensitive to
> differences among the eight conditions; if it is larger, the explicit
> acknowledgement of the "repetitions" (or "practice") factor was not
> helpful in detecting any such differences, and one might as well report
> the one-way ANOVA you recommend. But one can't tell without looking.
>
> On Thu, 10 Jul 2003, Jay Warner wrote in part:
>
> > Dare I leap in where angels fear to tread?
>
> Do they? I hadn't noticed. <grin>
>
> > Certainly D. Burrell [sic] has beat this question something fierce.
>
> And here I thought I was just being obvious...
>
> > ... Suppose I say that the 8 conditions are categorically different
> > - some aren't higher on any scale, they are simply different.
>
> Yep. That's all ANYone has said, so far.
>
> > Then I assert loudly that the 10 repetitions are performed all at
> > the same time, with one 'set-up' of the specific condition.
>
> Feel free. But be aware that you are asserting a falsehood. If these
> are, as claimed, performance tasks, and all done by one person, they
> cannot possibly have been performed all at the same time.
> (I had been seriously contemplating elaborating on this with a task of
> my choosing [since I'm speculating anyway], namely the playing on a pipe
> organ of J.S. Bach's Toccata in d minor, on which one could pretty well
> guarantee that performance on the 10th repetition would be superior to
> performance on the first; ... but let that pass.)
>
> > If I see no trends (or ignore the ones I see) in the ... repetitions,
> > I can ... set up a One-Way Analysis of Variance. ...
>
> Yes. But a two-way ANOVA will provide the one-way information, while
> showing whether there ARE trends (or other systematic variation) in the
> repetitions that would reduce the sensitivity of the one-way design.
>
> < snip >
>
> > then there is the intriguing possibility that your 8 conditions are
> > not simply "8 brands of automobile." Instead, you just might be
> > taking on a designed experiment, with three factors. This could be
> > interesting!
>
> Yes, "8" IS an interesting number, isn't it? A 2x2x2 design with
> repeated measures on the 10-level practice effect would not be at all
> unreasonable. It's almost an archetypal design in psychology...
>
> > If you now have 8 conditions, all related but different, your 10
> > reps could be seen as a nice way to reduce variation in the
> > individual measurements, but you will back it out later in the
> > analysis with an estimate.
>
> Another gloss on this is possible. If indeed the "practice" effect
> looks like the response times do indeed approach an asymptote, one might
> do well to take the last several repetitions, where (if this is indeed
> the case) the response times do not differ much from each other, as the
> measures for each condition. Variability within conditions might very
> well be noticeably greater in the early repetitions, and contribute
> visibly to the error variance, so that subtler differences between
> conditions might be detectable with the later measurements but not with
> the earlier, and perhaps not with all measurements included in the
> analysis.
>
> > Could you do say 2 reps, change to a different condition, then do 2
> > more, and keep this up until you have 10 sets of measurements
> > altogether? Then you could estimate variation due to each
> > measurement, and variation due to changing the specific donations of
> > a 'condition.'
>
> Definitely preferable. Even better with 1 rep; although the nature of
> the task and the repetitions may be such that the repetitions have to be
> presented in "massed" rather than "distributed" form. If they can be
> distributed, it would also be preferable to randomize the order in which
> the eight conditions are encountered.
>
> The impression I had was that data had already been collected, and it is
> not clear how handily it might be replicated. In any case, it would
> doubtless be salutary to analyze the present data, with an eye toward
> detecting any of the possible threats to validity that both of us have
> mentioned.
>
> Ciao! -- Don.
> -----------------------------------------------------------------------
> Donald F. Burrill [EMAIL PROTECTED]
> 56 Sebbins Pond Drive, Bedford, NH 03110 (603) 626-0816
--
Jay Warner
Principal Scientist
Warner Consulting, Inc.
4444 North Green Bay Road
Racine, WI 53404-1216
USA
Ph: (262) 634-9100
FAX: (262) 681-1133
email: [EMAIL PROTECTED]
web: http://www.a2q.com
The A2Q Method (tm) -- What do you want to improve today?
.
.
=================================================================
Instructions for joining and leaving this list, remarks about the
problem of INAPPROPRIATE MESSAGES, and archives are available at:
. http://jse.stat.ncsu.edu/ .
=================================================================
