- I finally get back to this topic -
On Fri, 16 Mar 2001 23:40:07 GMT, [EMAIL PROTECTED] (Jerry Dallal)
wrote:
> Rich Ulrich ([EMAIL PROTECTED]) wrote:
>
> : Notice, you can take out a 0.1% test and leave the main
> : test as 4.9%, which is not effectively different from 5%.
>
> I've no problem with having different probabilities in the
> two tails as long as they're specified up front. I say
> so on my web page about 1-sided tests. I have concerns about
> getting investigators to settle on anything other than
> equal tails, but that's a separate issue.
> The thing I've found interesting about
> this thread is that everyone who seems to be defending
> one-tailed tests is proposing something other than a
> standard one-tailed test!
>
> FWIW, for large samples, 0.1% in the unexpected tail
> corresponds to a t statistic of 3.09. I'd love to
> be a fly on the wall while someone is explaining to
> a client why that t = 3.00 is non-significant! :-)
= concerning the 5.1% solution; asymmetrical testing
with 0.05 as a one-sided, nominal level of significance,
and 0.001 as the other side (as a precaution).
Jerry,
In that last line, you are jumping to a conclusion.
Aren't you jumping to a conclusion?
If the Investigator was seriously headed toward a 1-sided
test -- which (as I imagine it) is how it must have been, that
he could have been talked-around to the prospect of a 5.1%
combined test instead -- then he won't be eager to
jump on t= 3.00 as significant.
I mean, it can be easier to "publish" if you pass magic size,
but it is easier to avoid "perishing" in the long run, with a series
of connected hypotheses.
I think of the Investigator as torn three ways.
a) Stick to the plan; ignore the t=3.0, which is *not quite* 0.001.
'It did not reach the previously stated, 0.001 nominal level, and I
still don't believe it. (And I don't want to furnish ammunition for
arguments for other things.)' Practically speaking, the risk of
earning blame for stonewalling like that is not high.
b) Run with it; claim that a two-sided test always *did* make
sense and the statistician was to blame for brain-fever, for
wanting 1-tailed in the first place. (Or, never mention it.)
The fly on the wall probably would not see this.
The statistician should have already quit.
c) Report the outcome in the same diffident style as would have
been earned by a 0.06 result in the other direction, "not quite
meeting the preset nominal test size, but it is suggestive."
Unlike the 6% result, this one is unwelcome.
T=3.00 will stir up investigation to try to undermine the implication
(such as it is).
I have trouble taking the imagined outcome much further without
speculating about where you have the trade-off between effect-size
and N; and whether the "experimental design" was thoroughly robust --
and there's a different slant to the arguments if you are explaining
or explaining-away the results of uncontrolled observation.
--
Rich Ulrich, [EMAIL PROTECTED]
http://www.pitt.edu/~wpilib/index.html
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
