At 07:23 PM 9/12/01 -0500, jim clark wrote:
>Hi
>
>
>What your table shows is that _both_ dimensions are informative.
>That is, you cannot derive effect size from significance, nor
>significance from effect size. To illustrate why you need both,
>consider a study with small n that happened to get a large effect
>that was not significant. The large effect should be "ignored"
>as being due to chance. Only having the effect size would likely
>lead to the error of treating it as real (i.e., non-chance.
or, another way to view it is that neither of the dimensions is very
informative
of course we know that significance does not mean "real" and non
significance does not mean "chance alone" ... there is no way to decipher
one from the other based on our significance tests
the distinction between significant or not ... is based on an arbitrary
cutoff point ... which has on one side ... the notion that the null seems
as though it might be tenable ... and the other side ... the notion that
the null does not seem to tenable ... but this is not an either/or deal ...
it is only a matter of degree
what if we juxtapose ... non significant findings with a large effect size
... with significant results with a small effect size ... which of these
two would most feel "had most import"?
surely, if both dimensions are seen as been equal parts of the overall
puzzle, then, both would seem to be more or less equal
but, if one opts for large effect size when results are not significant ...
then, this suggests that significance adds little if anything to the mix ...
however, if we opt for significance along with a small effect size, then
this suggests that significance is playing a more important role in one's eyes
the reality is too that effect sizes are, when categorized as small,
medium, and large ... again ... totally arbitrary ... which makes it even
harder still to make much sense of these ... just like it is difficult to
make much sense out of significance
and finally, effect sizes do NOT say anything about importance OF the
effect ... merely some statement about the SIZE of the effect ... so, it
could very well be that for many independent variables ... a small effect
size has a much greater impact consequence than a large effect size ...
even when both are found to have significance on their side
just to muddy the waters more
==============================================================
dennis roberts, penn state university
educational psychology, 8148632401
http://roberts.ed.psu.edu/users/droberts/drober~1.htm
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
