While I can see how the data available in this case might be
construed as being a sample from a larger population across time and
subjects (including persons not yet born), I have some difficulty thinking
of these data as representing a RANDOM or even a REPRESENTATIVE sample of
such a larger population of interest. That is, I'm inclined to think of
subjects as being a fixed rather than a random factor in the design.
An additional complication is that with a large sample size (I
assume it is quite large), even quite trivial effects will be found to be
"significant" if we test hypotheses. We all know that our usual audience
thinks that "significant" means "big and important." And no, they will pay
no attention to estimates of effect sizes and are not likely to be satisfied
with confidence intervals -- they will ask "but is it SIGNIFICANT?" Sigh.
Karl W.
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
Sent: Friday, April 04, 2003 8:29 AM
To: [EMAIL PROTECTED]
Subject: Re: T-test valid when whole population available?
In article <[EMAIL PROTECTED]>,
Alan McLean <[EMAIL PROTECTED]> wrote:
>Radford and I effectively made two different assumptions - I that the
>population of interest was the population measured, he that it it was
>wider than the population measured. With my assumption the t test is
>not relevant; with his, its relevance depends on whether the
>(sub)population measured can reasonably be considered a random sample
>from the population of interest.
Whether the t test in particular is the right tool is a detailed technical
issue that would depend on such things as whether it is reasonable to regard
the employees as independent (versus, for instance, a whole group of
same-sex friends having been hired by the company, because one of them got
hired and told the others how nice it
was.)
Regarding the more basic question of whether testing for statistical
significance is sensible at all, this does of course depend on what one
assumes is the population of interest. However, the recurring posts on this
topic seem to almost always be for situations in which testing for
significance IS appropriate, but somebody starts thinking too hard, saying,
"but we've got data on everyone..."
My guess is that situations where testing for significance is NOT
appropriate are usually so obvious that nobody gets confused. For instance,
suppose that the company is faced with a possible court ruling (sensible or
not) that would require it to raise the salaries of female employees to the
point where their average is the same as that of the men. The company wants
to know how much their payroll would increase if this happened. They
collect data on all the salaries, and from that figure out what the payroll
increase would be. Nobody would be silly enough to say - "Wait! The
difference between male and female salaries isn't statistically significant,
so maybe this court ruling won't cost us anything at all..."
Radford Neal
.
.
=================================================================
Instructions for joining and leaving this list, remarks about the
problem of INAPPROPRIATE MESSAGES, and archives are available at:
. http://jse.stat.ncsu.edu/ .
=================================================================
