Hi, Dennis!
Yes, as you point out, most elementary textbooks treat only SRS
types of samples. But while (as you also point out) some more realistic
sampling methods entail larger sampling variance than SRS, some of them
have _smaller_ variance -- notably, stratified designs when the strata
differ between themselves on the quantity being measured.
On Tue, 24 Jul 2001, Dennis Roberts wrote:
> most books talk about inferential statistics ... particularly those
> where you take a sample ... find some statistic ... estimate some error
> term ... then build a CI or test some null hypothesis ...
>
> error in these cases is always assumed to be based on taking AT LEAST a
> simple random sample ... or SRS as some books like to say ...
>
> but, we KNOW that most samples are drawn in a way that is WORSE than SRS
I don't think _I_ know this. I know that SOME samples are so drawn;
but (see above) I also know that SOME samples are drawn in a way that
is BETTER than SRS (where I assume by "worse" you meant "with larger
sampling variance", so by "better" I mean "with smaller sampling
variance").
> thus, essentially every CI ... is too narrow ... or, every test
> statistic ... t or F or whatever ... has a p value that is too LOW
>
> what adjustment do we make for this basic problem?
I perceive the "basic problem" as the fact that sampling variance is
(relatively) easily calculated for a SRS, while it is more difficult
to calculate under almost _any_ other type of sampling.
Whether it is enough more difficult that one would REALLY like to avoid
it in an elementary course is a judgement call; but for the less
quantitatively-oriented students with whom many of us have to deal, we
_would_ often like to avoid those complications. Certainly dealing with
the completely _general_ case is beyond the scope of a first course, so
it's just a matter of deciding how many, and which, specific types of
cases one is willing to shoehorn into the semester (and what "previews
of coming attractions" one wishes to allude to in higher-level courses).
Seems to me the most sensible "adjustment" (and of a type we make at
least implicitly in a lot of other areas too) is
= to acknowledge that the calculations for SRS are presented
(a) for a somewhat unrealistic "ideal" kind of case,
(b) to give the neophyte _some_ experience in playing this game,
(c) to see how the variance depends (apart from the sampling scheme)
on the sample size (and on the estimated value, if one is
estimating proportions or percentages),
(d) in despite of the fact that most real sampling is carried out
under distinctly non-SRS conditions, and therefore entails
variances for which SRS calculations may be quite awry; and
= to have yet another situation for which one can point out that for
actually DOING anything like this one should first consult a
competent statistician (or, perhaps, _become_ one!).
Some textbooks I have used (cf. Moore, "Statistics: Concepts &
Controversies" (4th ed.), Table 1.1, page 40) present a table giving the
margin of error for the Gallup poll sampling procedure, as a function of
population percentage and sample size. Such a table permits one to show
how Gallup's precision varies from what one would calculate for a SRS,
thus providing some small emphasis for the cautionary tale one wishes to
convey.
------------------------------------------------------------------------
Donald F. Burrill [EMAIL PROTECTED]
184 Nashua Road, Bedford, NH 03110 603-471-7128
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
