>Robert Dawson wrote:
>
>> As far as random samples are concerned: it is *very* rare for a true
>> random sample, based on an equal-probability sample of the population to
>> which the inference is intended to extend, to be taken. Say a researcher is
>> studying the behaviour of humans. (S)he may take a random sample from the
>> student subject pool, but not from the human race; and yet the paper
>> published will claim to be about "Artificially Inducing The Gag Reflex in
>> Humans", not "Artificially Inducing The Gag Reflex in Students Enrolled in
>> Psych 1000 at Miskatonic U. (Fall '00)".
well, perhaps journal editors should INSIST that the author say very
clearly ... that this only applies to students enrolled in psy 1000 at
miskatonic u ... fall 1999 ... since that is what it is ...
the only way we can get around this ... is to REPLICATE investigations and
see if we can find comparable results across disparate subject pools ..
but, unfortunately ... if you do like benton j underwood did many years
ago: studies in the meaningfulness of learning 1 ... then 2 ... then ... 29
... your tenure would be 'on hold' ... you are not allowed to replicate 10
times ... you must move onward and upward ...
we would be MUCH better off ... reducing drastically the NUMBER of things
we tried to be unique in (that no one else has done) ... and spend more
time replicating work ... that is deemed to be MORE important ... in the
long run ... our knowledge base would be better and stronger ... rather
than relying on some p value to suggest that THIS has been researched and
THE answer found ... now we should move on to something else ... just
another weight place on the poor little p ... when its back is already
crushing!
the more i think about it ... the more i think our overall effort is
misguided ... and this is but one reason why there is so much crappy
research ... and believe me ... there is plenty to go around across all the
disciplines
Even if some future world
>> government were to allow researchers access to a list of all humans alive at
>> some moment to use as a sampling frame, most researchers would not disclaim
>> any applicability of their research to those dead or not yet born. The
>> implicit "Platonic" population larger than that available for study is a
>> problem that is always with us; a bad sample is one in which this causes
>> bias. The situation in which the entire actual population is available for
>> study is an extreme case, of course.
i would suggest that inferential statistics .. as we know it ... is not
robust to cruddy samples and if samples are cruddy ... what's the point in
using some standard error that is BASED on the assumption that samples are
NOT cruddy ... but rather, have some connection to random error ...
you can't have it both ways ... either we make a good faith effort to
sample in a reasonble way .. such that our standard errors can be expected
to be about right ... or, we say that samples won't be good ... and if that
is true ... forget the notion of using the standard error in some rigid way
for making hypothesis tests ... confidence intervals ... and the like
when we use such error estimates as: stan error of the mean = S / sqrt n ..
does this apply no matter what?
there is a daisy chain here ... the hypothesis is about a population ...
and, we use the data from our sample to make a decision about that
hypothesized parameter ... BUT, if our sample cannot be considered (within
some fudge factor) to be representative of the population to which we have
made this hypothetical stab ... seems like we need to pack it in
let's say that we take a sample by any means ... and, the question we have
formulated is that .. in the population ... we will find some 6's ... AND,
we happen to find 1 or more 6's ... now, i don't care how you took the
sample ... good way or bad way ... we have confirmed our question ... but,
what if you don't find any 6's??? i would say in this case ... you are up
the creek ... since there is no model we can apply given we know nothing
about how the sample was taken ...
if we have to assume that samples can be anything ... since we can never
EXACTLY get a truly random sample ... then we are in a peck of trouble ...
i recall a number of posts that alan made ... arguing rather vehemently
about the fact that we need a model for our data ... well, what is the
model for our data if we have no control over our sampling ... nor any way
to have a crack at estimating the error BASED on that sample information?
but now ... in telling robert ... spot on ... in the context that robert is
implying that it is ok to go ahead and make these inferences even if our
sampling methods are poor ... so, the way i read this ... alan is more or
less agreeing with that .. and that does not appear to be a very consistent
approach to things ...
bottom line: how goes your samples ... that's where your inferences are headed
but that is just my read of it
>>
>> -Robert Dawson
>>
>> ===========================================================================
>> This list is open to everyone. Occasionally, less thoughtful
>> people send inappropriate messages. Please DO NOT COMPLAIN TO
>> THE POSTMASTER about these messages because the postmaster has no
>> way of controlling them, and excessive complaints will result in
>> termination of the list.
>>
>> For information about this list, including information about the
>> problem of inappropriate messages and information about how to
>> unsubscribe, please see the web page at
>> http://jse.stat.ncsu.edu/
>> ===========================================================================
>
>--
>Alan McLean ([EMAIL PROTECTED])
>Department of Econometrics and Business Statistics
>Monash University, Caulfield Campus, Melbourne
>Tel: +61 03 9903 2102 Fax: +61 03 9903 2007
>
>
>
>
>===========================================================================
>This list is open to everyone. Occasionally, less thoughtful
>people send inappropriate messages. Please DO NOT COMPLAIN TO
>THE POSTMASTER about these messages because the postmaster has no
>way of controlling them, and excessive complaints will result in
>termination of the list.
>
>For information about this list, including information about the
>problem of inappropriate messages and information about how to
>unsubscribe, please see the web page at
>http://jse.stat.ncsu.edu/
>===========================================================================
==============================================================
dennis roberts, penn state university
educational psychology, 8148632401
http://roberts.ed.psu.edu/users/droberts/droberts.htm
===========================================================================
This list is open to everyone. Occasionally, less thoughtful
people send inappropriate messages. Please DO NOT COMPLAIN TO
THE POSTMASTER about these messages because the postmaster has no
way of controlling them, and excessive complaints will result in
termination of the list.
For information about this list, including information about the
problem of inappropriate messages and information about how to
unsubscribe, please see the web page at
http://jse.stat.ncsu.edu/
===========================================================================
