Omissions in Journal Articles
There was a recent discussion here of errors in journal articles. A
related topic is incomplete information, or at least what I consider
incomplete information.
A recent article in the American Journal of Epidemiology (2001, Vol 153,
No 6, 596-603) contains some nicely laid out (and badly titled, but save
that for another thread) tables that show the mean value of a dependent
variable in male and female subsamples, broken down by a dichotomous
independent variable (exercise? Yes/No), and "adjusted" first by age
and then by age and several other numerical variables.
In a sociological journal I would most often see this analysis reported
in multiple regression form, but again, that's another thread. In health
related journals the convention is to speak of adjusted means, i.e. the
predicted dependent variable value for members of each category of the
independent variable, with the other predictors ("covariates") set to
specific values.
The article does not specify what values of age (and the other
predictors) were used to create the adjusted means. Instead there is a
footnote to the table that says:
"Adjusted means calculated by using analysis of covariance."
My question, directed to those of you who are more familiar with
journals in this area than I am, is whether this is a standard footnote
/ explanation, which is supposed to make clear to regular readers what
has been done?
>From my perspective it is inadequate, since the ANCOVA (or regression
analysis) has merely produced a predictive formula, and any values
whatsoever of the covariates could be plugged in to the equation.
Now I happen to know what SPSS v10 does when asked to produce
"estimated" means in its univariate GLM procedure: it plugs in the mean
values. The output actually contains the values of those means for the
record. (Is this true with other statistical software?) A user who knows
the syntax can actually specify the values, but the Windows point+click
screen doesn't allow that. Here's an example of the default subcommand
statement:
/EMMEANS = TABLES(exercise) WITH(age=MEAN xother=MEAN)
Using the mean can produce misleading adjusted values, especially when
the table contains subsample comparisons as in this article, where "all
analyses were sex-specific". If the default SPSS ANCOVA were followed,
the adjusted values would be created at different values of age, since
mean age differs by sex, and other covariate means are also different in
this study. (In addition, different tables contained analyses of 2 other
subgroups of the sample, with different mean Xs.)
This may or may not be a problem for the authors' interpretation of the
results, but it seems reasonable to expect editors to be more sensitive
to their readers' need to know exactly what is going on.
--
**
`o^o' * Neil W. Henry ([EMAIL PROTECTED]) *
-<:>- * Virginia Commonwealth University *
_/ \_ * Richmond VA 23284-2014 **
* http://www.people.vcu.edu/~nhenry *
***
=
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=
Re: Random Sampling and External Validity
--9F3BD71D2EDA80683B2FF9EA Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit dennis roberts wrote: > At 12:56 PM 3/25/01 -0500, Karl L. Wuensch wrote: > >Here is how I resolve that problem: Define the population from the sample, > >rather than vice versa -- that is, my results can be generalized to any > >population for which my sample could be reasonably considered to be a random > >sample. Maybe we could call this "transcendental sampling" ;-) -- it is > >somewhat like transcendental realism, defining reality from our percetion of > >it, eh? > > this sounds like the method of grounded theory in the qualitative bailiwick > ... > look at data you have and see what you can make of it > > that is ... there is no particular PLAN to the investigation ... data > gathering ... or, what you want to do with what you find after the fact > > i try to tell students this is not a very good strategy ... Dennis: This is a serious misrepresentation of qualitative research in general, and, in particular, of Glaser and Strauss's insistence that social theorizing be fundamentally be "grounded" in experience. The idea that qualitative research begins with "no particular PLAN" is simply ludicrous. Here's a couple of quotes from "Toward Reform of Program Evaluation" by Lee Cronbach and others (1980): "The evaluator will be wise not to declare allegiance to either a quantitative-manipulative-summative methodology or a qualitative-naturalistic-descriptive methodology. . . . Those who advocate an evaluation plan [sic] devoid of one kind of information or the other carry the burden of justifying such exclusion." (p. 223) "Writings on experimental design emphasize extrapolation much less than is appropriate for evaluation. Data on a particular set of program operations are collected at specific sites, but the evaluation is intended to answer a much broader question." . . . The design best for sharpening a limited statistical conclusion may not be the best basis for the broader inferences. " (p. 231) Introductory statistics classes, with their artificially created null hypotheses and impractical data gathering designs, often ignore these complexities. -- * `o^o' * Neil W. Henry ([EMAIL PROTECTED]) * -<:>- * Virginia Commonwealth University * _/ \_ * Richmond VA 23284-2014 * * http://www.people.vcu.edu/~nhenry * * --9F3BD71D2EDA80683B2FF9EA Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit dennis roberts wrote: At 12:56 PM 3/25/01 -0500, Karl L. Wuensch wrote: >Here is how I resolve that problem: Define the population from the sample, >rather than vice versa -- that is, my results can be generalized to any >population for which my sample could be reasonably considered to be a random >sample. Maybe we could call this "transcendental sampling" ;-) -- it is >somewhat like transcendental realism, defining reality from our percetion of >it, eh? this sounds like the method of grounded theory in the qualitative bailiwick ... look at data you have and see what you can make of it that is ... there is no particular PLAN to the investigation ... data gathering ... or, what you want to do with what you find after the fact i try to tell students this is not a very good strategy ... Dennis: This is a serious misrepresentation of qualitative research in general, and, in particular, of Glaser and Strauss's insistence that social theorizing be fundamentally be "grounded" in experience. The idea that qualitative research begins with "no particular PLAN" is simply ludicrous. Here's a couple of quotes from "Toward Reform of Program Evaluation" by Lee Cronbach and others (1980): "The evaluator will be wise not to declare allegiance to either a quantitative-manipulative-summative methodology or a qualitative-naturalistic-descriptive methodology. . . . Those who advocate an evaluation plan [sic] devoid of one kind of information or the other carry the burden of justifying such exclusion." (p. 223) "Writings on experimental design emphasize extrapolation much less than is appropriate for evaluation. Data on a particular set of program operations are collected at specific sites, but the evaluation is intended to answer a much broader question." . . . The design best for sharpening a limited statistical conclusion may not be the best basis for the broader inferences. " (p. 231) Introductory statistics classes, with their artificially created null hypoth
Re: Is there a test for causality?
"G. Anthony Reina" wrote:
> Is there a test that variable X causes variable Y?
No, not in the abstract general way you pose the question
> I was under the impression that the best statistics could do was
> correlation not causation. In order to prove causation, one would have
> to know the specific mechanism whereby X could cause Y and possibly vary
> the input X to see if Y changed accordingly.
Under such a situation "statistics" comes into play in the analysis of the
data, which tend not to be able to speak for themselves.
>
> However, I've seen some papers on a method called 'directed coherence'
> which uses something called Granger causality. I think the basic gist is
> that the 'directed coherence' is the probability of predicting something
> about Y given you know something about X.
>
The phrase "Granger Causality" is used by econometricians, and derives from
a 1969 article in Econometrica by someone named --- you probably guessed
already --- Granger. The title will give you an idea of his approach:
"Investigating causal relations by econometric models and cross-spectral
methods."
Mainstream statisticians tend to ignore the extensive literature on
causation. It is almost never mentioned in textbooks even at the graduate
level. For an overview of what some philosophers, historians and
statisticians have to say on the subject, look up
Causality in crisis? : statistical methods and the search for
causal knowledge in the social sciences edited by Vaughn R. McKim
and Stephen P. Turner.
Published: Notre Dame, Ind. : University of Notre Dame Press,
1997.
Notes: Essays derived from presentations at the conference held
in October 1993 at the University of Notre Dame.
The philosopher Clark Glymour and his collaborators claim to have developed
techniques for discovering causal structure in correlational data. Others
whose work you might look into include James Heckman ("selection modeling")
, Jamie Robins and Patrick Suppes ("probabilitic theory of
causality").
>
> Has anyone run across this?
>
> Thanks.
> -Tony Reina
--
*
`o^o' * Neil W. Henry ([EMAIL PROTECTED]) *
-<:>- * Virginia Commonwealth University *
_/ \_ * Richmond VA 23284-2014 *
*(804)828-1301 x124 *
*FAX: 828-8785 http://www.people.vcu.edu/~nhenry *
*
=
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=
