On Fri, 27 Apr 2001, Lise DeShea wrote in part:
> I teach statistics and experimental design at the University of
> Kentucky, and I give journal articles to my students occasionally with
> instructions to identify what kind of research was conducted, what the
> independent and dependent variables were, etc. For my advanced class,
> I ask them to identify anything that the researcher did incorrectly.
<snip, description of defective article>
> One of my students wrote on her homework, "It is especially hard to
> know when you are doing something wrong when journals allow bad
> examples of research to be published on a regular basis."
Mmmm. It isn't really any harder to _know_ when you're doing something
wrong; it may be somewhat more disheartening to realize that there may
be no adequate check on one's own silly mistakes, later.
I'd have pointed out to your student that one instance (possibly
selected by her professor with malice aforethought? -- and even if not,
the student wouldn't necessarily know that) hardly supports the phrase
"published on a regular basis". Just emphasizes the need to maintain a
healthy skepticism, and to be prepared to proofread with a critical eye.
(Just 'cause it's printed doesn't mean it's true...)
> I'd like to hear what other list members think about this problem and
> whether there are solutions that would not alienate journal editors.
Not to mention one's (you should pardon the expression) colleagues.
Depends partly on sensitivity of editors and/or authors to criticism.
Mainly, as TR once put it, speak softly (i.e., politely) and carry a
big stick (i.e., evidence that, even if politely phrased, clearly
illuminates the fact of an error). But it is worth remembering that
journal editors (at least, the ones I've known) are editors only for
limited terms: three years is not unusual, I think, and while an editor
may be reappointed for a subsequent (second, third, ...) term, it seems
to be more usual to serve for two terms and then let somebody else do it.
So even if you get off on a wrong foot with one editor, that misfortune
needn't carry over to the next editor.
Some years back I encountered a systematic error in a journal article.
The author had reported total scores from a series of Likert-like items,
and showed a histogram. The histogram displayed decided "spikes", about
twice as high as the surrounding landscape, at regular intervals: scores
of 20, 25, 30, 35, apparently. (Maximum score was 40, minimum 10.)
These were so interesting that the author spent a page or more
intepreting them (as the results of "patterned responses" by the
respondents, by which was meant responding with all 3's (e.g.) to all
items). And indeed, if such patterning were present to any great degree,
it would have showed up in just this way.
Only thing was, the histogram program used had been allowed to
set its own parameters, and in the range of, say, 20 to 30, where there
should have been ten scores, there were only eight histogram bars. The
"spikes" were of course the bars that contained two scores: 20 and 21,
25 and 26, 30 and 31, etc.
First thing I did was write to the author. Wasn't polite enough,
I guess (although I was trying to be), because he never acknowledged my
letter. Then I e-mailed the editor, who wanted a response from the
author before he took any action (which I thought reasonable enough),
and suggested that I write a letter to the editor identifying the
problem, which he'd then ask the author to reply to. Various things
intervened about then, and I never got that letter written, I'm afraid.
But I've frequently used that article as an example in class (usually
presenting it as a puzzle, to see if anyone is sharp-eyed enough to see
what's wrong, and usually presenting only the histogram and the relevant
paragraph or two in the article). Helps to illustrate the points
reported above: be skeptical, and sharp-eyed. And I take the
opportunity to point out that this error, obvious as it is once one has
seen it, eluded the author, the audience at the AERA session where the
paper was presented, the audience at a European meeting where it was
presented, at least two associate editors (that journal routinely farms
papers out to at least two readers before publishing), and the journal
editor himself. (And, presumably, most of the journal readership --
I never saw a critical letter from anyone else on this point.)
<snip, various economic concerns>
(Of course, you could always suggest that your _student_ to write a naive
little letter to the author, asking naive little questions...)
-- DFB.
------------------------------------------------------------------------
Donald F. Burrill [EMAIL PROTECTED]
348 Hyde Hall, Plymouth State College, [EMAIL PROTECTED]
MSC #29, Plymouth, NH 03264 603-535-2597
184 Nashua Road, Bedford, NH 03110 603-472-3742
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
