On Fri, 04 Oct 2013 18:04:05 -0700, Miguel Roig wrote:
This blog provides an interesting point of view on that Sokal-like
demonstration: http://www.michaeleisen.org/blog/?p=1439
Miguel
(Back on TIPS after a briefly being furloughed from TIPS by some
mysterious glitch)
Good to have you back.
----- Original Message -----
On Friday, October 4, 2013 3:41:49 PM, Karl L Wuensch wrote:
Many or most of them are crap.
http://www.sciencemag.org/content/342/6154/60.full.pdf
This is a really complicated situation and I just want to make a
few points for consideration:
(1) One way to view this situation is in terms of a 2x2 table where:
Rows represent submitted manuscripts that are either (a) good or
valid or represent a sincere scientific effort and (b) bad scientific
effort or outright fraud,
and
Columns represent journals that consider (a) good or publish
high quality work and (b) journals that publish almost anything
as long as they get paid.
Ideally, I think we would prefer the combination of (row a,
column a) and avoid the combination of (row b, column b).
An argument can be made for the combination of (row a,
column b) but, depending upon journal, it is likely that articles
published in such journals will have what Stevan Hanad calls
low "research impact" -- it is where journal articles go to die.
The combination of (row b, column a), that is, bad articles
in good journals should not occur very often but the realization
that this may occur relatively often has become the concern
of professional societies and researchers; for example, see the
APS special issue on replicability in psychology; see:
http://pps.sagepub.com/content/7/6.toc
The above "model" is based on traditional, peer-reviewed journals
(e.g., the first tier journals of the APA, journals of the Psychonomics
Society, etc.) and there is the question of whether open source
journals will produce a 2x2 table with comparable frequencies or
inflated numbers in the cell (row b, column b), that is, bad papers
in bad journals. The Science article linked to above suggests that
this might be case. The response to it by Eisen, which appears
to focus on the situation in (row b, column a) suggests that maybe
most scientists underestimate how many bad research papers get
published even in the "good" journals.
(2) As I read the articles linked to above, one thought came to mind:
"What is the rejection rates for open access journals?" The Science
article suggest that these rejection rates would be very low since
even the most obvious crap gets published. For first tier APA journals,
one can get the rejection rates from here for various years:
http://www.apa.org/pubs/journals/statistics.aspx
For 2012, the Journal of Applied Psychology had the highest rejection
rate of 93% (it typically is the highest to close to the highest) -- is
this
an indication of the quality of the journal or that it is a crap magnet?
For 2012, the journal "Experimental and Clinical Psychopharmacology"
has the lowest rejection of 52%; JEP: Animal Behavior Processes
is close behind with a rejection rate of 53%. Both also have the lowest
number of manuscripts (94 and 84, respectively, vs. 853 for JAP).
Is this an indication of the quality of these journals or the preference
of
researchers to publish in journals outside of the APA
(psychopharmacological
research has more of an impact if it is published in a biomedical
journal
and is listed in the Medline/PubMed database).
The Science article really is about whether the peer review process of
open access journals works as well as that of traditional journals. One
implication of this is that bad articles should still be rejected but
good
articles may have a higher probability of being published. One would
expect a lower average rejection rate (for APA first tier journal it is
76%)
but not one close to zero which the Science article suggests might be
the
case. I guess the rejection rate of open access journals should be
readily available for comparison purposes (well, all journals should
provide this information).
(3) Stevan Harnad, long time editor of the journal "Behavioral and
Brain Science" and founder of the e-journal Psycoloquy (see his
Wikipedia entry:
http://en.wikipedia.org/wiki/Stevan_Harnad )
Harnad provides what I think is a useful review of the different types
of publication models and distinctions (e.g., publishing for "research
impact", such publishing in scientific journal without the expectation
of any immediate financial return, versus "Publishing for Income"
which covers scientific writing for magazines, textbooks, popular
press books, etc., where research impact is less important than
making a buck). See:
http://eprints.soton.ac.uk/265617/
(4) I have a experience to share: I recently wrote a book review
of an edited volume based on a conference held at a U.S. university.
It essentially contained the papers presented at the conference and
responses. In doing background research on the participants and
the topics I came across a review article by the editors that was a
pretty fair summary of the focus and scope of the book. The problem
was determining where it was published. There are two journals
with almost the exact same name except one begins with "The".
The journal without "The" has a website that provides an ISSN
number for the journal, other library relevant information, and makes
it appear much like any other academic journal. The journal
*with* "The" lacked an ISSN number, had very little information
about the publisher, and seemed unlike the website of most
journals I was familiar with. This was the journal that the editors
had chosen to publish their review article which was available
for immediate download. I am not entirely sure what to make
of this situation but the Science article has suggested some ideas
as well as resources to examine. I thought it very odd that two
journals would have such similar names and would be so easily
confused with each other.
-Mike Palij
New York University
m...@nyu.edu
---
You are currently subscribed to tips as: arch...@jab.org.
To unsubscribe click here:
http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=28491
or send a blank email to
leave-28491-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
