Re: [Wiki-research-l] Wiki Research Jounal?

2012-11-04 Thread Ed H. Chi
There has been a lot of talk about how to start a journal.  The real
issue in starting a journal is not the editorial board, or the way it
is published, or whether it will gather the citation impact.  The real
issue is READERSHIP.

If you can get people to read the journal, then it will have editors
wanting to serve the journal, and it will gather citation impact.

The reason why WikiSym is changing is for the same reason.  People are
not going to the conference!  I think the attendance has been below
100 for some time now.  That's not a sustainable number for the amount
of work that goes into organizing a conference.

---
Ed H. Chi, Staff Research Scientist, Google
CHI2012 Technical Program co-chair

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


Re: [Wiki-research-l] Wiki Research Jounal?

2012-11-04 Thread Joe Corneli
On Sun, Nov 4, 2012 at 6:14 PM, Ed H. Chi c...@acm.org wrote:
 There has been a lot of talk about how to start a journal.  The real
 issue in starting a journal is not the editorial board, or the way it
 is published, or whether it will gather the citation impact.  The real
 issue is READERSHIP.

I like this observation.  A few natural follow up questions to people
here would be:

(1) Where do you currently read about wiki research?
(2) Where do you currently publish about wiki research?
(3) What's missing?

For me:

(1) I get a surprising amount of leads from conversations that happen
on this list, and I don't pay all that much attention to where I end
up grabbing the papers from in the end.

(2) I've published at WikiSym, and in (for instance) Proceedings of
the International Conference on Computational Science, or other
subject-specific conferences/workshops.  One of these papers was
picked up and republished by Digital Education Review.  I also
contributed to a paper that was published at Alt.CHI.

(3) For me, what seems most missing is a place to talk about the
future of research in a *productive* (not necessarily scholarly)
way.  For some thoughts gleaned from this list, see
http://meta.wikimedia.org/wiki/Wiki_Research_Ideas#Rethinking_the_future_of_research
-- but where to continue things like this?  Not sure.

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


Re: [Wiki-research-l] Wiki Research Jounal?

2012-11-04 Thread Juliana Bastos Marques
Ed and others, based on your observations, I'd like to pose a side question:

The impression that I get from many of these symposia (and journals) is
that there is not much space for research concerning Wikipedia and
Education, such as teaching methodologies, case studies and such, not on
the side of hard-science chunks of data. I know of lots of other professors
who are doing the same thing as myself, but I see not many places for
exchanging our experiences (conference-wise, not online channels, which,
franky, I don't think are working much). Do you feel there is good room to
topics such as mine?

Thanks,
Juliana.


On Sun, Nov 4, 2012 at 4:14 PM, Ed H. Chi c...@acm.org wrote:

 There has been a lot of talk about how to start a journal.  The real
 issue in starting a journal is not the editorial board, or the way it
 is published, or whether it will gather the citation impact.  The real
 issue is READERSHIP.

 If you can get people to read the journal, then it will have editors
 wanting to serve the journal, and it will gather citation impact.

 The reason why WikiSym is changing is for the same reason.  People are
 not going to the conference!  I think the attendance has been below
 100 for some time now.  That's not a sustainable number for the amount
 of work that goes into organizing a conference.

 ---
 Ed H. Chi, Staff Research Scientist, Google
 CHI2012 Technical Program co-chair

 ___
 Wiki-research-l mailing list
 Wiki-research-l@lists.wikimedia.org
 https://lists.wikimedia.org/mailman/listinfo/wiki-research-l




-- 
www.domusaurea.org
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


Re: [Wiki-research-l] Wiki Research Jounal?

2012-11-04 Thread Pierre-Carl Langlais


Yes, that's a good point. Nevertheless, readership is not an inherent  
quality. It depends on several factors, most of which are  
attractiveness (is the journal layout fine to read? does the selection  
brings up insightful content?), positioning (which scientific  
disciplines are concerned?) and mediatization (how do we come to  
guarantee a specific notability of the journal per se?).


To this extent, the journal has some valuable assets :
*Us :) There have been already a lot of discussion since september. So  
far, the issue is well-debated and well-explored.

*The Wikimedian communities, that comprise many academics.
*The Open Access communities that may well be interested in this kind  
of experiment. In the specific context of the Academic Spring, the  
journal may possibly receive some comment in the general press.
*Having the advantage of Eprint (wider access) without its drawback  
(economic model…).
*International scale. I have actually given some publicity to the  
concept on my French well-read blog : http://blogs.rue89.com/les-coulisses-de-wikipedia/2012/10/23/libre-acces-les-chercheurs-defendent-leurs-travaux-et-lesprit


So far, the editorial board may have a good latitude in managing (or  
not managing) to attract readership…


PCL

Le 4 nov. 12 à 20:00, Joe Corneli a écrit :


On Sun, Nov 4, 2012 at 6:14 PM, Ed H. Chi c...@acm.org wrote:

There has been a lot of talk about how to start a journal.  The real
issue in starting a journal is not the editorial board, or the way it
is published, or whether it will gather the citation impact.  The  
real

issue is READERSHIP.


I like this observation.  A few natural follow up questions to people
here would be:

(1) Where do you currently read about wiki research?
(2) Where do you currently publish about wiki research?
(3) What's missing?

For me:

(1) I get a surprising amount of leads from conversations that happen
on this list, and I don't pay all that much attention to where I end
up grabbing the papers from in the end.

(2) I've published at WikiSym, and in (for instance) Proceedings of
the International Conference on Computational Science, or other
subject-specific conferences/workshops.  One of these papers was
picked up and republished by Digital Education Review.  I also
contributed to a paper that was published at Alt.CHI.

(3) For me, what seems most missing is a place to talk about the
future of research in a *productive* (not necessarily scholarly)
way.  For some thoughts gleaned from this list, see
http://meta.wikimedia.org/wiki/Wiki_Research_Ideas#Rethinking_the_future_of_research
-- but where to continue things like this?  Not sure.

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


Re: [Wiki-research-l] Wiki Research Jounal?

2012-11-04 Thread Dariusz Jemielniak
actually, with our community, it is not. What other journals die for, we
have sort of provided. This is why a Wiki journal may have a better chance
than others, but only if it is prepared with the academic career paths and
full proper code of conduct nuances considered (double-blind scholarly peer
review, proper editorial board, PDFs with page numbers, etc.).

dj


On Sun, Nov 4, 2012 at 7:14 PM, Ed H. Chi c...@acm.org wrote:

 There has been a lot of talk about how to start a journal.  The real
 issue in starting a journal is not the editorial board, or the way it
 is published, or whether it will gather the citation impact.  The real
 issue is READERSHIP.

 If you can get people to read the journal, then it will have editors
 wanting to serve the journal, and it will gather citation impact.

 The reason why WikiSym is changing is for the same reason.  People are
 not going to the conference!  I think the attendance has been below
 100 for some time now.  That's not a sustainable number for the amount
 of work that goes into organizing a conference.

 ---
 Ed H. Chi, Staff Research Scientist, Google
 CHI2012 Technical Program co-chair

 ___
 Wiki-research-l mailing list
 Wiki-research-l@lists.wikimedia.org
 https://lists.wikimedia.org/mailman/listinfo/wiki-research-l





-- 

__
dr hab. Dariusz Jemielniak
profesor zarządzania
kierownik katedry Zarządzania Międzynarodowego
i centrum badawczego CROW
Akademia Leona Koźmińskiego
http://www.crow.alk.edu.pl
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


Re: [Wiki-research-l] Wiki Research Jounal?

2012-11-04 Thread Ziko van Dijk
This is exactly my problem nowadays. I am a historian and don't have
much to say about software and data mining, but would like to read
more about the humanities approach with regard to Wikipedia. Readers
experiences and expectations, Wikipedia in school and university etc.
Kind regards
Ziko


2012/11/4 Juliana Bastos Marques domusau...@gmail.com:
 Ed and others, based on your observations, I'd like to pose a side question:

 The impression that I get from many of these symposia (and journals) is that
 there is not much space for research concerning Wikipedia and Education,
 such as teaching methodologies, case studies and such, not on the side of
 hard-science chunks of data.

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


Re: [Wiki-research-l] Wiki Research Jounal?

2012-11-04 Thread Kerry Raymond
So long as Google scholar is indexing the journal, I see no problem with
finding a readership.

How often do you start your quest for relevant literature by sitting down
with a set of journal titles? Yes, that's how we used to do it. But now
everyone just sits down in front of Google Scholar and types keywords and
relevant papers are found regardless of the journal they are published in.

Kerry

 


___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


Re: [Wiki-research-l] Wiki Research Journal?

2012-11-04 Thread Chitu Okoli

  
  
I'll first say that I've never been on an editorial board, so my
comments might be somewhat limited. Like my students, I learn best
when I'm shown where I'm mistaken, so I would like to learn from you
all!

On one hand, I agree that readership is very important. On the other
hand, I don't believe it is the most critical issue. I think the
most critical matter in starting a new journal is to *attract
high-quality submissions* --that is, to get researchers who do high
quality work to submit some of their best research to the journal.
If that can be achieved, then a high readership is virtually
guaranteed. Other benefits should readily follow: high citations,
ISI listing, etc.

I've been reading all the comments on this thread carefully, and I
see two distinct issues that are being mixed into one: (1) a new
journal dedicated to research about wikis, and maybe also about
related phenomena; and (2) a journal with new kind of wiki-based
peer review system. I think it is best to rather keep these issues
distinct.

First, why do we need a new journal dedicated to wiki research? I
would think we don't want a new journal that publishes mainly
low-quality research; we want a new journal that publishes
high-quality research. So, is there such a need for wiki
researchers? That is, do publishers of high-quality wiki-related
research have a hard time finding high-quality journal outlets to
publish their research? Based on the excellent wiki-based research
published in a wide variety of journals, I don't think such a
problem exists. So, why would a researcher with high-quality wiki
research risk publishing their hard work in a new, unproven, even
experimental journal? In my case, I have tenure, so I might consider
taking such risks. However, many of my colleagues are working
towards establishing their research careers, and I would definitely
advise them against publishing their best work in anything but
proven journals. 

My point is not that a new journal cannot attract high-quality
research; rather, my suspicion is that it can do so only if it is
filling a void for high-quality research on topics that are
difficult to publish in existing high-quality outlets. I'm yet to
see this issue addressed in this discussion.

Second, concerning a new kind of wikified peer review: I think that
such an experiment is very much worthwhile and should be attempted.
However, from a scientific perspective, an experiment to test
phenomenon W (wikified peer review) should control for all other
possibly confounding phenomena to make sure that the end result is
an accurate reflection of a proper test of phenomenon W. In this
case, the risks of a new journal with a poorly justified research
focus (as I argued above) is a major confound that blurs the results
of testing for W. In short, I think the best way to test wikified
peer review is to work with an existing journal that has already
established its viability and ability to attract high-quality
submissions.

The Journal of Peer Production has been mentioned as a target
candidate, and their description of their peer review philosophy
indicates that they might be quite open to such an experiment, if
not with all papers, at least with some:
http://peerproduction.net/peer-review/process/. However, it is still
a new journal, and doesn't seem to have yet reached the state of
releasing regular issues, so its newness might yet be a confound for
testing W.

~ Chitu



  Dariusz Jemielniak a crit:
  

actually, with our community, it is not. What other
  journals die for, we have sort of provided. This is why a Wiki
  journal may have a better chance than others, but only if it is
  prepared with the academic career paths and full proper code of
  conduct nuances considered (double-blind scholarly peer review,
  proper editorial board, PDFs with page numbers, etc.).
  

  
  dj
  

On Sun, Nov 4, 2012 at 7:14 PM, Ed H.
  Chi c...@acm.org
  wrote:
  There has
been a lot of talk about how to start a journal. The real
issue in starting a journal is not the editorial board, or
the way it
is published, or whether it will gather the citation impact.
The real
issue is READERSHIP.

If you can get people to read the journal, then it will have
editors
wanting to serve the journal, and it will gather citation
impact.

The reason why WikiSym is changing is for the same reason.
People are
not going to the conference! I think the attendance has
been 

Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs. single-blind review

2012-11-04 Thread Chitu Okoli

  
  
Although in theory double-blind review is superior to single-blind,
in practice double-blind vs. single-blind review has very little to
do with journal quality. It is much more a matter of disciplinary
culture. (Single-blind: authors don't know who the reviewers are,
but reviewers do know who authors are; Double-blind: neither authors
nor reviewers know who each other are)

Yes, I am certainly aware of studies that show that double-blind
reviewing does indeed reduce the bias towards publishing famous
researchers and reduces the bias against publishing work by minority
researchers and women. So, I believe that double-blind reviewing is
indeed meaningful. However, my general observation is that the
decision for a journal to be double-blind or single-blind is mainly
a matter of disciplinary tradition. To make a very gross
generalization, social science journals are generally double-blind,
whereas natural science and mathematical science journals are
generally single-blind. This observation is very relevant to this
discussion, because the wiki-research-l community sits in between
this divide. My perception is that 90% of people who post on this
list are in information systems (double-blind), computer science
(single-blind) or information science (either double- or
single-blind).

If we can accept that single-blind journals can be considered as
high-quality as well, then I feel a journal with wikified peer
review would do much better being single-blind, especially if its
subject matter is wiki-related topics. I understand that one of the
primary reasons many journals decide against going double-blind is
because of the ridiculous gymnastics that have to be undertaken in
many cases to try to hide authors' identity. In computer science,
where many researchers make their programs available on the Web, and
much of their research concerns particular websites that they have
developed, double-blinding would often be impossible if
attempted--reviewers couldn't properly evaluate the work without
knowing who created it. I think this is very much the case for a
wiki-based peer review system for much wiki-related research. 

Of course, the official reviewers should remain anonymous. (I know
that open peer review has often been proposed--authors and reviewers
know each others' identities--and has even been experimented with on
several occasions, but it does not seem to be a quality
improvement--I can post citations if anyone is interested.) It is
much easier to hide the identity of the official reviewers than it
is to hide that of the authors, and the benefits of single-blinding
are substantial and proven.

~ Chitu



  Dariusz Jemielniak a crit:
  

actually, with our community, it is not. What other
  journals die for, we have sort of provided. This is why a Wiki
  journal may have a better chance than others, but only if it is
  prepared with the academic career paths and full proper code of
  conduct nuances considered (double-blind scholarly peer review,
  proper editorial board, PDFs with page numbers, etc.).
  

  
  dj

  


___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


[Wiki-research-l] sustainability of conferences (was Re: Wiki Research Journal?)

2012-11-04 Thread James Salsman
... WikiSym is changing is for the same reason.  People are
 not going to the conference!  I think the attendance has been below
 100 for some time now.  That's not a sustainable number for the amount
 of work that goes into organizing a conference.

I would like to see an honest comparison of, for example, the reported
benefits of in-person conferences compared to their social and
economic costs. Meeting people in person is valuable, but I think it
happens more often than it needs to in most fields. Until people get
serious about organizing workflow around teleconferencing, huge and
expensive inefficiencies will persist. People love deductible junkets,
but where is the cost-benefit analysis?

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


Re: [Wiki-research-l] Wiki Research Journal?

2012-11-04 Thread Ward Cunningham
I wonder if a better place to innovate might be in the conduct of research, 
rather than the reporting, review and publication of research?

While wiki speeds collaboration within a community, the research literature 
favors long-lasting contributions outside the community. Wiki or wiki-like 
collaboration tools might be better applied to stimulating and conducting 
high-quality research that will then usefully feed existing publications.

I assume here that Wikipedia and the like have been sufficiently successful to 
raise more research questions and to enable more research methods than this 
community has time and/or ability to pursue. (If topics were scarce then they 
might be usefully hoarded.)

Of course there are many ways to share methods and directions and lots have 
already been applied in this community. However, I get the impression that 
results still take months or years to produce. 

Aside: I have built a data mining tool and methodology, Exploratory Parsing, 
that can read all of Wikipedia in 10 seconds for a useful notion of read.  I 
have also created a Federated Wiki that promotes wiki-like sharing without need 
for a common vision or agreed social norms. I intend to combine the two to make 
an experiment manager where setups are easily shared and where methodological 
improvements and/or research redirections easily build on other's work in 
progress.

What would our environment have to be like before our collaborative cycle time 
is reduced to days? 


On Nov 4, 2012, at 2:27 PM, Chitu Okoli wrote:

 On one hand, I agree that readership is very important. On the other hand, I 
 don't believe it is the most critical issue. I think the most critical matter 
 in starting a new journal is to *attract high-quality submissions* --that is, 
 to get researchers who do high quality work to submit some of their best 
 research to the journal. If that can be achieved, then a high readership is 
 virtually guaranteed. Other benefits should readily follow: high citations, 
 ISI listing, etc.
 
 I've been reading all the comments on this thread carefully, and I see two 
 distinct issues that are being mixed into one: (1) a new journal dedicated to 
 research about wikis, and maybe also about related phenomena; and (2) a 
 journal with new kind of wiki-based peer review system. I think it is best to 
 rather keep these issues distinct.

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


Re: [Wiki-research-l] Wiki Research Journal?

2012-11-04 Thread Jack Park
Wondering further, I recently became acquainted with the Sage
Bioinformatics Synapse platform:

https://synapse.sagebase.org/

by way of a keynote at the O'Reilly Strata Rx conference by the Sage
president, Stephen Friend;

http://www.youtube.com/watch?v=m4Pvq4sldbQ

A later talk (apparently not online) by Erich Huang made it clear that
their goal is to move research to the open source, open access arena.
Erich spoke in terms of putting everything from research journals, to
data, to software used for analysis online (e.g. at GitHub), making it
available for continuous peer review, evaluation, forking and
evolution, to augment the way science is communicated.

That notion seems very much in the spirit of Ward's Wiki Way.

I am still investigating Synapse. I hope to know much more about it soon.

Jack

On Sun, Nov 4, 2012 at 4:57 PM, Ward Cunningham w...@c2.com wrote:
 I wonder if a better place to innovate might be in the conduct of research,
 rather than the reporting, review and publication of research?

 While wiki speeds collaboration within a community, the research literature
 favors long-lasting contributions outside the community. Wiki or wiki-like
 collaboration tools might be better applied to stimulating and conducting
 high-quality research that will then usefully feed existing publications.

 I assume here that Wikipedia and the like have been sufficiently successful
 to raise more research questions and to enable more research methods than
 this community has time and/or ability to pursue. (If topics were scarce
 then they might be usefully hoarded.)

 Of course there are many ways to share methods and directions and lots have
 already been applied in this community. However, I get the impression that
 results still take months or years to produce.

 Aside: I have built a data mining tool and methodology, Exploratory Parsing,
 that can read all of Wikipedia in 10 seconds for a useful notion of read.
 I have also created a Federated Wiki that promotes wiki-like sharing without
 need for a common vision or agreed social norms. I intend to combine the two
 to make an experiment manager where setups are easily shared and where
 methodological improvements and/or research redirections easily build on
 other's work in progress.

 What would our environment have to be like before our collaborative cycle
 time is reduced to days?


 On Nov 4, 2012, at 2:27 PM, Chitu Okoli wrote:

 On one hand, I agree that readership is very important. On the other hand, I
 don't believe it is the most critical issue. I think the most critical
 matter in starting a new journal is to *attract high-quality submissions*
 --that is, to get researchers who do high quality work to submit some of
 their best research to the journal. If that can be achieved, then a high
 readership is virtually guaranteed. Other benefits should readily follow:
 high citations, ISI listing, etc.

 I've been reading all the comments on this thread carefully, and I see two
 distinct issues that are being mixed into one: (1) a new journal dedicated
 to research about wikis, and maybe also about related phenomena; and (2) a
 journal with new kind of wiki-based peer review system. I think it is best
 to rather keep these issues distinct.



 ___
 Wiki-research-l mailing list
 Wiki-research-l@lists.wikimedia.org
 https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


Re: [Wiki-research-l] Wiki Research Journal?

2012-11-04 Thread Steven Walling
On Sun, Nov 4, 2012 at 4:57 PM, Ward Cunningham w...@c2.com wrote:

 Aside: I have built a data mining tool and methodology, Exploratory
 Parsing, that can read all of Wikipedia in 10 seconds for a useful notion
 of read.  I have also created a Federated Wiki that promotes wiki-like
 sharing without need for a common vision or agreed social norms. I intend
 to combine the two to make an experiment manager where setups are easily
 shared and where methodological improvements and/or research redirections
 easily build on other's work in progress.

 What would our environment have to be like before our collaborative cycle
 time is reduced to days?


I would highly encourage researchers interested in collaborative systems to
take a look at Federated Wiki. Collaboration among experts is a very
interesting potential use case, considering the way the wiki handles data
and visualization, as well as the git-like way is allows for collaboration.

-- 
Steven Walling
https://wikimediafoundation.org/
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


Re: [Wiki-research-l] Wiki Research Journal?

2012-11-04 Thread Pierre-Carl Langlais
Le 5 nov. 2012 à 01:57, Ward Cunningham w...@c2.com a écrit :

 I wonder if a better place to innovate might be in the conduct of research, 
 rather than the reporting, review and publication of research?
 
 While wiki speeds collaboration within a community, the research literature 
 favors long-lasting contributions outside the community. Wiki or wiki-like 
 collaboration tools might be better applied to stimulating and conducting 
 high-quality research that will then usefully feed existing publications.

I may sound a bit overractive, but can't we do both? I would easily imagine the 
following two-way system:
*A wiki-laboratory, which hosts quick and less quick projects in progress. That 
could also include some reactive analysis by social scientists to recent 
wikimedian and collaborative knowledge phenomena.
*A wiki-journal, that should give a definitive and quotable form to the 
preceding researches and enlarge the discussions to wider disciplinary debates. 
By publishing external submissions it could give some new food for thought to 
the wiki-laboratory.
The relationships between the two structures would be a relevant analogy to the 
mutually-rewarding dialogue between the practical and theoretical side of 
scientific research.

 
 
 On Nov 4, 2012, at 2:27 PM, Chitu Okoli wrote:
 
 On one hand, I agree that readership is very important. On the other hand, I 
 don't believe it is the most critical issue. I think the most critical 
 matter in starting a new journal is to *attract high-quality submissions* 
 --that is, to get researchers who do high quality work to submit some of 
 their best research to the journal. If that can be achieved, then a high 
 readership is virtually guaranteed. Other benefits should readily follow: 
 high citations, ISI listing, etc.
 
 I've been reading all the comments on this thread carefully, and I see two 
 distinct issues that are being mixed into one: (1) a new journal dedicated 
 to research about wikis, and maybe also about related phenomena; and (2) a 
 journal with new kind of wiki-based peer review system. I think it is best 
 to rather keep these issues distinct.
 
 ___
 Wiki-research-l mailing list
 Wiki-research-l@lists.wikimedia.org
 https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs. single-blind review

2012-11-04 Thread Kerry Raymond
I think a compromise position is to use single-blind unless the authors request 
double-blind and are therefore prepared to undertake the ridiculous 
gymnastics required.

Certainly (in computer science) I would agree that it is very hard for any 
established researcher publishing in their normal field to successfully 
disguise their identify. 

Sent from my iPad

On 05/11/2012, at 8:30 AM, Chitu Okoli chitu.ok...@concordia.ca wrote:

 Although in theory double-blind review is superior to single-blind, in 
 practice double-blind vs. single-blind review has very little to do with 
 journal quality. It is much more a matter of disciplinary culture. 
 (Single-blind: authors don't know who the reviewers are, but reviewers do 
 know who authors are; Double-blind: neither authors nor reviewers know who 
 each other are)
 
 Yes, I am certainly aware of studies that show that double-blind reviewing 
 does indeed reduce the bias towards publishing famous researchers and reduces 
 the bias against publishing work by minority researchers and women. So, I 
 believe that double-blind reviewing is indeed meaningful. However, my general 
 observation is that the decision for a journal to be double-blind or 
 single-blind is mainly a matter of disciplinary tradition. To make a very 
 gross generalization, social science journals are generally double-blind, 
 whereas natural science and mathematical science journals are generally 
 single-blind. This observation is very relevant to this discussion, because 
 the wiki-research-l community sits in between this divide. My perception is 
 that 90% of people who post on this list are in information systems 
 (double-blind), computer science (single-blind) or information science 
 (either double- or single-blind).
 
 If we can accept that single-blind journals can be considered as high-quality 
 as well, then I feel a journal with wikified peer review would do much better 
 being single-blind, especially if its subject matter is wiki-related topics. 
 I understand that one of the primary reasons many journals decide against 
 going double-blind is because of the ridiculous gymnastics that have to be 
 undertaken in many cases to try to hide authors' identity. In computer 
 science, where many researchers make their programs available on the Web, and 
 much of their research concerns particular websites that they have developed, 
 double-blinding would often be impossible if attempted--reviewers couldn't 
 properly evaluate the work without knowing who created it. I think this is 
 very much the case for a wiki-based peer review system for much wiki-related 
 research. 
 
 Of course, the official reviewers should remain anonymous. (I know that open 
 peer review has often been proposed--authors and reviewers know each others' 
 identities--and has even been experimented with on several occasions, but it 
 does not seem to be a quality improvement--I can post citations if anyone is 
 interested.) It is much easier to hide the identity of the official reviewers 
 than it is to hide that of the authors, and the benefits of single-blinding 
 are substantial and proven.
 
 ~ Chitu
 
 
 Dariusz Jemielniak a écrit :
 actually, with our community, it is not. What other journals die for, we 
 have sort of provided. This is why a Wiki journal may have a better chance 
 than others, but only if it is prepared with the academic career paths and 
 full proper code of conduct nuances considered (double-blind scholarly peer 
 review, proper editorial board, PDFs with page numbers, etc.).
 
 dj
 ___
 Wiki-research-l mailing list
 Wiki-research-l@lists.wikimedia.org
 https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l