[Wiki-research-l] BBC article: Wiki wars: Do Wikipedia's internal tiffs deter newcomers?

[Chitu Okoli]

Hi everyone,

I just read an article from BBC titled, "Wiki wars: Do Wikipedia's 
internal tiffs deter newcomers?"

http://www.bbc.com/news/technology-28426674

It introduces the general public to Wikipedia edit wars, inclusionism 
vs. deletionism, talk page battles, the male editor overrepresentation, 
and Wikipedia's organized anarchy. I found it a generally accurate 
treatment of a delicate topic, and it's filled with lots of great, 
specific examples.


Regards,
Chitu Okoli

---
Associate Professor in Business Technology Management
John Molson School of Business, Concordia University, Montréal
http://chitu.okoli.org/pro



___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Wikipedia reviews published on content and on readership

[Chitu Okoli]

Hi everyone,

We have two Wikipedia literature reviews recently accepted for 
publication, both at the Journal of the American Society for Information 
Science and Technology. Open access versions are available on the 
Concordia University institutional repository:


* Wikipedia in the eyes of its beholders: A systematic review of 
scholarly research on Wikipedia readers and readership 
(https://spectrum.library.concordia.ca/978617/). This article reviews 
studies on ranking and popularity; Wikipedia as a knowledge source; 
student readership; and commercial aspects of Wikipedia, among other 
topics.


* “The sum of all human knowledge”: A systematic review of scholarly 
research on the content of Wikipedia 
(https://spectrum.library.concordia.ca/978618/). This article reviews 
studies on the quality of Wikipedia (including reliability, 
comprehensive, and antecedents to quality) and the size of Wikipedia.


These are part of our larger literature review on Wikipedia (working 
paper at http://ssrn.com/abstract=2021326; online database of studies at 
http://wikilit.referata.com). We are currently working on detailed, 
focused review papers on other important Wikipedia research topics like 
motivations to participation, collaborative culture, Wikipedia as a 
textual corpus, and other topics.


Unfortunately, because of the tremendous breadth of the topic, the 
reviews mainly cover journal articles and doctoral theses (with several 
important conference papers) up to 2011-2012.


We would very much appreciate your comments and feedback on the two 
accepted papers, and on the working paper with the other topics.


Regards,
Chitu Okoli
Mohamad Mehdi
Mostafa Mesgari
Finn Årup Nielsen
Arto Lanamäki

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] different language Wikipedias and knowledge migration

[Chitu Okoli]

Dear Lior,

The WikiLit website has a category on "Cultural and linguistic effects 
on participation": 
http://wikilit.referata.com/wiki/Category:Cultural_and_linguistic_effects_on_participation. 
You might find some useful studies among the 10 listed there.


Regards,
Chitu


Sujet:
[Wiki-research-l] different language Wikipedias and knowledge migration
De:
lior gimel mailto:lior.gi...@gmail.com>>
Date :
Wed, 17 Jul 2013 14:06:39 +0300

Pour:
Research into Wikimedia content and communities 
>



Dear all,
I'm currently writing a research proposal focused on Wikipedia as a case 
study in cross-cultural migration of epistemic cultures, and I need some 
help on the lit. review part.
I'm looking for studies dealing directly with the attempt to adopt 
policies from one language version to another (such as the attempt to 
introduce flagged revisions on the English Wikipedia), but also for ones 
comparing between standards and policies in different language Wikipedias.
I would greatly appreciate any help, as I only found a handful of 
possibly relevant articles.

Best,
Lior

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs.single-blind review

[Chitu Okoli]

  
  
I know this is an old branch of this topic, but since it generated
quite some interest on this list, I thought I'd share this excellent
article I recently found on double-blind vs. single-blind reviewing.
It addresses most of the issues that were discussed in this
sub-thread.

In brief, the editor of a prominent computer science journal
recounts his exploration on which way to go with his journal. The
journal decided to go double-blind because of its benefits (mainly
protecting less powerful researchers), but his journal has a very
detailed set of instructions (in the appendix) that do an excellent
job addressing computer science projects that are difficult to
anonymize, which I think are very much applicable to a wiki-based
journal as discussed in this thread.


  Snodgrass, Richard T. 2007. Editorial:
Single- versus double-blind reviewing. ACM Trans. Database
  Syst. 32, 1.

  

If you don't have access to the ACM Digital Library, you can get a
PDF of the article from Google Scholar.

~ Chitu

  


___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Wiki Research Journal? - Conference vs. journal publication

[Chitu Okoli]

  
  
I can't speak for every field, but at least for my own field of
information systems, where conferences count for zero, at least
among the most research-intensive universities:

Counting conference publications or not is in no way a judgment
either way of the quality of the papers. In information systems, it
is well known that some high-quality conferences (such as ICIS,
HICSS and AoM-OCIS) regularly field higher quality papers than many
journals. However, such publication often counted as zero in
promotion and tenure considerations.

What is going on is that in our field (and I suspect also in similar
fields) *conferences are not considered terminal publication
outlets--only journals are*. That is, when you present a paper in a
conference, even when it is published in the proceedings, you are
expected to later publish a significantly revised and significantly
extended version of that paper in a journal article (and I would
guess that in 90% of the time, this is what happens, at least for
high-quality papers). A high-quality conference paper is expected to
yield a high-quality journal article. Thus, *to avoid
double-counting*, conference publications are ignored in promotion
and tenure considerations.

From what I understand, in fields like computer science where
conferences are terminal publication outlets (that is, conference
papers are often not republished in journals), then it naturally
makes sense that the conference papers should be considered the
measure of a researcher's productive quality.

~ Chitu



  Aaron Halfaker a écrit :
  


  
  
  As for disciplines that do not count conference papers, I
cannot comment because my discipline (Computer Science) looks at
top tier conference publications in a similar way to journal
publications.  However, I'd argue that anyone who does not value
a publication purely because the venue is called a "conference"
regardless of the impact/restrictiveness is making a mistake.
 I've seen people include the acceptance rates on their CV to
avoid this situation.
  
  
  -Aaron
  


  


___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs. single-blind review

[Chitu Okoli]

  
  
Here are some references about the pros and cons of double-blind
peer review:

* Book: Editorial Peer Review: Its Strengths & Weaknesses by Ann
C. Weller [1]. This book covers not only double-blind peer review,
but empirical studies about all kinds of peer review (including open
peer review, where even the reviewers are not anonymous). This book,
and papers it summarizes, is my primary source of information on
this topic. If your library doesn't have it, you could ask them to
get it (that's how I got a hold of it for myself).

* Nature magazine report on an international survey about peer
review [2]. Highlights pertinent to this discussion: 
- "Support [for double-blind peer review] is highest with those who
have experienced it (the humanities and social sciences) or *where
it is perceived to do the most good (among female authors)*. The
least enthusiastic group is editors."
- "The one bright light in favour of double-blind peer review is the
measured reduction in bias against authors with female first names
(shown in numerous studies, such as ref. 4).

This suggests that authors submitting papers to traditionally minded
journals should include the given names of authors only on the
final, published version."
- "The double-blind approach is predicated on a culture in which
manuscripts-in-progress are kept secret. This is true for the most
part in the life sciences. But some physical sciences, such as
high-energy physics, share preprints extensively through arXiv, an
online repository. *Thus, double-blind peer review is at odds with
another 'force for good' in the academic world: the open sharing of
information.* The PRC survey found that highly competitive fields
(such as neuroscience) or those with larger commercial or applied
interests (such as materials science and chemical engineering) were
the most enthusiastic about double-blinding, whereas fields with
more of a tradition for openness (astronomy and mathematics) were
decidedly less supportive."

* Two open discussions on Nature magazine blogs about double-blind
peer review from 2005 [3] and 2008 [4]. The 2008 discussion was in
response to the editorial mentioned above.

~ Chitu


[1]
http://www.amazon.ca/Editorial-Peer-Review-Strengths-Weaknesses/dp/1573871001
[2]
http://www.nature.com/nature/journal/v451/n7179/full/451605b.html
[3]
http://blogs.nature.com/actionpotential/2005/12/doubleblind_peer_review.html
[4]
http://blogs.nature.com/peer-to-peer/2008/02/working_doubleblind.html




  Manuel Palomo Duarte a écrit :
  

The woman discrimination is something the journal
  should care about. Any idea on how to face it?
  


  


___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs. single-blind review

[Chitu Okoli]

  
  
Here are a few scenarios:

* The research topic concerns a public website. The website
identifies the authors. The paper makes no sense without explicitly
identifying the website. Thus, authors should be able to request
single-blind review. Note that this scenario very much applies to
this entire discussion of a new research journal that uses
wiki-based research development. I don't know if you caught Kerry
Raymond's comment on this thread (I copy it below), which explains
this point very succinctly.

* Authors have posted a working paper which has been on the web for
a long time, and is known to most researchers in that field of
interest (i.e. most potential and qualified reviewers for the
peer-reviewed version). In this case, I would think that reviewers
should not be excluded for no reason other than they know the
authors' identity. One of the most backward policies I've ever seen
related to this is JIBS's policy to protect double-blind review:
"Authors should also not post their submitted manuscript (including
working papers and prior drafts) on websites where it could be
easily discovered by potential reviewers." [1] Apparently, they
consider double-blind review a more sacred ideal than early
dissemination of research through working papers.

* The research critically involves a multimedia artifact, such as a
video, that cannot be easily be submitted as supporting materials
for peer review. The video is better posted on a website. Here's a
case of requested "gymnastics" I've seen in order to protect
double-blind peer review even in such cases: "We ask each author to
create his/her own account with an open access provider of choice
(e.g., linked video could be hosted in Vimeo or YouTube).  Please
use a pseudo user name in order to maintain anonymity during the
review process." [2]

Although I do believe in the benefits of double-blind review (I'll
send a separate post with a few citations), in my own research I am
increasingly confronted with the fact that new approaches to
research that favour openness and mass collaboration are
fundamentally in conflict with the idea of anonymity in the identity
of the authors of a manuscript submitted for peer review.
Personally, I prefer to forge ahead with innovative modes of
research conduct, even if double-blind review is sacrificed. For me,
a perfect compromise is to default to double-blind, but fall back to
single-blind when the nature of the research project calls for it.

~ Chitu 


[1]
http://www.palgrave-journals.com/jibs/author_instructions.html#Ethical-guidelines
[2] http://icis2011.aisnet.org/Paper_Submission.html#B



  Dariusz Jemielniak a écrit :
  

just out of curiosity, what could be the reasonable
  expected purposes for requesting a single-blind review instead of
  a standard double-blind in your model? 
  
  
  best,
  
  
  dj
  


On Tue, Nov 6, 2012 at 3:56 AM, Chitu
  Okoli <chitu.ok...@concordia.ca>
  wrote:
  
 Actually, I think it
  is more reasonable to use double-blind by default unless
  authors request single-blind. If single-blind were the
  default, it would be difficult to request double-blind as
  exceptions:
  
  * If there is a "big name" researcher who wants to take
  advantage of his/her reputation, he/she would not request
  double-blind.
  * If there is a "big name" researcher who is modest and
  does not think highly of himself/herself, he/she would not
  request double-blind.
  * If there is a minority or woman researcher afraid of
  discrimination, if he/she requested double-blind, the
  reviewers would reasonably guess that the author(s) are
  minorities or women.
  
  Thus, I think double-blind as a default for everyone with
  single-blind as special exception would be the more
  practical and fairer general policy. With the increase of
  preprints and working papers (e.g. arXiv and SSRN), I
  think author anonymity is becoming increasingly
  impractical.
  
  In any case, these comments mainly apply to social science
  journals; I still think that single-blind makes more sense
  for computer science journals.
  
  ~ Chitu
  

  

  



  Kerry Raymond a écrit :
  

I would note that the use of 

Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs. single-blind review

[Chitu Okoli]

  
  
Actually, I think it is more reasonable to use double-blind by
default unless authors request single-blind. If single-blind were
the default, it would be difficult to request double-blind as
exceptions:

* If there is a "big name" researcher who wants to take advantage of
his/her reputation, he/she would not request double-blind.
* If there is a "big name" researcher who is modest and does not
think highly of himself/herself, he/she would not request
double-blind.
* If there is a minority or woman researcher afraid of
discrimination, if he/she requested double-blind, the reviewers
would reasonably guess that the author(s) are minorities or women.

Thus, I think double-blind as a default for everyone with
single-blind as special exception would be the more practical and
fairer general policy. With the increase of preprints and working
papers (e.g. arXiv and SSRN), I think author anonymity is becoming
increasingly impractical.

In any case, these comments mainly apply to social science journals;
I still think that single-blind makes more sense for computer
science journals.

~ Chitu



  Kerry Raymond a écrit :
  


  I think a compromise position is to use single-blind unless the authors request double-blind and are therefore prepared to undertake the "ridiculous gymnastics" required.

Certainly (in computer science) I would agree that it is very hard for any established researcher publishing in their normal field to successfully disguise their identify. 

Sent from my iPad

On 05/11/2012, at 8:30 AM, "Chitu Okoli"  wrote:


  
Although in theory double-blind review is superior to single-blind, in practice double-blind vs. single-blind review has very little to do with journal quality. It is much more a matter of disciplinary culture. (Single-blind: authors don't know who the reviewers are, but reviewers do know who authors are; Double-blind: neither authors nor reviewers know who each other are)

Yes, I am certainly aware of studies that show that double-blind reviewing does indeed reduce the bias towards publishing famous researchers and reduces the bias against publishing work by minority researchers and women. So, I believe that double-blind reviewing is indeed meaningful. However, my general observation is that the decision for a journal to be double-blind or single-blind is mainly a matter of disciplinary tradition. To make a very gross generalization, social science journals are generally double-blind, whereas natural science and mathematical science journals are generally single-blind. This observation is very relevant to this discussion, because the wiki-research-l community sits in between this divide. My perception is that 90% of people who post on this list are in information systems (double-blind), computer science (single-blind) or information science (either double- or single-blind).

If we can accept that single-blind journals can be considered as high-quality as well, then I feel a journal with wikified peer review would do much better being single-blind, especially if its subject matter is wiki-related topics. I understand that one of the primary reasons many journals decide against going double-blind is because of the ridiculous gymnastics that have to be undertaken in many cases to try to hide authors' identity. In computer science, where many researchers make their programs available on the Web, and much of their research concerns particular websites that they have developed, double-blinding would often be impossible if attempted--reviewers couldn't properly evaluate the work without knowing who created it. I think this is very much the case for a wiki-based peer review system for much wiki-related research. 

Of course, the official reviewers should remain anonymous. (I know that open peer review has often been proposed--authors and reviewers know each others' identities--and has even been experimented with on several occasions, but it does not seem to be a quality improvement--I can post citations if anyone is interested.) It is much easier to hide the identity of the official reviewers than it is to hide that of the authors, and the benefits of single-blinding are substantial and proven.

~ Chitu


  


  


___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs. single-blind review

[Chitu Okoli]

  
  
Although in theory double-blind review is superior to single-blind,
in practice double-blind vs. single-blind review has very little to
do with journal quality. It is much more a matter of disciplinary
culture. (Single-blind: authors don't know who the reviewers are,
but reviewers do know who authors are; Double-blind: neither authors
nor reviewers know who each other are)

Yes, I am certainly aware of studies that show that double-blind
reviewing does indeed reduce the bias towards publishing famous
researchers and reduces the bias against publishing work by minority
researchers and women. So, I believe that double-blind reviewing is
indeed meaningful. However, my general observation is that the
decision for a journal to be double-blind or single-blind is mainly
a matter of disciplinary tradition. To make a very gross
generalization, social science journals are generally double-blind,
whereas natural science and mathematical science journals are
generally single-blind. This observation is very relevant to this
discussion, because the wiki-research-l community sits in between
this divide. My perception is that 90% of people who post on this
list are in information systems (double-blind), computer science
(single-blind) or information science (either double- or
single-blind).

If we can accept that single-blind journals can be considered as
high-quality as well, then I feel a journal with wikified peer
review would do much better being single-blind, especially if its
subject matter is wiki-related topics. I understand that one of the
primary reasons many journals decide against going double-blind is
because of the ridiculous gymnastics that have to be undertaken in
many cases to try to hide authors' identity. In computer science,
where many researchers make their programs available on the Web, and
much of their research concerns particular websites that they have
developed, double-blinding would often be impossible if
attempted--reviewers couldn't properly evaluate the work without
knowing who created it. I think this is very much the case for a
wiki-based peer review system for much wiki-related research. 

Of course, the official reviewers should remain anonymous. (I know
that open peer review has often been proposed--authors and reviewers
know each others' identities--and has even been experimented with on
several occasions, but it does not seem to be a quality
improvement--I can post citations if anyone is interested.) It is
much easier to hide the identity of the official reviewers than it
is to hide that of the authors, and the benefits of single-blinding
are substantial and proven.

~ Chitu



  Dariusz Jemielniak a écrit :
  

actually, with our community, it is not. What other
  journals die for, we have sort of provided. This is why a Wiki
  journal may have a better chance than others, but only if it is
  prepared with the academic career paths and full proper code of
  conduct nuances considered (double-blind scholarly peer review,
  proper editorial board, PDFs with page numbers, etc.).
  

  
  dj

  


___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Wiki Research Journal?

[Chitu Okoli]

  
  
I'll first say that I've never been on an editorial board, so my
comments might be somewhat limited. Like my students, I learn best
when I'm shown where I'm mistaken, so I would like to learn from you
all!

On one hand, I agree that readership is very important. On the other
hand, I don't believe it is the most critical issue. I think the
most critical matter in starting a new journal is to *attract
high-quality submissions* --that is, to get researchers who do high
quality work to submit some of their best research to the journal.
If that can be achieved, then a high readership is virtually
guaranteed. Other benefits should readily follow: high citations,
ISI listing, etc.

I've been reading all the comments on this thread carefully, and I
see two distinct issues that are being mixed into one: (1) a new
journal dedicated to research about wikis, and maybe also about
related phenomena; and (2) a journal with new kind of wiki-based
peer review system. I think it is best to rather keep these issues
distinct.

First, why do we need a new journal dedicated to wiki research? I
would think we don't want a new journal that publishes mainly
low-quality research; we want a new journal that publishes
high-quality research. So, is there such a need for wiki
researchers? That is, do publishers of high-quality wiki-related
research have a hard time finding high-quality journal outlets to
publish their research? Based on the excellent wiki-based research
published in a wide variety of journals, I don't think such a
problem exists. So, why would a researcher with high-quality wiki
research risk publishing their hard work in a new, unproven, even
experimental journal? In my case, I have tenure, so I might consider
taking such risks. However, many of my colleagues are working
towards establishing their research careers, and I would definitely
advise them against publishing their best work in anything but
proven journals. 

My point is not that a new journal cannot attract high-quality
research; rather, my suspicion is that it can do so only if it is
filling a void for high-quality research on topics that are
difficult to publish in existing high-quality outlets. I'm yet to
see this issue addressed in this discussion.

Second, concerning a new kind of wikified peer review: I think that
such an experiment is very much worthwhile and should be attempted.
However, from a scientific perspective, an experiment to test
phenomenon W (wikified peer review) should control for all other
possibly confounding phenomena to make sure that the end result is
an accurate reflection of a proper test of phenomenon W. In this
case, the risks of a new journal with a poorly justified research
focus (as I argued above) is a major confound that blurs the results
of testing for W. In short, I think the best way to test wikified
peer review is to work with an existing journal that has already
established its viability and ability to attract high-quality
submissions.

The Journal of Peer Production has been mentioned as a target
candidate, and their description of their peer review philosophy
indicates that they might be quite open to such an experiment, if
not with all papers, at least with some:
http://peerproduction.net/peer-review/process/. However, it is still
a new journal, and doesn't seem to have yet reached the state of
releasing regular issues, so its newness might yet be a confound for
testing W.

~ Chitu



  Dariusz Jemielniak a écrit :
  

actually, with our community, it is not. What other
  journals die for, we have sort of provided. This is why a Wiki
  journal may have a better chance than others, but only if it is
  prepared with the academic career paths and full proper code of
  conduct nuances considered (double-blind scholarly peer review,
  proper editorial board, PDFs with page numbers, etc.).
  

  
  dj
  

On Sun, Nov 4, 2012 at 7:14 PM, Ed H.
  Chi 
  wrote:
  There has
been a lot of talk about how to start a journal.  The real
issue in starting a journal is not the editorial board, or
the way it
is published, or whether it will gather the citation impact.
 The real
issue is READERSHIP.

If you can get people to read the journal, then it will have
editors
wanting to serve the journal, and it will gather citation
impact.

The reason why WikiSym is changing is for the same reason.
 People are
not going to the conference!  I think the attendance has
 

[Wiki-research-l] Updated Wikipedia literature review working paper

[Chitu Okoli]

  
  
Hi colleagues,

Our research team has posted an updated version of our Wikipedia
literature review at http://ssrn.com/abstract=2021326. The previous
version was quite rough, so we never announced it on this list,
though the Wikimedia Research Newsletter did pick it up. This
version is much better polished and more complete. We would
appreciate your looking at the paper and giving us feedback. Here
are some key features of this updated version of our review:

* It is very long (around 500 references), so we recommend that you
follow the table of contents and only read the sections that cover
topics that actually interest you.
* We include a review of reviews, where we comment on all literature
reviews of Wikipedia that we know of. Please let us know if we
missed any!
* We explain our search and inclusion criteria in detail. We
couldn't include everything, but we would appreciate comments.
* We describe and explain the WikiLit website in detail
(http://wikilit.referata.com).
* We explain with an example how to use the website to support your
own literature reviews in searching for studies on Wikipedia.
* We include an extensive list of all Wikipedia-related researcher
resources that we know of. Please suggest anything that we're
missing!

We are still very actively compiling and cleaning the data on our
WikiLit website (http://wikilit.referata.com). We thank everyone who
has already contributed to the site, and ask others to please take a
look at it:
* In particular, please at least look at your own papers and make
sure that what we have recorded is accurate.
* If we've missed any of your papers, please add them! ("Add
publication" on the left menu.) 
* The answer to the challenge question for anonymous contributors is
"Wikipedia". 
* Please create an account to skip the challenge question and so
that we can acknowledge you in our paper.

Our next major step will be to do various quantitative analyses
based on the website data (e.g. what research methodologies do
studies of Wikipedia reliability normally use; what kind of
Wikipedia data do studies of participation employ; etc.). Please
suggest any such kinds of analyses you might like to see. 

Regards,
  Chitu
  For the WikiLit project team: Arto Lanamäki, Mohamad Mehdi,
  Mostafa Mesgari, Finn Årup Nielsen, Chitu Okoli
  


___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Introduction and a simple question

[Chitu Okoli]

Here's another possibly relevant article: 
http://wikilit.referata.com/wiki/Wikipedia_and_lesser-resourced_languages

~ Chitu


 Message original 
Sujet: Re: [Wiki-research-l] Introduction and a simple question
De : Chitu Okoli 
Pour : wiki-research-l@lists.wikimedia.org
Date : 6 Septembre 2012 09:29:32

Hi Hrafn,

On WikiLit, there is a topic category called "Cultural and linguistic effects on 
participation": 
http://wikilit.referata.com/wiki/Category:Cultural_and_linguistic_effects_on_participation.
 Some of the articles listed there would probably be valuable to you, such as:

* New technologies and terminological pressure in lesser-used languages : the Breton 
Wikipedia, from terminology consumer to potential terminology provider 
<http://wikilit.referata.com/wiki/New_technologies_and_terminological_pressure_in_lesser-used_languages_:_the_Breton_Wikipedia,_from_terminology_consumer_to_potential_terminology_provider>
* Issues of cross-contextual information quality evaluation-the case of Arabic, 
English, and Korean Wikipedias 
<http://wikilit.referata.com/wiki/Issues_of_cross-contextual_information_quality_evaluation-the_case_of_Arabic,_English,_and_Korean_Wikipedias>

~ Chitu


Hrafn H Malmquist a écrit :

Good day everyone

My name is Hrafn Malmquist, I am an Icelandic student of library and
information science at the University of Iceland, writing a master's thesis
on the Icelandic Wikipedia (http://is.wikipedia.org) which I have
personally actively contributed to for about six years
(http://is.wikipedia.org/wiki/Notandi:Jabbi). It has currently 34,478
articles and a very active user base of probably less than 30 users. My
approach is wholistic, recounting the general history of Wikipedia, the
Icelandic Wikipedia, the statistical development and possibly conduct
interviews with contributing users.

Any pointers on interesting research - especially with regard to small
language communities - would be well appriciated.

In searching for sources on the general history of Wikipedia, the best
overview I found is Andrew Lih's The Wikipedia Revolution
(http://en.wikipedia.org/wiki/The_Wikipedia_Revolution). I find it to be
interesting but incomplete and rather sloppy when it comes to citing
sources. He should have finished it off with more care. Does anyone know of
a better alternative?

Best regards, Hrafn

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l




___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] [pre-print] Value production in a collaborative environment

[Chitu Okoli]

Hi Taha,

That's a very nice review you're working on.

For finding potentially missing related papers, browsing the topical directory of WikiLit 
should help you find a lot of papers that you consider "related": 
http://wikilit.referata.com/wiki/Category:Topics.

Unfortunately, one of WikiLit's major weaknesses is that, although it includes 
most peer-reviewed journal articles, it is missing many conference papers 
(though it has almost 100).

~ Chitu



Taha Yasseri a écrit :

Hello Everybody,
Few days ago, we have submitted a manuscript, reviewing some of our recent work 
+ comparisons to others  + some new results.
A pre-print is at:
http://arxiv.org/abs/1208.5130

The aim of the paper is to provide a mini review especially for those ones who 
are not very familiar with the field. However, the paper is clearly biased in 
coverage in favour of our topics of interest and also mentioning only those 
papers that I come across! Since the first characteristic, being limited in 
topical coverage, is fine, the second one, potential missing of related papers 
should be cured as much as possible.

That would be highly appreciated if you could give me feedbacks of any kind, 
especially on the missing literatures.

Cheers,
.Taha Yasseri


___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Introduction and a simple question

[Chitu Okoli]

Hi Hrafn,

On WikiLit, there is a topic category called "Cultural and linguistic effects on 
participation": 
http://wikilit.referata.com/wiki/Category:Cultural_and_linguistic_effects_on_participation.
 Some of the articles listed there would probably be valuable to you, such as:

* New technologies and terminological pressure in lesser-used languages : the Breton 
Wikipedia, from terminology consumer to potential terminology provider 

* Issues of cross-contextual information quality evaluation-the case of Arabic, 
English, and Korean Wikipedias 


~ Chitu


Hrafn H Malmquist a écrit :

Good day everyone

My name is Hrafn Malmquist, I am an Icelandic student of library and
information science at the University of Iceland, writing a master's thesis
on the Icelandic Wikipedia (http://is.wikipedia.org) which I have
personally actively contributed to for about six years
(http://is.wikipedia.org/wiki/Notandi:Jabbi). It has currently 34,478
articles and a very active user base of probably less than 30 users. My
approach is wholistic, recounting the general history of Wikipedia, the
Icelandic Wikipedia, the statistical development and possibly conduct
interviews with contributing users.

Any pointers on interesting research - especially with regard to small
language communities - would be well appriciated.

In searching for sources on the general history of Wikipedia, the best
overview I found is Andrew Lih's The Wikipedia Revolution
(http://en.wikipedia.org/wiki/The_Wikipedia_Revolution). I find it to be
interesting but incomplete and rather sloppy when it comes to citing
sources. He should have finished it off with more care. Does anyone know of
a better alternative?

Best regards, Hrafn

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Social Network Analysis of Wikipedia

[Chitu Okoli]

Searching the WikiLit site for "social network" (in quotes) also brings up many 
relevant studies (though there are a few false matches):
http://wikilit.referata.com/w/index.php?title=Special%3ASearch&search=%22social+network%22&fulltext=Search

~ Chitu


 Message original 
Sujet: Re: [Wiki-research-l] Social Network Analysis of Wikipedia
De : Dario Taraborelli
Pour : Research into Wikimedia content and communities
Date : 5 Septembre 2012 16:09:34

you should also check out:

Laniado, David, Riccardo Tasso, Y. Volkovich, and Andreas Kaltenbrunner. When 
the Wikipedians talk: network and tree structure of Wikipedia discussion pages. 
In Proceedings of the Fifth International AAAI Conference on Weblogs and Social 
Media (ICWSM '11), 177-184, 2011.  
http://www.aaai.org/ocs/index.php/ICWSM/ICWSM11/paper/viewPDFInterstitial/2764/3301

summarized here: 
https://meta.wikimedia.org/wiki/Research:Newsletter/2011-07-25#The_anatomy_of_a_Wikipedia_talk_page

Dario

On Sep 5, 2012, at 1:05 PM, Brian Keegan wrote:


There's a good amount of research

Jullien 2012 has an excellent (although by no means exhaustive) lit review of 
extant Wikipedia research including many network analysis papers:
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2053597

Welser, et al. 2011 use network analysis approaches to identify and 
differentiate users social roles:
http://www.connectedaction.net/wp-content/uploads/2011/04/Welser.Cosley.plus_.Wiki_.Roles_.pdf

Antin, et al. 2012 use some centrality-like metrics to measure the diversity of 
editing behavior:
http://faculty.poly.edu/~onov/Antin_Chehsire_Nov_WPP_CSCW_2012.pdf

Kane 2009 on how network position influences article quality:
http://www.profkane.com/uploads/7/9/1/3/79137/kane_2009_ocisa.pdf

Kane, et al. 2012 on how membership turnover/retention influences article 
quality:
http://www.samransbotham.com/sites/default/files/RansbothamKane_WikiDemotion_2012_MISQ.pdf


Descriptive analysis of Wikipedia's response and networks to the 2011 Tohoku 
earthquake and tsunami:
http://www.brianckeegan.com/papers/WikiSym11.pdf

Developing a statistical model of whether Wikipedia collaborations as a 
bipartite network of editors and authors are more strongly influenced by 
features of editors or features of articles:
http://www.brianckeegan.com/papers/CSCW12.pdf

Developing a unipartite network of Wikipedia collaborations as "document 
passing" network among editors on a single article:
http://www.brianckeegan.com/papers/WikiSym12.pdf


On Wed, Sep 5, 2012 at 7:43 PM, Jeremy Foote  wrote:
I am a brand new Master's student at Purdue. For my Social Network Analysis 
class, I'm thinking about doing a project about whether a Wikipedian's 
centrality in a network can be used as a predictor of future participation. 
I've spent the afternoon looking for relevant literature. I found the very 
interesting

"Validity Issues in the Use of Social Network Analysis with Digital Trace Data" 
by Howison, Wiggins, and Crowston
and
"Network analysis of collaboration structure in Wikipedia" by Brandes et al.

I'm wondering if there are other papers about how to translate Wikipedia into a 
network structure, or even more specifically relating to node-level centrality 
measures and participation measures.

Very many thanks,
Jeremy Foote

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l




--
Brian C. Keegan
Ph.D. Student - Media, Technology, & Society
School of Communication, Northwestern University

Science of Networks in Communities, Laboratory for Collaborative Technology
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] quality control - bitter feelings toward academe??

[Chitu Okoli]

Thanks for the clarification about the Journal of American History; I guess I 
was mistaken.

Responding to your other comment, I am surprised at your frequent jabs at "the bitter feelings 
that are obvious among the Wikipedians here toward academe". This is not at all obvious to me, 
and I doubt it is "obvious" to most people on this list. I've been editing Wikipedia off 
and on since 2003, have been involved in various Wikipedia lists (like this one), and have never 
experienced this perceived anti-academic sentiment (though I do hear people talk about it 
sometimes, like you now).

For sure, I don't go about editing and presenting myself as some sort of 
authority--I assume that any authority I might have should show in my 
contributions, and I let them speak for themselves (not that I am an extensive 
contributor--I am not). I've never faced this kind of problem that you refer to.

On the contrary, I frequently encounter the sentiment of trying to get more academics to 
participate in Wikipedia. Ironically, this very list is one of the strongest expressions of that 
pro-academia sentiment, which makes it especially odd that you somehow feel animosity from 
"the Wikipedians here". Or perhaps I misunderstood your reference to "here".

Chitu Okoli
Associate Professor in Management Information Systems
John Molson School of Business
Concordia University, Montréal

Phone: +1 (514) 993-6648
http://chitu.okoli.org/pro


 Message original 
Sujet: Re: [Wiki-research-l] quality control
De : Richard Jensen 
Pour : Research into Wikimedia content and communities 

Date : 22 Mai 2012 23:21:35

What's relevant to Wikipedia is that Wiki editors are not allowed to do original 
research. We are required to base our articles on published reliable secondary 
sources. In history we do not do very well -- Wikipedia is good at military 
history, mediocre at political history and poor at social & cultural history. 
Despite the bitter feelings that are obvious among the Wikipedians here toward 
academe, that Wikipedia depends upon paid professionals for its material. -- I am 
referring of course not to the thousands of Wiki articles on video game or TV 
characters but to the serious material that bears resemblance to the Encyclopedia 
Britannica. Better yet, compare Wiki with the hundreds of other academic 
encyclopedias that you can find in university libraries. The quality of content of 
those paper history encyclopedias, in my professional judgment, is significantly 
better than Wikipedia.

Richard Jensen


___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Access2research petition - the nature of open access mandates

[Chitu Okoli]

I'm glad this conversation was steered back to the original concern that 
Richard Jensen raised, because I too don't think it has been adequately 
addressed.

That said, it seems that his objection is based on a fundamental 
misunderstanding of the nature of open access mandates. It seems that he is 
reacting as if the mandate is for scholarly journals to provide their articles 
for free (or force them out of business as researchers switch to free access 
journals), but I think this is a gross mischaracterization. I don't claim to be 
know the details of every mandate that has been made, but here's what I 
understand a grant agency open access mandate to usually entail:

*Instead of readers of scholarly articles paying for the privilege to read the 
articles, the cost of dissemination should be shifted to the grant funding 
agency.* Thus, whenever a grant funding agency mandates open access 
publication, it always (to my knowledge) provides funds to cover such 
publication.

In the petition that Dario posted, there is a claim that "the highly successful 
Public Access Policy of the National Institutes of Health proves that this can be done 
without disrupting the research process" [1]. In the referenced policy, NIH 
explicitly affirms that they pay publication costs as part of their mandate policy [2]. 
Thus, I understand the petition to request a mandate that all US-government funded 
research do the same: mandate open access publication plus provide the funds to pay 
journals for such publication.

A common response to such policies is that an increasing number of publishers (including 
our beloved Elsevier [3]) are adopting a "hybrid" open access policy. Very 
simply, this means that they still charge regular journal access fees, but if anyone 
insists on open access, then the publisher is more than happy to oblige to make an 
individual author's article open access as long as the author (usually funded by their 
grant agency) forks over $2,000 to $3,000 to release their article from paywall bondage.

In short, Richard Jensen, this proposed mandate does not attempt to undermine 
the funding structure of scholarly journals. The only significant change it 
would push on journals is to adopt a hybrid open access policy, in which they 
would ask authors to show them the money if their grant funder requires open 
access publication. While the high cost of open access publication might be 
debatable, I see no financial threat to scholarly journals, as long as they are 
willing to make basic changes in their funding structure to keep up with the 
times.


On a related note, referring to the related thread "real scholarship is expensive", I 
have to question the description of the costs involved in producing /The Journal of American 
History/. On looking it up, I find that it is the flagship journal of The Organization of American 
Historians [4]. According to my general observation, a journal like that, in addition to its 
function as a leading scholarly publication, also serves as a cash cow for funding a non-profit 
scholarly society. I am not questioning a scholarly society's need for funding, but I question that 
the journal alone needs such a large full-time paid staff, beyond the volunteer "staff" 
of reviewers and editors that is typical of scholarly journals. I might be mistaken, but the staff 
described sounds to me like the (necessary) staff of a scholarly society office, not that of a 
standalone scholarly journal.

Regards,

Chitu Okoli
Associate Professor in Management Information Systems
John Molson School of Business
Concordia University, Montréal
http://chitu.okoli.org/pro


[1] http://access2research.org/Petition
[2] http://publicaccess.nih.gov/FAQ.htm#810
[3] http://www.elsevier.com/wps/find/intro.cws_home/open_access
[4] http://www.oah.org/publications/



 Message original 
Sujet: Re: [Wiki-research-l] Access2research petition
De : David Golumbia 
Pour : Research into Wikimedia content and communities 

Date : 22 Mai 2012 20:29:26

i'm sorry but this is a *complete* red herring with regard to the discussion 
Richard has raised.

i know of *no* for-profit publishing in humanities journals, and a very few and 
marginal ones (SAGE, John Benjamins) in social sciences. that goes for books, 
too, which I am half-expecting to come under attack here next.

what we are talking about here is /non-profit publishing/. that is what I and presumably 
Richard see as under attack on this list, for reasons that are both clear and very 
disturbing to me. not only not making "billions": making/no profit at all/. 
JSTOR, previously attacked here, is a complete non-profit, and nobody has yet cogently 
argued that JSTOR wasted the funds it was paid to archive over 100 years of academic 
journals. I do not know why it is somehow morally wrong for them to have been paid a 
reasonable, non-profit figure to do go

Re: [Wiki-research-l] WikiPapers has now over 1, 000 publications - why WikiLit?

[Chitu Okoli]

Hi Piotr,

http://WikiLit.referata.com is the project website for a comprehensive 
literature review of scholarly research on Wikipedia. Unlike permanent 
long-term sites like AcaWiki (which compiles summaries of scholarly studies of 
all subjects) and WikiPapers (which compiles bibliographic and other details of 
scholarly research on wikis), our site is a temporary project site for a 
literature review of a very narrow subset of studies: those that focus on 
Wikipedia. Our goal is to host our extracted literature review data there while 
we are in the final stages of the review, complete and clean it up, and then 
when completed, eventually export it all to long-term sites like AcaWiki and 
WikiPapers.

We have no intention whatsoever to provide an alternative to these sites--we 
truly appreciate the amazing amount of work that has gone into them and that is 
required to keep them going, and we have no intention to do that kind of work 
ourselves :-). On the contrary, once our data collection is completed and the 
data is cleaned, we hope to support these sites by feeding high-quality data to 
them. (Admitted, though, there might be license incompatibilities with AcaWiki, 
since our data is licensed CC-BY-SA whereas theirs is CC-BY.) At that point, 
our site will probably be closed for edits, and we will only point people to 
those other sites if they want to add anything new, and we also intend to 
provide exports of the dataset in various formats, for which we've already 
received quite a few requests.

Our project team is constantly working heavily on the site--we've made over 
4,000 internal edits involving adding and cleaning up data since we first 
launched the site on March 1, not counting edits by external 
contributors--which we greatly appreciate, since people who edit their own 
articles are probably the most accurate! You see, this is actually an internal 
project site that we're exposing externally to 1) get feedback and help; and 2) 
share data in beta stage of development. We hope to complete the project in the 
next couple months.

I hope that clears up the difference. We certainly hope that the existence of 
our site is not distracting or discouraging anyone from contributing to 
WikiPapers, which we wholeheartedly support!

Regards,
Chitu


Luiz Augusto a écrit :

Good question.

A more detailed explanation would be welcome, but it seems that WikiLit is 
focused on research done exclusively at Wikipedias, and WikiPapers is focused 
on all wikis researchs.

Maybe a fusion on project goals and data already available in one single big project make 
it more interesting to contribute or even to get moved to Wikimedia Foundation servers 
(as part of the revamp on Proposals for new projects being drafted on the "Sisters 
projects committes" too on draft stage 
http://meta.wikimedia.org/wiki/Sister_Projects_Committee ) ...

On Tue, May 1, 2012 at 12:18 PM, Piotr Konieczny mailto:p...@pitt.edu>> wrote:

Can somebody tell me why we have both the 
http://wikipapers.referata.com/wiki/Main_Page and 
http://wikilit.referata.com/wiki/Main_Page ?

--
Piotr Konieczny


___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Fwd: Re: Experimental study of informal rewards in peer production

[Chitu Okoli]

 Message original 
Sujet:  Re: [Wiki-research-l] Experimental study of informal rewards in peer 
production
Date :  Thu, 26 Apr 2012 15:50:44 -0400
De :Michael Restivo 
Pour :  Chitu Okoli , Research into Wikimedia content and 
communities 



Hi Chitu,

Yes, your conjecture is spot-on. Here is a more detailed response that I sent 
to Joseph. I tried sending this to the wiki-research-l but the email keeps 
bouncing back to me. If you're interested and willing to share it with the 
list, that would be acceptable to me.

   We thought about this question quite extensively and there are a few reasons 
why we sampled the top 1% (which we didn't get around to discussing in this 
brief paper). First, because of the high degree of contribution inequality in 
Wikipedia's editing community, we were primarily interested in how status 
rewards affect the all-important core of highly-active editors. There is also a 
lot of turn-over in the long tail of the distribution, and even among the most 
active editors, there is considerable heterogeneity. Focusing on the most 
active users ensured us sufficient statistical power. (Post-hoc power analysis 
suggests that our sample size would need to be several thousand users in the 
80-90th percentiles, and several hundred in the 90-99th percentiles, to discern 
an effect of the same strength.) Also, we considered the question of construct 
validity:  which users are deserving (so to speak) of receiving an editing 
award or social recognition of their work?

   You are right that it should be fairly easy to extend this analysis beyond 
just the top 1%, but just how wide a net to cast remains a question. The issue 
of power calculation and sample size becomes increasingly difficult to manage 
for lower deciles because of the power-law distribution. And I don't think it 
would be very meaningful to assess the effect of barnstars on the bottom half 
of the distribution, for example, for the substantive reasons I mentioned 
above. Still, I'd be curious to hear what you think, and whether there might be 
some variations on this experiment that could overcome these limitations.

   In terms of data dredging, that is always a concern and I completely 
understand where you are coming from. In fact, as both and author and consumer 
of scientific knowledge, I'm rarely ever completely satisfied. For example, a 
related concern that I have is the filing cabinet effect - when research 
produces null (or opposite) results and hence the authors decide to not attempt 
to have it published.


   In this case, I actually started this project with the hunch that barnstars would lead 
to a slight decline in editing behavior; my rationale was that rewards would act as 
social markers that editors' past work was sufficient to earn social recognition and 
hence receiving such a reward would signal that the editor had "done enough" 
for the time being. In addition to there being substantial support for this idea in the 
economics literature, this intuition stemmed from hearing about an (unpublished) 
observational study of barnstars by Gueorgi Kossinets (formerly at Cornell, now at 
Google) that suggested editors receive barnstars at the peak of their editing activity. 
Of course, we chose an experimental design precisely to help us to tease out the causal 
direction as well as what effect barnstars have for recipients relative to their 
unrewarded counterparts. We felt like no matter what we found - either a positive, 
negative, or even no effect - it would have been interesting
   enough to publish, so hopefully that alleviates some of your concerns.

   Please let me know if you have any other questions, and I'd love to hear 
your thoughts about potential follow-ups to this research.



Regards,
Michael



On Thu, Apr 26, 2012 at 3:30 PM, Chitu Okoli mailto:chitu.ok...@concordia.ca>> wrote:

   One obvious issue is that it would be unethical to award barnstars to 
contributors who did not deserve them. However, the 1% most productive 
contributors, by definition, deserved the barnstars that the experimenter 
awarded them. Awarding barnstars to undeserving contributors for experimental 
purposes probably would not have flown so easily by the ethical review board. 
As the article notes:

   --
   This study's research protocol was approved by the Committees on Research 
Involving Human Subjects (IRB) at the State University of New York at Stony 
Brook (CORIHS #2011-1394). Because the experiment presented only minimal risks 
to subjects, the IRB committee determined that obtaining prior informed consent 
from participants was not required.
   --

   This is my conjecture; I'd like to hear the author's comments.

   ~ Chitu

    Message original 
   Sujet: [Wiki-research-l] Experimental study of informal rewards in peer 
production
   De : Joseph Reagle mailto:joseph.2...@rea

Re: [Wiki-research-l] Experimental study of informal rewards in peer production

[Chitu Okoli]

One obvious issue is that it would be unethical to award barnstars to 
contributors who did not deserve them. However, the 1% most productive 
contributors, by definition, deserved the barnstars that the experimenter 
awarded them. Awarding barnstars to undeserving contributors for experimental 
purposes probably would not have flown so easily by the ethical review board. 
As the article notes:

--
This study's research protocol was approved by the Committees on Research 
Involving Human Subjects (IRB) at the State University of New York at Stony 
Brook (CORIHS #2011-1394). Because the experiment presented only minimal risks 
to subjects, the IRB committee determined that obtaining prior informed consent 
from participants was not required.
--

This is my conjecture; I'd like to hear the author's comments.

~ Chitu

 Message original 
Sujet: [Wiki-research-l] Experimental study of informal rewards in peer 
production
De : Joseph Reagle 
Pour : michael.rest...@stonybrook.edu
Copie à : Research into Wikimedia content and communities 

Date : 26 Avril 2012 11:42:01


In this 
[study](http://www.plosone.org/article/info:doi%2F10.1371%2Fjournal.pone.0034358):

 > Abstract: We test the effects of informal rewards in online peer production. 
Using a randomized, experimental design, we assigned editing awards or “barnstars” 
to a subset of the 1% most productive Wikipedia contributors. Comparison with the 
control group shows that receiving a barnstar increases productivity by 60% and 
makes contributors six times more likely to receive additional barnstars from 
other community members, revealing that informal rewards significantly impact 
individual effort.

I wonder why it is limited to the top 1%? I'd love to see the analysis repeated 
(should be trivial) on each decile. Besides satisfying my curiosity, some 
rationale and/or discussion of other deciles would also address any 
methodological concern about data dredging.

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] # of citations on Wikipedia?

[Chitu Okoli]

I did a search on "citation*" on our Wikipedia lit review site. I'm not sure any of the articles 
we've identified so far answers your question, but the search returns a list that could contain some 
interesting articles: 
http://wikilit.referata.com/w/index.php?title=Special:Search&redirs=1&ns0=1&search=citation*&limit=50&offset=0.

~ Chitu


Heather Ford a écrit :

I've been collaborating with the group lens folks on citations. They've done 
basic statistics of sources of cites etc across .en. Will ask them about 
sending.

Sent from my iPhone

On Apr 20, 2012, at 11:16 AM, Jessie Wild mailto:jw...@wikimedia.org>> wrote:


I talked about something like this with Liam that GLAM was trying to develop 
and utilize to inform it's work.

Liam - sorry to call you out like this, but any thoughts on Phoebe's question?


On Fri, Apr 20, 2012 at 5:31 PM, phoebe ayers mailto:phoebe.w...@gmail.com>> wrote:

Hi all,

Has there been any research done into: the number of citations (e.g.
to books, journal articles, online sources, everything together) on
Wikipedia (any language, or all)? The distribution of citations over
different kinds or qualities of articles? # of uses of citation
templates? Anything like this?

I realize this is hard to count, averages are meaningless in this
context, and any number will no doubt be imprecise! But anything would
be helpful. I have vague memories of seeing some citation studies like
this but don't remember the details.

Thanks,
-- phoebe

--
* I use this address for lists; send personal messages to phoebe.ayers
 gmail.com  *

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org 

https://lists.wikimedia.org/mailman/listinfo/wiki-research-l




--
/Jessie Wild
Global Development, Manager
Wikimedia Foundation
/

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org 
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] [Wikimedia-l] User retention statistics?

[Chitu Okoli]

Hi Yaroslav,

I don't have any specific guidance for your questions, but in a literature 
review I'm currently conducting with some colleagues, we have so far identified 
eleven articles concerning participation trends, which might be helpful: 
http://wikilit.referata.com/wiki/Category:Participation_trends.

Regards,

Chitu Okoli
Associate Professor in Management Information Systems
John Molson School of Business
Concordia University, Montréal

Phone: +1 (514) 993-6648
http://chitu.okoli.org/pro



 Message original 
Sujet: Re: [Wiki-research-l] [Wikimedia-l] User retention statistics?
De : WereSpielChequers 
Pour : Research into Wikimedia content and communities 

Date : 19 Avril 2012 10:20:44

It is an interesting but complex topic.

Complications include:

 1. People shift project - some of the early EN wikipedians may now be mostly 
active in their own language versions of Wikipedia. The recent high profile 
retirement at EN wikipedia is an editor still active on Commons
 2. People shift Account. Some of our "former" accounts are known to belong to 
people who now edit under other names. Some others may be doing so without disclosing 
their former account.
 3. We have had some deaths

That said there is definitely a pattern of the most active Wikipedians sticking around, and 
I'm pretty sure that whether or not someone is an admin makes quite a difference there. We 
have circa 750 active admins - 732 today 
http://en.wikipedia.org/w/index.php?title=Wikipedia:List_of_administrators&action=history 
<http://en.wikipedia.org/w/index.php?title=Wikipedia:List_of_administrators&action=history>
 and we've appointed less than 250 in the last three years. so the vast majority of our 
admins have been admins for much longer and editors for even longer than that. I suspect that 
our failure to appoint more admins is one of the things that is holding the community back 
and slowing recruitment into the core of editors who stick around and are our most active.

Of the nearly 1500 Wikipedian admins (active and inactive) only 150 first edited in the last five years, and that includes 8 
bots. 
http://en.wikipedia.org/w/index.php?title=Special:ListUsers&dir=prev&offset=3020778&username=&group=sysop&creationSort=1
 
<http://en.wikipedia.org/w/index.php?title=Special:ListUsers&dir=prev&offset=3020778&username=&group=sysop&creationSort=1>
 So at least 90% of the human admins first edited over five years ago, and the other 10% are heavily skewed to the 48-60 
month group.

I appreciate that we can't expect someone to pass RFA within a year of their 
first edit. But that still leaves a huge gap - a generation of editors who 
started in 2008-2010 and aren't admins or even necessarily still editors.

WereSpielChequers


On 19 April 2012 13:19, emijrp mailto:emi...@gmail.com>> 
wrote:

This thread is a good candidate for wiki-research-l. Forwarding...

2012/4/18 Yaroslav M. Blanter mailto:pute...@mccme.ru>>

My message is inspired by discussion in this thread 
(http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard#Loss_of_more_and_more_and_more_established_editors_and_administrators)
 on Englush Wikipedia. Whereas the thread itself is not relevant to this list, 
and the points get re-iterated on a regular basis, there were statements made 
there which contain quantitative estimates (for instance that 90% established 
users who leave do it because they get a new job or have their external life 
changed in some other way, and not because of harassment etc). Most probably 
these numbers are not really justified, but then I wanted to know what real 
numbers are. I am an Rcom member, but I can not recollect such research being 
accomplished (I might be wrong of course). I could not find data easily either 
(I spent half an hour because I remembered we had a Community Health initiative 
group which somehow evolved into the Movement Roles, but the Movement Roles 
pages on
Meta do not talk about community health at all, and I could not even 
find an appropriate page to ask the question).

After this long introduction, does somebody know / can point out the 
answers to the questions:

1. What is the average lifetime of a Wikipedia editor (for instance the 
one with at leat 1000 contributions)? I recollect smth about two years, but I 
am pretty sure I have never seen any research on this. How does it depend on 
the number of contributions?

2. What are the main reasons why these editors stop editing? Is this 
correct, for instance, that external reasons are much more important than 
internal (on-wiki troubles and wiki-related harassment) reasons? The same for 
say those above 1 edits?

Thanks in advance
Cheers
Yaroslav

___
Wikimedia-l mailing 

[Wiki-research-l] FW: Acknowledgement Search Engine - AckSeer

[Chitu Okoli]

Hi everyone,

Although the item below that I'm forwarding is not directly related to wiki 
research, it is of general interest to researchers in a wide variety of fields, 
and particularly to bibliometricians, of which I know there are a lot on this 
list.

Regards,
Chitu

-

Date: Mon, 26 Mar 2012 12:56:43 -0400
From: Lee Giles
To: AISWORLD Information Systems World Network
Subject: [AISWorld] Acknowledgement Search Engine - AckSeer


AckSeer is a beta automatic acknowledgment indexing search engine that
explores automatic identification, entity extraction and indexing of
acknowledgements from papers. Also, acknowledged entities are
extracted within the acknowledgment passages.

Currently, AckSeer indexes acknowledgments from more than 500,000 papers
in CiteSeerX. These acknowledgements contain more than 4 million
acknowledged entities with approximately 2 million of them unique.
Entity extraction is based on AlchemyAPI and OpenCalais. Acknowledged
entities are ranked by citation. Feedback is most welcomed.

http://ackseer.ist.psu.edu

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Announcing WikiLit beta dataset

[Chitu Okoli]

Thanks, Jack.

~ Chitu

 Message original 
Sujet: Re: [Wiki-research-l] Announcing WikiLit beta dataset
De : Jack Park 
Pour : Research into Wikimedia content and communities 

Date : 16 Mars 2012 16:12:00

The top link is a typo, though it's corrected later:
http://wikilit.referata.com/wiki/Main_Page

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Announcing WikiLit beta dataset

[Chitu Okoli]

Hi everyone,

As I mentioned in an e-mail last week, we are on the final stages of a large 
literature review on scholarly research on Wikipedia. We have extracted and 
organized most of the data and have published it to a Semantic MediaWiki wiki 
at http://wikilit.refarata.com. [Thanks to emijpr for inspiring the structure 
through WikiPapers.] As I indicated, this is sort of an incubator site to 
finish up our data to prepare for publication, after which we intend to export 
the data permanently to other sites like AcaWiki and WikiPapers.

Thanks for your responses to my inquiries; we have included abstracts, and the 
data is dual-licensed as CC-BY-SA and ODC-ODbl 
(http://opendatacommons.org/licenses/odbl/summary/) [thanks, Dario, for the 
links!], except for copyrighted abstracts. We have submitted a related 
presentation proposal for Wikimania 2012 at 
http://wikimania2012.wikimedia.org/wiki/Submissions/Wikilit:_Ten_years_of_Wikipedia_research.

We are asking the Wikipedia research community to please help us verify the 
accuracy of our data extraction so far. Practically, if you could at least take 
a look at your own publications and the publications you know well, that would 
be great. It's an open wiki, so please make any corrections directly, even 
anonymously. (However, if you want us to acknowledge your contributions, please 
create a user account and identify yourself on your user page.) In particular, 
please help us with the following:
* Please correct any inaccuracies you see, or e-mail us at wiki...@okoli.org to 
notify us of them.
* Please point out any peer-reviewed journal articles or PhD dissertations we 
have missed that were published before July 2011; we will certainly add these. 
(After that, the Wikimedia Research Newsletter began.)
* Please point out any other scholarly studies (especially conference articles 
and significant non-peer-reviewed work) that you feel should definitely be 
analyzed in detail. Although we have listed 1,500 conference papers 
(http://wikilit.referata.com/wiki/List_of_conference_papers), our limited time 
and resources only permits us to analyze a fraction of them in detail. So, 
please help us highlight the most important ones that we have not analyzed in 
detail, with a brief explanation of why they are particularly important.
* Please add any published scholarly studies about Wikipedia that we have left 
out, regardless of peer review or publication type! Please add your own work! 
Our restrictions in what we include are purely pragmatic due to time and 
resource limitations. However, if you add a new article, please be sure to 
*complete as many input fields as possible*, since we will generally exclude 
any article with incomplete data in our final analysis.
* Please suggest any data analysis or visualizations you would like to see as 
we synthesize the data.
* Please give any other feedback or suggestion that can help us make this 
dataset more useful to researchers! Send comments to wiki...@okoli.org.

The data is publicly available, but this is a beta release and there are probably a lot 
of errors. We hope to have a stable and very clean dataset within a couple months, both 
from community help and from our own internal quality control processes; we'll make 
another announcement when we feel the dataset has reached "featured" quality. 
In particular, please wait a bit before exporting the data to other research collection 
websites and wikis until it is in a cleaner state; by then, we'll help make it available 
in as many export formats as practical.

Regards,
Chitu
For the WikiLit project team: Arto Lanamäki, Mohamad Mehdi, Mostafa Mesgari, 
Finn Årup Nielsen, Chitu Okoli

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Wikilit question about site license

[Chitu Okoli]

Hi everyone,

As I mentioned in a separate thread, in the week or so we expect to release the 
main dataset of our extraction to a SMW wiki for comments and feedback before 
we proceed with further analysis. We're not planning to set up a permanent, 
dynamic website; we just want to post our data, get communal feedback to clean 
it up, and then eventually export the data to relevant long-term sites like 
AcaWiki, Brede Wiki and WikiPapers.

We are wondering how to license our site. We are naturally inclined to 
CC-BY-SA, like Wikipedia (and also Brede Wiki and WikiPapers), but we noticed 
that AcaWiki, one of our main export targets, is licensed CC-BY, which cannot 
use CC-BY-SA material. We like CC-BY-SA because it keeps derivatives free for 
communal reuse; however, we recognize that CC-BY is more flexible, since it 
permits publishing derivatives in journals that require full copyright 
transfer. (Of course, this is talking about copyright and copyright transfer; 
the CC-BY and academic standards against plagiarism would require attribution 
of the source, which is not the issue here.)

Perhaps we should be asking this question on an AcaWiki forum, but we'd like to 
hear thoughts on licensing considerations for sites that publish academic 
summaries and specific recommendations for our upcoming data release.

Regards,
Chitu
For the Wikilit project team: Arto Lanamäki, Mohamad Mehdi, Mostafa Mesgari, 
Finn Årup Nielsen, Chitu Okoli
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Wikilit question about posting abstracts

[Chitu Okoli]

Hi everyone,

Last year we asked for feedback about a large literature review we are 
currently conducting on peer-reviewed studies on Wikipedia. We thank everyone 
who gave us comments and pointers, and we have carefully listened to 
everything. We're glad to mention that we've joined forces with Finn Årup 
Nielsen, and so we'll be fully merging our review into his (his latest working 
paper is available at
http://www2.imm.dtu.dk/pubdb/views/edoc_download.php/6012/pdf/imm6012.pdf).

In the next week or so we expect to release the main dataset of our extraction 
to a public SMW wiki for comments and feedback before we proceed with further 
analysis. However, we have two questions to clarify about how to release our 
data. I'll be posting them in two distinct threads for more organized 
responses. Here I'll ask a question about posting published abstracts, and in a 
separate posting I'll ask about license.

We would really love to put up all the abstracts of the articles that we post 
(and indeed have already done so on http://en.wikipedia.org/wiki/WP:ACST), but 
I recently encountered the following article which basically argues that mass 
posting of abstracts is not justifiable as fair use, at least in the context of 
Wikipedia:
http://journalology.blogspot.com/2010/05/why-you-cant-copy-abstracts-into.html

We don't think this argument necessarily applies to websites that list 
abstracts (for example, PubMed and Medeley published thousands of abstracts), 
but we would really liketo hear the community's comments on whether posting 
copyrighted abstracts (in our case, over 400) generally qualifies as fair use.

Regards,
Chitu
For the Wikilit project team: Arto Lanamäki, Mohamad Mehdi, Mostafa Mesgari, 
Finn Årup Nielsen, Chitu Okoli
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Research into women's involvement on Wikimedia Foundation projects

[Chitu Okoli]

On Thu, Dec 29, 2011 at 5:06 AM, Chitu Okoli mailto:chitu.ok...@concordia.ca>> wrote:


a recent First Monday article classifies Wikipedia research by country of 
provenance, among other criteria:

http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/3492/3031



Any analysis of the gender of people doing that research by country? 
http://en.wikipedia.org/wiki/Academic_studies_about_Wikipedia#Demographics 
shows only one analysis that mentions gender but not country.  Are demographic 
studies and content studies by nationality and gender not being done?  Any one 
done an article that has large scale uses of data like 
http://meta.wikimedia.org/wiki/File:Polish_Gender_Wikiproject.png or something 
similar?



The study used bibliometric data (that is, data that is recorded in the journal 
or conference publication in structured format). That does not include gender 
information, so the study has no such information.

~ Chitu
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Research into women's involvement on Wikimedia Foundation projects

[Chitu Okoli]

Hi Laura,

I'm not sure if this is what you're looking for, but a recent First Monday 
article classifies Wikipedia research by country of provenance, among other 
criteria:
http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/3492/3031

Perhaps that might give you some leads.

Regards,
Chitu


Laura Hale a écrit :

Has anyone done any research on a by country basis?  I'm trying to fill out 
http://meta.wikimedia.org/wiki/WikiWomenCamp/FAQ/Perspectives a bit and I'm 
looking for assistance.  I know there has been some research but not sure what 
and where, and what the breakdown was when analysis has been done.  Has anyone 
done comprehensive research on this subject in terms of participation?

Thanks,
Laura Hale

--
twitter: purplepopple
blog: ozziesport.com 


___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Closed-sourced papers on open source communities

[Chitu Okoli]

Great idea, Stuart. This is definitely a great, albeit partial, solution!

I'll take it a couple steps further: First, in information systems 
publications--I don't know about other fields--it is quite common to borrow and 
adapt tables, not just figures, from other sources. As far as I understand 
(someone should please correct me if I'm mistaken), this is done under fair 
use/fair dealing laws. As long as the figures or tables copied or modified are 
just one small part of the article from which they are taken, copyright 
infringement has never been an issue. I've certainly copied third-party figures 
in this way (with appropriate citation, of course), and the journal's editorial 
office never mentioned a word about it.

Your solution makes this case even stronger: if the journals have no issue with 
borrowing third-parties' articles or tables under fair use/fair dealing laws, I 
couldn't imagine they would have an issue with using our own relicensed work.

My second extension: Extending this principle to tables should fill in the gap 
for qualitative research, which you mentioned in your blog as a possible 
limitation. You are right that most qualitative research might not be so 
meaningfully captured in graphs (though there are numerous creative 
exceptions); however, qualitative articles can effectually summarize the entire 
substance of their arguments and findings in well-written textual tables. I'm 
hoping that the rejuvenated AcaWiki (according to the thread from a couple 
months past) could provide a CC publication outlet for such tables, in the same 
spirit as Wikimedia Commons serving for graphs.

~ Chitu
Regards,

---
Chitu Okoli
Associate Professor in Management Information Systems
John Molson School of Business
Concordia University, Montréal

Phone: +1 (514) 848-2424 x2985
http://chitu.okoli.org/pro


From: R.Stuart Geiger
Date: 2011/6/13
Subject: [Wiki-research-l] Closed-sourced papers on open source
communities
To: Research into Wikimedia content and communities<
wiki-research-l@lists.wikimedia.org>


Greetings wikiresearchers,

As many of you know (and as we've discussed on this list before),
the
copyright licensing of academic papers about communities like
Wikipedia is a huge issue.  I've just written up a blog post
about
this, but the tl;dr is that I have a bit of a solution, be it a
partial one.  The gist is basically that asking academics to
release
*papers* under a free license is the wrong strategy.  Instead, we
should encourage academics to release *research* under a free
license,
and that this can be done in such a way that still makes it
complies
with most of the contradictory obligations we have found
ourselves in.

It is quite possible to document a research project, its
motivations,
its methods, its background, its findings, and even all those
charts
and graphs on Meta, using the new Research: namespace and
corresponding templates that were *just* launched -- which
everyone
should check out anyway.  And while I'd love some legal
non-advice on
this, I think we can do this in such a way that whenever it comes
time
to assign copyright to the ACM, all of the CC-BY/CC-BY-SA
licensed
graphs can be "used with permission" in a published research
paper.
Anyways, the link is below, and I'd love to get some feedback on
it:
http://www.stuartgeiger.com/wordpress/random-thoughts/2011/06/12/closed-source-papers-on-open-source-communities-a-problem-and-a-partial-solution/

Thanks!
Stuart Geiger

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Article on teaching with wikipedia - where to publish?

[Chitu Okoli]

Hi Piotr,

Your dilemma is one I sympathize with and struggle with myself. The route I am thinking 
of following myself is the "hybrid journal" format: a traditional paid journal 
that will enable certain articles open access for a fee.

I looked up Teaching Sociology, and found that they are in the Sage journals family. Sage 
recently launched a hybrid policy, called "Sage Choice": 
http://www.sagepub.com/sagechoice.sp. This describes what I'm talking about: if the 
author pays the bounty to release an article from journal jail, the publisher will gladly 
go open access--for that article only. Sage's rate is $3,000. Other journal prices I've 
seen are typically in the $2,000 to $3,000 range per article. This is the fair market 
price of publishing in a high-quality open access journal (e.g. 
http://www.plos.org/journals/pubfees.php).

Teaching Sociology is not in the Sage Choice program, but I dare say that if 
you could raise $3,000 in publishing fees, you could negotiate a special deal 
with Sage to release an article open access. Of course, I'm only using Teaching 
Sociology and Sage as examples, since that is the journal you mentioned; I 
would think that many other publishers would be flexible to negotiate a special 
arrangement, as long as you pay the bounty. Money talks, after all.

~ Chitu

Piotr Konieczny a écrit :

Dear all,

I have just finished my second "teaching with Wikipedia" article. I'd
like to publish it in an established academic journal that, if possible,
supports open content. Unfortunately, I do not have much experience with
this sector of the journals (teaching/education/pedagogy journals), nor
with journal impact magic, and thus I'd very much appreciate your
suggestions where to publish. I have, of course, quickly Google'd few
open content teaching journals, but I admit, selfishly, that entering
the job market, I'd prefer my CV to include, if possible, higher-end
journals...

(In my sociology field, the most respected educational journal,
"Teaching Sociology", is, sadly, not open content...).

If anybody is interested in reading and commenting on my article in
question (tentatively titled "Wikis and Wikipedia as a Teaching Tool:
Five Years Later"), I have made it available on Google Docs (just let me
know and I'll send you a link, and enable commenting for your account).

PS. My old 2007 article (titled, unsurprisingly, "Wikis and Wikipedia as
a Teaching Tool") was published here:
http://www.itdl.org/Journal/Jan_07/article02.htm
I am still content with it for what it was in 2007, but by 2011, it is,
I'll be the first to admit it, rather obsolete.

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] A new home for Wikimedia research? Come share your thoughts

[Chitu Okoli]

Hi Dario,

I for one strongly support all the goals here. The lack of a unified researcher 
portal on Meta:
* Makes it hard to find related research projects; and
* Discourages me from posting information about what I'm currently doing, since 
it seems like no one will really look at it.
I would definitely try to keep my current projects up-to-date if I knew there 
was one central portal that all other research pages on Meta or Wikipedia 
project pages pointed to, with minimal redundancies.

Thanks for leading this project.

~ Chitu


 Message original 
Sujet: [Wiki-research-l] A new home for Wikimedia research? Come shareyour 
thoughts
De : Dario Taraborelli 
Pour : Research into Wikimedia content and communities 

Copie à : The Wikimedia Foundation Research Committee mailing list 

Date : 13 Mai 2011 19:19:40

The Wikimedia Research Committee [1] is currently considering a major overhaul 
of the research section on Meta-Wiki:

http://meta.wikimedia.org/wiki/Research/2011_overhaul

The practical reason to start this process is to clean up and streamline pages 
used by the Wikimedia community and by the Foundation to document internal 
research projects and policies. The ambitious goal it to make Meta:Research the 
main hub where all research on Wikimedia projects (be it internal or external) 
is discussed, reviewed and tracked. The objectives we are hoping to achieve in 
the short term with this project are the following:

* make it easy for researchers to find the resources and WMF support they 
are looking for
* bring as much transparency as possible to research involving the 
Wikimedia community, by reducing attrition between the community and 
researchers and making sure research is not disruptive of editor activity
* design a scheme of incentives to increase researcher participation and to 
increase the number of projects included in the Wikimedia research directory
* design a series of incentives to nudge researchers towards releasing 
their datasets under an open license and publishing/self-archiving their 
research results via open access outlets/repositories.


Our long-term vision aims to:

* provide support to the publication of research data on Wikimedia projects 
via a unified open data infrastructure [2]
* integrate structured bibliographic data into Meta:Research via whatever 
solution the community decides to adopt [3]


Many on this list are already actively involved in editing and maintaining Meta 
research pages. Your feedback and suggestions on this project would be very 
valuable.

Dario

[1] http://meta.wikimedia.org/wiki/Research_Committee
[2] We are currently reviewing a number of solutions to set up a central 
repository of open research data: http://bit.ly/OpenDataPlatforms
[3] See the long discussion started on this list with this thread: 
http://lists.wikimedia.org/pipermail/wiki-research-l/2011-March/001361.html


--
Dario Taraborelli, PhD
Senior Research Analyst
Wikimedia Foundation

http://wikimediafoundation.org
http://nitens.org/taraborelli


___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Wikipedia & Emotions // Wikipedia & old people

[Chitu Okoli]

Hi Mayo,

My research group is currently working on a comprehensive literature review of 
Wikipedia research. For now, almost all the studies we've found are posted at 
these locations:

* Peer-reviewed journal articles: 
http://en.wikipedia.org/wiki/Wikipedia:Academic_studies_of_Wikipedia#Journal_articles
* Doctoral and some other theses: 
http://en.wikipedia.org/wiki/Wikipedia:Academic_studies_of_Wikipedia#Theses
* Peer-reviewed conference papers (temporary location): 
http://en.wikipedia.org/wiki/User:Moudy83/conference_papers

Most of the abstracts are included, though hidden. To access all the abstracts 
for searching, you might want to either display the wiki source code (by 
clicking "Edit") or view the source of the HTML page. Then you could do a 
simple keyword search on "emotion" or "elderly" or other keywords.

Of course, this is just a rough search suggestion, but hopefully you should 
locate a few studies that way.

~ Chitu

 Message original 
Sujet: [Wiki-research-l] Wikipedia & Emotions // Wikipedia & old people
De : Fuster, Mayo 
Pour : wiki-research-l@lists.wikimedia.org 
Date : Lun. 25 Avril 2011 10:10:13 EST

Dear everybody!

I hope that you are happy and fine!

I am looking for previous studies, data, writing, suggestions on two issues I 
am working at the moment:

* Wikipedia and emotions, from a plurality of perspectives (i.e. data which 
emerged on studies in terms of emotions that wikipedians express, analysis from 
the perspective of sociology of emotions on wikipedia case, etc.)

* Old people (people of more than 55 years old) participaiting in Wikimedia 
projects (I.e. stadistics of participation, attends to reach old people, etc.).

Thank you in advance. Have a nice day! Mayo

«·´`·.(*·.¸(`·.¸ ¸.·´)¸.·*).·´`·»
«·´¨*·¸¸« Mayo Fuster Morell ».¸.·*¨`·»
«·´`·.(¸.·´(¸.·* *·.¸)`·.¸).·´`·»

Research Digital Commons Governance: http://www.onlinecreation.info

Ph.D European University Institute
Postdoctoral Researcher. Institute of Govern and Public Policies. Autonomous 
University of Barcelona.
Visiting scholar. Internet Interdisciplinary Institute. Open University of 
Catalonia (UOC).
Visiting researcher (2008). School of information. University of California, 
Berkeley.
Member Research Committee. Wikimedia Foundation

http://www.onlinecreation.info
E-mail: mayo.fus...@eui.eu
Skype: mayoneti
Phone Spanish State: 0034-648877748

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Proposal: build a wiki literature review wiki-style

[Chitu Okoli]

 Message original 
Sujet: Re: [Wiki-research-l] Proposal: buildawikiliterature
reviewwiki-style
De : Reid Priedhorsky 
Pour : wiki-research-l@lists.wikimedia.org
Date : March-24-11 11:53:05 AM

On 3/23/11 2:56 PM, Chitu Okoli wrote:
Oh, I definitely agree that grad student contributions are tremendously
valuable! (especially having been on until very recently)

My point was this: that writing for a lay audience and writing for
fellow researchers (grad students included) are different tasks, and
mixing them leads to reduced value for each audience.

I am fine with each paper having a "for laypeople" and "for researchers"
section to the summary.

Sorry, Reid; I misunderstood you. I should have said so in my last post, but I certainly do agree 
with you that it would dilute the usefulness of the articles if they were all written to be 
accessible to "laypeople"; this kind of simplification would not be as useful to many 
researchers. I agree that a good solution would be to have the main summary or description written 
for researchers (who would be the primary audience), but also to include a "For 
laypeople" section, so that if anyone is inclined to rewrite the main summary for a more 
general audience, they could do so without affecting the more technical summary.


Right; what I meant was that while AW does use MW it doesn't *look like*
it does, and that's a barrier to entry, which matters. The default skin
needs to look more like default MediaWiki.

  >

Actually, I don't agree with Reid on this point. Appearance is very much
a subjective issue. Here's my purely subjective opinion:
* I find it irritating that hundreds or thousands of MediaWiki instances
all look like Wikipedia, as if MediaWiki didn't didn't have any skinning
flexibility. (I'm assuming that when Reid says "look like the default
MediaWiki", what he effectively means is "look like Wikipedia"; Reid,
please correct me if I'm misunderstanding you.)
* I like the AcaWiki interface; I wouldn't want to change it to look
like Wikipedia.

Less subjectively, I don't think that the appearance is a significant
barrier to entry. Saying, "It works just like Wikipedia" should do the
job fine to communicate the familiarity of the wiki language.

My concern is less with aesthetics than what the interface looks like it
does (the "apparent affordances" to use some jargon). As an analogy, I'm
sure many of you have encountered Java and Flash applications which have
all the same GUI widgets (buttons, scroll bars, etc.) as native OS apps,
but they look slightly different. Obviously one can overcome the
differences, but unfamiliarity makes the apps harder to use and turns
off newbies (or even experienced people who are sick of the
"specialness"). (Kai's Power Tools is a classic offender in this regard
- where are the controls in this screen shot and how do you use them?
http://en.wikipedia.org/wiki/File:Kai%27s_Power_Tools.jpg).

Wow, I don't know what he was thinking, but I guess Kai figured that displaying 
the power of Photoshop art was more important than usability! Maybe I should 
keep this example for the UI design portion of my systems design class :-)

I could certainly be wrong, but this is professional rather than
personal opinion, as someone with an HCI education. Sorry for the lack
of citations. I do agree that aesthetics is to some degree subjective.

I don't necessarily believe that we need it to be the standard MW look
in all respects (though I personally like the consistency), but the wiki
controls need to be consistent with other MW installs (most importantly,
Wikipedia) so people can see easily that it's a wiki and in particular
one they've used before.

Actually, the controls seem to me to be quite similar to the standard Wikipedia 
layout. For example, look at 
http://acawiki.org/Measuring_user_influence_in_Twitter:_The_million_follower_fallacy.
 The page edit controls are on the top of the article, and the navigation bar 
is on the left, all very similar to Wikipedia. Since these key functional 
elements are very similar to the default, I assumed that your comments had more 
to do with the aesthetic elememts. Could you perhaps point out some specific 
differences in the core MediaWiki functionality elements that you think might 
confuse new users who are familiar with editing Wikipedia?

Actually, another reason for my comments is that I would assume that the core 
audience of contributors (academic researchers who are willing to share their 
research summaries online) would not have trouble trying to learn how to edit, 
even if AcaWiki used something other than MediaWiki. Other user categories 
(such as academic researchers who are hesitant to release their contributions 
as CC-BY) might be less tech-savvy, but I assume that th

Re: [Wiki-research-l] Proposal: build a wiki literature review wiki-style

[Chitu Okoli]

Just a few comments adding on to Jodi and Reid's recent posts:


 Message original 
Sujet: Re: [Wiki-research-l] Proposal: build a wikiliteraturereview
wiki-style
De : Reid Priedhorsky 
Pour : Research into Wikimedia content and communities 

Copie à : Mike Linksvayer 
Date : March-23-11 2:18:34 PM



Neeru&  Mike, can you comment on who's doing sysadmin work now?

My point here is: I would like to depend on pros for the sysadmin work,
rather than volunteers, because there's no need for us to be sysadmins.
Let the experts be expert on what they're expert in and all that.

Bottom line: right now I'm not persuaded that the AcaWiki hosting
situation is stable. The key example is letting the domain expire and
the apparent lack of access to someone who can fix it (see
http://lists.ibiblio.org/pipermail/acawiki-general/2011-March/21.html and
http://code.creativecommons.org/issues/msg2778).



I don't know about the history of AcaWiki stability, but I strongly agree in 
principle that this project (whether we go with AcaWiki or WikiScholar or 
whatever) requires a reliable solution where someone is paid for sysadmin work 
to keep it stable.


The main interest, from my perspective (others may be able to add their
own), is in making research more accessible. Several AcaWiki users are
grad students who are writing summaries in order to consolidate their
own knowledge or prepare for qualifier exams.

OK, that's somewhat different than the goals being proposed in this thread.

I think that's a problem, but perhaps a surmountable one if different
communities can have different standards for their papers. We (or I)
need to be able to focus on writing "summaries" aimed at other
researchers; if someone else wants to come along and add additional
summaries for laypeople, that's fine. But (for example) if other people
start rewriting our lit review text because it's too technical, I don't
think it will work out.


Actually, I feel grad student summaries are an excellent contribution. Although 
they might not be perfect, grad student seminar assignments would probably be 
the largest single source of stub articles. Multiply that by every academic 
field that requires grad students to summarize articles, and I think promoting 
the wiki as an outlet for grad student work would be the single most effective 
strategy to make it huge in just one or two years. I, for one, very much see 
grad students as a major contributing community.



* It doesn't look like a MediaWiki. Since the MW software is so
dominant, that means pretty much everyone who knows about editing wikis
knows how to use MW - and not looking like MW means there's no immediate
"aha! I can edit this". There's a lot of value in familiarity.

Actually, AcaWiki uses MediaWiki -- specifically Semantic Media Wiki.

Right; what I meant was that while AW does use MW it doesn't *look like*
it does, and that's a barrier to entry, which matters. The default skin
needs to look more like default MediaWiki.

Actually, I don't agree with Reid on this point. Appearance is very much a 
subjective issue. Here's my purely subjective opinion:
* I find it irritating that hundreds or thousands of MediaWiki instances all look like Wikipedia, 
as if MediaWiki didn't didn't have any skinning flexibility. (I'm assuming that when Reid says 
"look like the default MediaWiki", what he effectively means is "look like 
Wikipedia"; Reid, please correct me if I'm misunderstanding you.)
* I like the AcaWiki interface; I wouldn't want to change it to look like 
Wikipedia.

Less subjectively, I don't think that the appearance is a significant barrier to entry. 
Saying, "It works just like Wikipedia" should do the job fine to communicate 
the familiarity of the wiki language.


My only experience with "scraping" pages is with Zotero, and it does it
beautifully. I assume (but don't know) that the current generation of
other bibliography software would also do a good job. Anyway, Zotero has
a huge support community, and scrapers for major sources (including
Google Scholar for articles and Amazon for books) are kept very well up
to date for the most part.


> Perhaps I'm just unlucky, then - I've only ever tried it on ACM papers
> (which it failed to do well, so I stopped).

>> Zotero used to scrape quite well from the ACM digital library -- now that
>>  they've changed their site again the scraper needs to be updated (not hard
>> to do). Last time I tried, Zotero scraped ok from certain ACM pages (item
>> pages) but not from search results: YMMV.
>> -Jodi

That's probably the issue. We also were stuck for a while when ACM's 
reformatting of their page structure broke the Zotero translators. For our 
review, Mohamad on our team had to rewrite the ACM Zotero translator from 
scratch. I think the problem has been fixed now, though, on the official Zotero 
translator package. Thus, Reid, you probably just happened to get a bad egg. 
Unfortunately, whenever publishers reform

Re: [Wiki-research-l] Wikipedia literature review

[Chitu Okoli]

thanks, Paolo; we'll definitely be referring to Finn's working paper.

~ Chitu


 Message original 
Sujet: Re: [Wiki-research-l] Wikipedia literature review - include or exclude 
conference articles
De : paolo massa 
Pour : afo...@gatech.edu, Research into Wikimedia content and communities 

Date : March-23-11 5:47:00 AM

I just posted on by blog at

http://www.gnuband.org/2011/03/23/paper_wikipedia_research_and_tools_review_and_comments/

a post about the draft paper “Wikipedia research and tools: Review and
comments” by Finn Arup Nielsen.

It is a 56-pages resource highlighting key areas of research for
Wikipedia (with citations to relevant work already published).
The cited papers (with annotations!) are 236!

You might want to check it and maybe involve Finn in the effort of
aggregating the literature relevant to Wikipedia research.
Take care!



___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Wikipedia literature review - include or exclude conference articles

[Chitu Okoli]

Thanks, Andrea. We're going to play around with citation analysis from ACM 
Digital Library, since it's better structured than Google Scholar (though, of 
course, it doesn't have nearly as many citations). I'll send an update when 
we've got something.

~ Chitu


 Message original 
Sujet: Re: [Wiki-research-l] Wikipedia literature review - include or exclude 
conference articles
De : Andrea Forte 
Pour : Research into Wikimedia content and communities 

Date : March-22-11 4:19:15 PM

Chitu - these are not machine-generated results. :) It's the work of a human!

I searched for "wikipedia" and "wikipedian" in google scholar and rejected the 
ones that I felt were not actually *about* wikipedia itself, then I searched for fernanda's paper 
because I know it is one of the top cited papers about wikipedia (note that it does NOT have 
wikipedia in the title) to come up with a set of 10-15 readings that our team could all start with.

Andrea


On Mon, Mar 21, 2011 at 5:06 PM, Chitu Okoli mailto:chitu.ok...@concordia.ca>> wrote:

Hi Andrea,

I can see from yours and others' comments that conference ranking sites are 
not that helpful (to put it mildly). I'll respond to the matter of the shared 
bibliography in the separate thread on that topic.

I'm leaning towards exploring citation counts more. How did you get the 
counts you gave from Google Scholar? Could you give us your exact search 
keywords? I couldn't reproduce what you gave us; perhaps I don't know how to 
use Google Scholar that flexibly.

One useful tool for citation analysis with Google Scholar is Harzing's Publish or 
Perish (freeware, not free software): http://www.harzing.com/pop.htm. By doing a simple 
search on "Wikipedia" in the General Citations keywords, I got the following as 
the top ten results:

Cites   Authors Title   YearSource  ArticleURL
913 J Giles Internet encyclopaedias go head to head 2005
Nature  
http://www.nature.com/nature/journal/v438/n7070/full/438900a.html?&$NMW_TRANS$=ext 
<http://www.nature.com/nature/journal/v438/n7070/full/438900a.html?&$NMW_TRANS$=ext>
385 E Gabrilovich…  Computing semantic relatedness using 
wikipedia-based explicit semantic analysis 2007Proceedings of the 
20th International Joint …   
http://www.aaai.org/Papers/IJCAI/2007/IJCAI07-259.pdf
358 M Völkel, M Krötzsch, D Vrandecic…  Semantic wikipedia  
2006Proceedings of the …http://portal.acm.org/citation.cfm?id=1135863
281 L Denoyer…  The wikipedia xml corpus2006ACM 
SIGIR Forum http://portal.acm.org/citation.cfm?id=1147210
260 A Bruns Blogs, Wikipedia, Second Life, and beyond: From 
production to produsage 2008

228 M Strube…   WikiRelate! Computing semantic relatedness 
using Wikipedia  2006Proceedings of the National Conference on … 
http://www.aaai.org/Papers/AAAI/2006/AAAI06-223.pdf
201 A Lih   Wikipedia as participatory journalism: Reliable 
sources? metrics for evaluating collaborative media as a news resource  2003
Nature  
197 J Voss  Measuring wikipedia 2005International 
Conference of the International Society for … 
http://eprints.rclis.org/handle/10760/6207
161 SP Ponzetto…Deriving a large scale taxonomy from Wikipedia  
2007Proceedings of the national conference on … 
http://scholar.google.comhttps://www.aaai.org/Papers/AAAI/2007/AAAI07-228.pdf
158 BT Adler…   A content-driven reputation system for the 
Wikipedia2007… of the 16th international conference on … 
http://portal.acm.org/citation.cfm?id=1242608


These are pretty different from your results. Could you tell me how you got 
yours?

Thanks,
Chitu


 Message original 
Sujet: [Wiki-research-l] Wikipedia literature review - include or exclude 
conference articles
De : Andrea Forte  <mailto:andrea.fo...@gmail.com>

Pour : Research into Wikimedia content and communities
 
<mailto:wiki-research-l@lists.wikimedia.org>
Date : 18/03/2011 9:33 AM

Hi all, after following these links:


* Top Tier and 2nd tier conferences from
http://webdocs.cs.ualberta.ca/~zaiane/htmldocs/ConfRanking.html  
<http://webdocs.cs.ualberta.ca/%7Ezaiane/htmldocs/ConfRanking.html>
* A-ranked conferences in Information and Computing Sciences from
http://lamp.infosys.deakin.edu.au/era/?pageÏorsel10  
<http://lamp.infosys.deakin.edu.au/era/?page%CForsel10>

I discovered that CHI is not a top-tier HCI conference. Or even 2nd or 3rd. :) 
Neither is CSCW, Group, etc. (For those not in CS/HCI, CHI is *the* top tier conference 
with acceptance rates lower than most journals in the a

Re: [Wiki-research-l] Proposal: build a wiki literature review wiki-style

[Chitu Okoli]

Hi Reid,

My responses to your responses are inline.


 Message original 
Sujet: Re: [Wiki-research-l] Proposal: build a wiki literaturereview
wiki-style
De : Reid Priedhorsky 
Pour : wiki-research-l@lists.wikimedia.org
Date : March-22-11 11:56:24 AM


(I don't know the details of MediaWiki search well, so some of the
following may not be quite right.) What MediaWiki would give us is
fulltext search. So while it would be easy to search for "John Smith",
and that query would find papers authored by John Smith plus perhaps
other stuff; however, one cannot search for "author = John Smith" and
get only results where the author field matches John Smith and no others.

However, it does seem like Semantic MediaWiki has this type of search
and otherwise behaves much like plain MediaWiki.



I actually wasn't familiar with the full functionality of Semantic MediaWiki 
(http://semantic-mediawiki.org/wiki/Semantic_MediaWiki) until I looked it up 
after your comments. From what I can see, it certainly seems to have the 
capabilities to maintain all the key metadata that would be necessary for 
myself and I assume most other researchers (e.g. authors, dates, publication 
source, URLs to HTML or PDF versions, etc.).


There also appear to be various options for Semantic MediaWiki hosting:
Wikia, Referata, etc. It would be nice to not have to deal with the
sysadmin aspects of the project.

I agree that going with a reliable host would be the way to go. I think that 
for the nature of our project, choosing a paid Referata plan would probably be 
better than going for Wikia. I for one could probably easily find grant funding 
to keep it going.


One final note on bibliographic software: many of these claim to do
automatic import of a reference simply by pointing the software at the
publisher's web page for the references. But I have never seen this work
correctly; always, the imported data needs significant cleanup, enough
that personally I'd rather type it in manually anyway. For example,
titles of ACM papers aren't even correctly cased on the official ACM
pages (e.g., http://dx.doi.org/10.1145/1753326.1753615)!

My only experience with "scraping" pages is with Zotero, and it does it 
beautifully. I assume (but don't know) that the current generation of other bibliography 
software would also do a good job. Anyway, Zotero has a huge support community, and 
scrapers for major sources (including Google Scholar for articles and Amazon for books) 
are kept very well up to date for the most part.


Bibliographic software then also typically does not include the proper
metadata for automatically lower-casing titles in citations. For
example, the title "Path Selection: Novel Interaction Technique for
Wikipedia" should be lower-cased as "Path selection: Novel interaction
technique for Wikipedia". But so often I see papers with "Path
selection: novel interaction technique for wikipedia". It's embarrassing.

That's definitely a software design flaw; Zotero is certainly rather bad at 
this point.

But, if we were writing our own (e.g.) MediaWiki ->  BibTeX export
script, we could automatically note that "Novel" should be capitalized
(because it begins the subtitle) as well as provide for people to
indicate explicitly title words that should remain capitalized. (In this
instance, the proper BibTeX export syntax would be "Path Selection:
{Novel} Interaction Technique for {Wikipedia}".)

I like the idea of including export facilities in our SMW version, giving users 
the option of what they would like to export to.

Would it be feasible to have both, and use them concurrently so that
  researchers could use one or the other, or both, as they prefer? I'm
  thinking of something like this (for purpose of illustration, let's
call the chosen MediaWiki instance MW and the chosen dedicated online
shared bibliographic tool BT):

Bi-directional synchronization is hard to get right, particularly when
the two sides have different data models. I think we are much
better off declaring one or the other to be the master and the rest
should remain read-only (i.e. export rather than synchronization).

I like this idea; with SMW as the primary, editable source, a read-only Zotero 
library imported from the SMW would work well. The problem, though, is that 
duplicate detection would need to prevent imports from adding existing 
articles. A complete overwrite would not work, since this would break article 
IDs for word processor integration. Zotero has been slow on implementing 
duplicate detection, but they finally have a very impressive solution in alpha 
(http://www.zotero.org/blog/new-release-multilingual-zotero-with-duplicates-detection/).

Thanks, Reid, for your great suggestions. I hope this can become a reality.

~ Chitu
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Wikipedia literature review - include or exclude conference articles

[Chitu Okoli]

Thanks, John. It's great to get an Australian perspective on the ERA ranking. 
You've definitely confirmed that it's not very useful for our purposes here. 
So, I'll pass on the CSV copy of the 2012 version :-(

~ Chitu

 Message original 
Sujet: Re: [Wiki-research-l] Wikipedia literature review - include or exclude 
conference articles
De : John Vandenberg 
Pour : Research into Wikimedia content and communities 

Date : Lun. 21 Mars 2011 23:06:06 EST

On Thu, Mar 17, 2011 at 8:41 AM, Chitu Okoli  wrote:
...

* A-ranked conferences in Information and Computing Sciences from
http://lamp.infosys.deakin.edu.au/era/?page=cforsel10: This is the most
exhaustive journal ranking exercise I have ever found anywhere.

With regards to John Lamps journal list, it is a copy of the *first*
ERA journal list.

http://en.wikipedia.org/wiki/Excellence_in_Research_for_Australia

There is a second ERA journal list being compiled for 2012.
Submissions closed yesterday, and review of ranking is now underway.

The journal list can be browsed via the website.

https://roci.arc.gov.au/

However there is no publicly download-able dataset available yet.

If anyone wants a copy of the second ERA journal list in xml or csv, I
can provide it offlist.

Public consultation about the ranking is open until April 4.

Unfortunately, I like you have serious questions about the face validity of
these rankings; I think they heavily overrate many conferences in my own
field of information systems; I assume the same is true with other fields
that I don't know so well. (My primary reservation with conference or
journal rankings by professors is that I strongly suspect that one of the
main criteria for their rankings is whether or not they have published in
that outlet before.) Unfortunately, I don't know of anything that approaches
this ranking in comprehensiveness.

One important point to note in regards to conferences in that journal
list is that conferences are only ranked for the disciplines of

* 08 Information and computer science
http://www.abs.gov.au/AUSSTATS/abs@.nsf/Latestproducts/4C3249439D3285D6CA257418000470E3?opendocument

* 09 Engineering
http://www.abs.gov.au/AUSSTATS/abs@.nsf/Latestproducts/050A7395E86A9719CA257418000477A2?opendocument

* 12 Built environment and design
http://www.abs.gov.au/AUSSTATS/abs@.nsf/Latestproducts/B20002D4CAD6966DCA257418000498EA?opendocument

IMO the ranked conference list was useless in the 2010 ERA process and results.
I've yet to see any improvement in this area for the 2012 ERA.

--
John Vandenberg

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Request to verify articles for Wikipedia literature review

[Chitu Okoli]

Hi Piotr,

Thanks for adding the article. I see that Interface is a relatively new 
journal, and open access as of November 2010. Unfortunately, open access 
journals are often poorly indexed in major commercial research databases, so 
this is exactly the kind of paper we're looking for by making this request. 
Thanks!

Yes, please do send me your 2007 working paper; I'd certainly like to take a 
look at it. Please e-mail it directly to me.

Regards,
Chitu


 Message original 
Sujet: Re: [Wiki-research-l] Request to verify articles for Wikipedia
literature review
De : Piotr Konieczny 
Pour : Research into Wikimedia content and communities

Date : 19/03/2011 10:16 PM

Chitu Okoli wrote:


We have now updated the following page with the peer-reviewed journal articles 
and doctoral theses we have identified: 
http://en.wikipedia.org/wiki/Wikipedia:Academic_studies_of_Wikipedia. Please 
note that we have only updated the sections on peer-reviewed journal articles 
and on theses; we have not updated other sections with newly identified 
studies, except for correcting some misclassified items.


I am glad somebody found time and will to update this page again. I stopped 
doing so two or three years back, sadly.


To help us in identifying all eligible studies, we would really appreciate it 
if you could look at the sections on peer-reviewed journal articles and theses 
in http://en.wikipedia.org/wiki/Wikipedia:Academic_studies_of_Wikipedia, and 
send us any citations (by yourself or others) that you know are missing. In 
particular,  please inform us of:


I see you found three out of my four papers, I will add the missing one :>

A while ago I started a draft of a paper in which I attempted to review the 
Wikipedia research up to 2007. I never finished it, but perhaps you'd find it 
of use. Let me know if you'd like a copy.




___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Proposal: build a wiki literature review wiki-style

[Chitu Okoli]

Hi everyone,

I see two distinct lines forming from this thread. Most people are advocating a 
MediaWiki instance dedicated to semantically rich shared summaries of scholarly 
articles (primarily Wikipeda-focused, but such a platform could obviously be 
used for any kind of scholarly article). However, Dario and Felipe seem to be 
arguing (Dario more than Felipe) that a dedicated bibliographic software tool 
would be far more useful and easier to maintain long term.

Personally, with the energy I see on the list, I believe that a MediaWiki 
instance (e.g. AcaWiki or BredeWiki) would be viable, if we tried it again, 
this time. I for one would be very happy to make such a resource my repository 
of research summaries--I am currently building a collection on my personal 
website, but a shared resource would obviously be 1,000 times better. If one 
MediaWiki instance could be agreed on by this community, I believe we could 
definitely build a very valuable, ongoing resource. It would be far easier to 
edit and contribute to such an instance than to any chosen dedicated 
bibliographic tool.

However, I do strongly share Dario's feelings that a dedicated bibliographic 
tool would be far more useful for a variety of uses. One of my most important 
functionalities would be automatic citations into papers that I'm working with. 
I haven't used a wide variety of citation managers, but the functionality in 
EndNote and Zotero is what I'm talking about; I just don't see how a MediaWiki 
instance could do that, unless some standardized bibliographic information be 
embedded into each article page to begin with. Moreover, as Dario and Felipe 
explained, while far from perfect, the search capabilities of dedicated 
bibliography managers is far superior to what I presently see in MediaWiki.

Given that there is the need for both resources (MediaWiki instance and a 
dedicated bibliographic tool), I think it is much easier to automatically and 
regularly export from a shared resource like Zotero (for example) to the 
MediaWiki tool than to go the other way around. For this reason, I tend to 
favour Dario's proposal.

Would it be feasible to have both, and use them concurrently so that 
researchers could use one or the other, or both, as they prefer? I'm thinking 
of something like this (for purpose of illustration, let's call the chosen 
MediaWiki instance MW and the chosen dedicated online shared bibliographic tool 
BT):

* All the articles would be represented on both MW and BT.
* Automatic exports would be enabled via custom scripts from BT to MW; there 
would be no automatic script for going the other direction.
* Each MW article would have two sections:
-- The top section would be automatically generated from BT, and would not be 
user editable from within MW; it could only be edited by the equally open BT.
-- The bottom section would be standard MW editable text. Users would be 
expected to not duplicate information already in the top BT section.
* Users who use both MW and BT could copy contributions from the MW bottom 
(editable) section into BT. Then after these are automatically exported 
(perhaps once a week), MW users could remove the duplicate information from MW.
* Thus, MW would always have a superset of BT information (at least, after the 
weekly export).

The benefit is that those who need full BT functionality would have it, along 
with most of the summary information included; those who don't need the 
functionality and don't want to deal with the disadvantages of the BT 
technology would have all the BT information (including metadata) within MW, 
and would also be able to contribute to MW, which would always have at least as 
much information as BT.

Does this make sense? Would it be useful? Does it sound feasible?

Regards,
Chitu



 Message original 
Sujet: Re: [Wiki-research-l] Fwd: Proposal: build a wiki literature review
wiki-style (was: Re: Wikipedia literature review - include orexclude 
conference articles)
De : Felipe Ortega 
Pour : Research into Wikimedia content and communities

Date : 18/03/2011 2:29 PM

Hi all,

I've been following this discussion with interest. Please let me add some
comments inline, complementing Dario's answer.

- Mensaje original 

De: Dario Taraborelli
Para: afo...@gatech.edu; Research into Wikimedia content and communities

Enviado: vie,18 marzo, 2011 17:30
Asunto: Re: [Wiki-research-l] Fwd: Proposal: build a wiki literature review
wiki-style (was: Re: Wikipedia literature review - include or exclude conference
articles)

I was glad to see this thread on wikiresearch-l as I have been recently
discussing a similar proposal with other members of the Wikimedia Research
Committee.

To make a long story short: I see major problems about *maintaining* a shared
reference pool (along with a lit review system) on wiki pages, no matter how
standard the format we may come up with to do so.
There are excellent free and standards-based services ou

Re: [Wiki-research-l] Wikipedia literature review - include or exclude conference articles

[Chitu Okoli]

Hi Andrea,

I can see from yours and others' comments that conference ranking sites are not 
that helpful (to put it mildly). I'll respond to the matter of the shared 
bibliography in the separate thread on that topic.

I'm leaning towards exploring citation counts more. How did you get the counts 
you gave from Google Scholar? Could you give us your exact search keywords? I 
couldn't reproduce what you gave us; perhaps I don't know how to use Google 
Scholar that flexibly.

One useful tool for citation analysis with Google Scholar is Harzing's Publish or Perish 
(freeware, not free software): http://www.harzing.com/pop.htm. By doing a simple search 
on "Wikipedia" in the General Citations keywords, I got the following as the 
top ten results:

Cites   Authors Title   YearSource  ArticleURL
913 J Giles Internet encyclopaedias go head to head 2005
Nature  
http://www.nature.com/nature/journal/v438/n7070/full/438900a.html?&$NMW_TRANS$=ext
385 E Gabrilovich...Computing semantic relatedness using 
wikipedia-based explicit semantic analysis 2007Proceedings of the 
20th International Joint ... 
http://www.aaai.org/Papers/IJCAI/2007/IJCAI07-259.pdf
358 M Völkel, M Krötzsch, D Vrandecic...Semantic wikipedia  2006
Proceedings of the ...  http://portal.acm.org/citation.cfm?id=1135863
281 L Denoyer...The wikipedia xml corpus2006ACM SIGIR Forum 
http://portal.acm.org/citation.cfm?id=1147210
260 A Bruns Blogs, Wikipedia, Second Life, and beyond: From 
production to produsage 2008

228 M Strube... WikiRelate! Computing semantic relatedness using 
Wikipedia  2006Proceedings of the National Conference on ...   
http://www.aaai.org/Papers/AAAI/2006/AAAI06-223.pdf
201 A Lih   Wikipedia as participatory journalism: Reliable sources? 
metrics for evaluating collaborative media as a news resource  2003Nature  
197 J Voss  Measuring wikipedia 2005International Conference of the 
International Society for ...   http://eprints.rclis.org/handle/10760/6207
161 SP Ponzetto...  Deriving a large scale taxonomy from Wikipedia  2007
Proceedings of the national conference on ...   
http://scholar.google.comhttps://www.aaai.org/Papers/AAAI/2007/AAAI07-228.pdf
158 BT Adler... A content-driven reputation system for the Wikipedia
2007... of the 16th international conference on ... 
http://portal.acm.org/citation.cfm?id=1242608


These are pretty different from your results. Could you tell me how you got 
yours?

Thanks,
Chitu


 Message original 
Sujet: [Wiki-research-l] Wikipedia literature review - include or exclude 
conference articles
De : Andrea Forte 
Pour : Research into Wikimedia content and communities

Date : 18/03/2011 9:33 AM

Hi all, after following these links:


* Top Tier and 2nd tier conferences from
http://webdocs.cs.ualberta.ca/~zaiane/htmldocs/ConfRanking.html
* A-ranked conferences in Information and Computing Sciences from
http://lamp.infosys.deakin.edu.au/era/?pageÏorsel10

I discovered that CHI is not a top-tier HCI conference. Or even 2nd or
3rd. :) Neither is CSCW, Group, etc.

(For those not in CS/HCI, CHI is *the* top tier conference with
acceptance rates lower than most journals in the area and the others I
listed are nearly as competitive. And venues like WikiSym that have
higher acceptance rates attract top-tier work as well - many papers of
equivalent quality have been published there.)

So... I would suggest that, to review the Wikipedia literature, you
need to choose a field (or set of fields) you know and become deeply
knowledgeable about the literature in that field by reading it and
following citations, etc. Searching for every paper ever written that
mentions "Wikipedia" in the title/abstract will catch you lots of
peripheral work that is not *about* Wikipedia, but that uses Wikipedia
as a context for studying something else. Machines aren't good at
literature reviews. :)

That said, it would be incredibly useful to have a common repository
of citations that can be annotated, discussed etc. Reid Priedhorsky,
Phoebe Ayers, Brent Hecht, Darren Gergle, and Mako Hill and I have
been talking about doing a literature review as well and have come to
the conclusions that A) there's too much to cover in just one paper
and B) a place to collaboratively assemble knowledge about the
literature is a prerequisite for such an endeavor. We were thinking a
MediaWiki with templates to structure citation data for export would
be better than any of the bib software out there. But actually... I
remember reading that Tiki Wiki now explicitly supports citation, I
think?

The bottom line is, B) Is something this community could really do a
great job of developing and it would be mutually beneficial. No one of
us is going to cover all the Wikipedia literature alone, unless we
make it a full time job. :)

Re: [Wiki-research-l] Wikipedia literature review - include or exclude conference articles

[Chitu Okoli]

Hi Jack,

Actually, the reason we're talking about top-tier is based on the same reason 
we talk about peer-reviewed versus non-peer-reviewed. No one can argue that 
non-peer-reviewed work (such as working papers) often have completely novel 
ideas. The problem is that someone has to wade through tens of thousands of 
works of hugely varying quality to find a few pearls. The peer-review process 
does this wading; while it might miss a few novel items, it would probably get 
most of the high-quality ones. Similarly, there are at least 2,000 Wikipedia 
studies. Since we can't go through all of them, we hope that most of the 
high-quality novel ideas do appear in publication outlets that are universally 
recognized to be of higher quality than average.

Thanks,
Chitu


 Message original 
Sujet: Re: [Wiki-research-l] Wikipedia literature review - include or exclude 
conference articles (was Request to verify articles for Wikipedia literature 
review)
De : Jack Park 
Pour : Research into Wikimedia content and communities

Date : 15/03/2011 5:26 PM

When you consider a "top tier conference", how do you know you are not
excluding contributions that might be not just novel but also truly
important?

It seems that page rank plays the role of beauty contest in the sense
that top-ranked pages are those already in the view of others. I have
seen comments that this filters against novelty, possibly crucial
novelty.

Jack

On Tue, Mar 15, 2011 at 11:56 AM, Chitu Okoli  wrote:

We considered including top-tier conferences, but the question is, what is a
"top conference"? In trying to answer this, we looked at a couple of
sources:
* Top Tier and 2nd tier conferences from
http://webdocs.cs.ualberta.ca/~zaiane/htmldocs/ConfRanking.html
* A-ranked conferences in Information and Computing Sciences from
http://lamp.infosys.deakin.edu.au/era/?pageÏorsel10
* We also considered including all WikiSym articles on Wikipedia


___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Wikipedia literature review - conference articles

[Chitu Okoli]

Hi Jack,

We're using the paid version and have plenty of space. Our issue is not storage 
space; it is that Zotero chokes on syncing huge databases, and ours is much 
larger than most: 
http://forums.zotero.org/discussion/8413/2/syncing-large-collections/

~ Chitu


 Message original 
Sujet: Re: [Wiki-research-l] Wikipedia literature review - conference
articles
De : Jack Park 
Pour : Research into Wikimedia content and communities

Date : 15/03/2011 5:28 PM

Chitu,

Are you using the not-free cloud storage for synching on Zotero? The
free storage saturates quite quickly, but the cloud version seems
limitless (other than the cpu cycles it takes to synch).

Jack

On Tue, Mar 15, 2011 at 11:37 AM, Chitu Okoli  wrote:

Hi Jodi,

We're using Zotero. We like it because it's open source, but we've found
that its syncing capabilities are simply not yet capable of handling a
literature review as large as ours. But that's what we've got for now.
Zotero references can easily be exported into many other formats.

Do you have some ideas?

~ Chitu
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Proposal: build a wiki literature review wiki-style

[Chitu Okoli]

Hi Reid,

This is a fabulous idea. In fact, it is in this line that we put out all our 
results so far on the WP:ACST page, though we never thought of something so 
ambitious. Here are my responses to your five steps:

1. Create a public Mediawiki instance: I take it that your group is handling 
this? I have a server I could contribute, but we probably wouldn't be able to 
handle the server admin.
2. Decide on a relatively standardized format of reviewing each paper (metadata 
formats, an infobox, how to write reviews of each, etc.): In our initial thread 
deciding this project, we listed a bunch of research questions; this could be 
used as a draft of some key aspects of papers that researchers would like to 
know.
3. Upload your existing Zotero database into this new wiki (I would be happy to 
write a script to do this): We'll be glad to share our Zotero database as you 
suggested. We could export it as Zotero RDF XML (which would save almost all 
the data, though we have a huge bank of PDFs as well), or as any format into 
which Zotero exports.
4. Proceed with paper readings, with the goal that every single paper is looked 
at by human eyes: Comments on this below. However, we hope that the results of 
our literature review could give a major boost to this project.
5. Use this content to produce one or more review articles: Comments on this 
below

The stickiest part of this project, though, has to do with authorship issues. 
In fact, so sticky and so important that I'll continue it on another thread. 
However, I am very interested in such a collaboration. And I fully agree with 
you that it is sufficiently important to all of us that it needs to go forward 
in some form, even if all authorship issues are not yet resolved.

And yes, I'm subscribed to the list, as are Arto, Mohamad and Mostafa. I've 
been a silent lurker for a couple years now :-). So, no need to CC me 
separately.

Thanks,
Chitu

 Message original 
Sujet: Proposal: build a wiki literature review wiki-style (was: Re: Wikipedia 
literature review - include or exclude conference articles)
De : Reid Priedhorsky 
Pour : Research into Wikimedia content and communities 

Copie à : Chitu Okoli 
Date : March-16-11 12:20:47 PM

Chitu and others,

I too see great need for a comprehensive survey paper in this field. My own 
personal interest is in one that covers wiki research in general, not just 
research of Wikipedia; this of course makes the intractable number of papers 
even more intractable.

In fact, I am involved with a team of researchers with the same goal as you, 
though we are just getting started.

It seems to me that you are in a very difficult position. As others have noted, 
the scoping filter you propose is not a good one, but the number of papers is 
simply intractable without a very aggressive filter that excludes 2/3 or more 
of the known papers. (To further complicate the issue, I am skeptical of 
machine filtering period, fearing that any useful filter would necessarily be 
complex and difficult to justify in a writeup.)

However, I believe that there is a solution, and that is to dramatically 
increase the team size by doing the analysis wiki style. Rather than a small 
team creating the review, do it in public with an open set of contributors. 
Specifically, I propose:

1. Create a public Mediawiki instance.
2. Decide on a relatively standardized format of reviewing each paper (metadata 
formats, an infobox, how to write reviews of each, etc.)
3. Upload your existing Zotero database into this new wiki (I would be happy to 
write a script to do this).
4. Proceed with paper readings, with the goal that every single paper is looked 
at by human eyes.
5. Use this content to produce one or more review articles.

The goals of the effort would be threefold.

* Create an annotated bibliography of wiki research that is easy to keep up to 
date.
* Identify the N most important papers for more focused study and synthesis 
(perhaps leading towards more than one survey article).
* Provide metadata on the complete set of papers so that it can be described 
statistically.

Simply put, I believe that we as modern researchers need to be able to build 
survey articles which analyze 2,000-5,000 or more papers, and maybe this is a 
way to do that.

I and the other members of my team have already planned significant time 
towards this effort and would be very excited to join forces to lead such a 
mass collaboration.

Why use Mediawiki rather than Zotero or some other bibliography manager? First, 
it would be easy for anyone to participate because there is no software to 
install, no database to import, etc. Second, I personally have found Zotero, 
CiteULike, and every other bibliography manager I've tried to be clunky and 
tedious to use and not flexible enough for my needs (for example, three-state 
tags that let us say a paper has, does not have, or we do not know if it has, a 
certain pro

Re: [Wiki-research-l] Wikipedia literature review - include or exclude conference articles

[Chitu Okoli]

Hi Travis,

I thought that was you when I read your post; yes, we did indeed talk. Actually, it was 
after our talk that I went through extensive searching to find what is considered 
top-tier in computer science. Here are brief comments I should have included earlier 
explaining how I came up with the three sources of computer science "high 
quality" conferences:

* Top Tier and 2nd tier conferences from
http://webdocs.cs.ualberta.ca/~zaiane/htmldocs/ConfRanking.html: In extensive 
searching for computer science conference rankings, this is the absolute best I 
could find, and most other rankings I found have either referred to or copied 
from this list.
* A-ranked conferences in Information and Computing Sciences from
http://lamp.infosys.deakin.edu.au/era/?page=cforsel10: This is the most 
exhaustive journal ranking exercise I have ever found anywhere. Unfortunately, 
I like you have serious questions about the face validity of these rankings; I 
think they heavily overrate many conferences in my own field of information 
systems; I assume the same is true with other fields that I don't know so well. 
(My primary reservation with conference or journal rankings by professors is 
that I strongly suspect that one of the main criteria for their rankings is 
whether or not they have published in that outlet before.) Unfortunately, I 
don't know of anything that approaches this ranking in comprehensiveness.
* We also considered including all WikiSym articles on Wikipedia: This is not 
because of any statement of WikiSym's quality, but simply because WikiSym is 
probably the closest thing that exists to an academic conference specifically 
for Wikipedia-related research.

Is there no widely-accepted listing of computer science conference rankings? You say, 
"Everyone in my field (HCI) pretty much knows what the first tier conferences are 
where wikipedia research is published." The problem is that I could say the same 
thing about my field, but another researcher would have a different list. There is 
generally consensus about the top two or three in any field, but the huge grey zone comes 
when you try to draw a line. Even your idea of getting small groups of experts to 
validate a number of conferences is pretty shaky, since another small group of experts 
would almost definitely give different results.

Citation counts are always a sticky issue; they depend mainly on indexing by 
citation count databases and recency of articles. However, I do consider them 
one of the most objective (not necessarily one of the best, but one of the most 
objective) criteria for paper quality. Based on your suggestion, I just now 
discovered that ACM Digital Library includes citation counts for conference 
papers. By way of brainstorming, I'm thinking of this possible inclusion rule:

* Calculate (a) the average citation count for Wikipedia articles (either only 
journal average, only conference average, or average of both), and (b) average 
citation count for each journal and/or conference that publishes Wikipedia 
research. (b) is basically (a) grouped by journal/conference.
* Rather than doing raw citation counts, we could try to calculate citations 
per year or some other weighting that recognizes that more recent articles 
would have fewer citations than older ones.
* Include all conference papers greater than the average (whichever average we 
choose) and/or include conference papers from all conferences greater than the 
average. Or we could include all conference papers whose average citations per 
year are greater than the average for journal articles. Or just include the top 
100 ranked conference papers, or however many we can handle.

Although still somewhat artificial, this could give possibly give us a somewhat objective 
basis to filter up the "higher quality" conference papers based on citation 
analysis.

I don't know if I'm trying to go far with this citation count possibility, but 
what do you all think?

Thanks again,
Chitu


 Message original 
Sujet: Re: [Wiki-research-l] Wikipedia literature review - include or exclude 
conference articles (was Request to verify articles for Wikipedia literature 
review)
De : Travis Kriplean 
Pour : Research into Wikimedia content and communities

Copie à : Chitu Okoli 
Date : 15/03/2011 5:26 PM

Hey there,

I sympathize with your dilemma...and I think we might have actually talked 
about this at Wikimania 2009. Unfortunately, while you may be satisfied that 
600 journal articles + theses is enough (I certainly would be too), you should 
be equipped to recognize that if you keep it that way you are systematically 
excluding large, significant bodies of research deriving from computer science 
and HCI. As you make this choice, read through one or two of these conference 
papers and measure it against the quality of a randomly selected set of journal 
articles in your set:
   - http://dub.wa

Re: [Wiki-research-l] Wikipedia literature review - include or exclude conference articles

[Chitu Okoli]

Sorry for the late responses; with classes, meetings, office hours, baby, and 
so on, I can't respond as fast as I'd like, but I'm really grateful for all the 
great responses.

Thanks, James for the ideas you've suggested; I summarize them thus:
* Publication date cut-off: We'll play with these and see how many we're left 
with.
* Randomize: ha ha ha
* Topic/empirical vs. conceptual/quantitative vs. qualitative: Actually, one of 
the features of our review is that we explicitly want to include non-computer 
science works in our review, many of which are conceptual, qualitative, and 
covering unusual topics (e.g. music). Any of these criteria would 
systematically exclude these articles. Unfortunately, we see that our journal 
vs. conference cut-off systematically excludes many computer science articles 
:-(
* Cited articles: We hadn't thought about this; I'll talk more about it in 
responding to Travis' thread.
* Adding more reviewers: I'll follow up on this in responding to Reid's thread.

Thanks a lot.

~ Chitu

 Message original 
Sujet: Re: [Wiki-research-l] Wikipedia literature review - include or
exclude conference articles (was Request to verify articlesfor Wikipedia 
literature review)
De : James Howison 
Pour : Research into Wikimedia content and communities

Date : 15/03/2011 4:57 PM

I am a little sheepish; clearly you've really struggled with this, it's 
certainly a huge amount of papers.

I'm tempted to ask what happens if you cut by publication date, but I suspect 
that that doesn't help much because of the accelerating rate of publication. In 
any case not entirely sure of the justification for not including older things, 
it's not as though one stops knowing them :)

Ah, I know: randomize ;) Ok, that's not really in the spirit of a review 
article.

Have you considered cutting by some first quick pass characteristics, such as 
topic (using some framework relevant to your interests, we used 
Input-Process-Output for organizing studies of FLOSS)/empirical vs 
conceptual/perhaps even quant. vs qual.  That is, of course, a lot of work just 
there but it seems to deal with the selection bias the best. That would also 
help give a conceptual focus to the review article.

To avoid the full selection bias of excluding conferences, perhaps you could include only 
those that are cited in your journal articles? (hmmm, issues there, but perhaps worth 
thinking about; could one seek out some variant of "the connected set" of 
articles, with some cutting factor on the strength of linkage to bring the number down to 
something managable?).

Adding people to your review team is another option, I'm sure you've thought 
about that. Difficulties there are obvious (a good review goes beyond 'tagging' 
articles and conducts cross-cutting conceptually organized perspective, hard to 
coordinate or build through disconnected work).

Best wishes for the work,
James

On Mar 15, 2011, at 14:56, Chitu Okoli wrote:


James and Travis, you bring up a point that we have struggled back and forth 
with for several months. We really, really would like to include conference 
articles, but we just can't see how we could handle many more articles than 
what we've got now. We've been working on and off on this project for over two 
years now. (You can find works in progress at the link at the bottom to my 
website.) We'd like to get it done eventually, and we can only handle so many 
articles.


___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Wikipedia literature review - include or exclude conference articles (was Request to verify articles for Wikipedia literature review)

[Chitu Okoli]

James and Travis, you bring up a point that we have struggled back and forth 
with for several months. We really, really would like to include conference 
articles, but we just can't see how we could handle many more articles than 
what we've got now. We've been working on and off on this project for over two 
years now. (You can find works in progress at the link at the bottom to my 
website.) We'd like to get it done eventually, and we can only handle so many 
articles.

We considered including top-tier conferences, but the question is, what is a "top 
conference"? In trying to answer this, we looked at a couple of sources:
* Top Tier and 2nd tier conferences from 
http://webdocs.cs.ualberta.ca/~zaiane/htmldocs/ConfRanking.html
* A-ranked conferences in Information and Computing Sciences from 
http://lamp.infosys.deakin.edu.au/era/?page=cforsel10
* We also considered including all WikiSym articles on Wikipedia

We identified which of the 1,500 conference papers from 
http://en.wikipedia.org/wiki/User:Moudy83/conference_papers were "top 
conferences" by those definitions, and we found over 400. On top of our 600 journal 
articles and doctoral theses, we think 1,000 papers is just too much for us to handle.

If we could somehow narrow it down to 100 relevant conference papers, we could add that 
in, but no more. However, how do we select which conferences are "must 
includes" while unfortunately leaving out the rest? We just don't know how to do 
this in a non-arbitrary, objective manner that would truly identify the top 100 
conference papers on Wikipedia that contribute to scholarly knowledge.

Any ideas on how to do this would be very much appreciated.

Regards,
Chitu



 Message original 
Sujet: Re: [Wiki-research-l] Request to verify articles for Wikipedia
literature review
De : Travis Kriplean 
Pour : Research into Wikimedia content and communities

Date : 14/03/2011 3:46 PM

As an HCI/CS researcher who has published at top peer-reviewed conferences 
about Wikipedia, but not journals, I'd like to echo James' statements. Journals 
are not the norm in CS/HCI research. Knowledge is shared through conferences, 
not journals.

On 3/14/11 11:32 AM, James Howison wrote:

Hi there,

Great project; massive but will be much appreciated.  We did something similar 
for empirical studies of Open Source, recently accepted at ACM Computing 
Surveys (PDF pre-print available here [1], article not in print until 2012 (!! 
that's another email entirely, bah))

I recognize the need to cut down the number of articles for review, we reviewed 
around 600 and that was a multi-year effort. We did that mainly by excluding 
conceptual (hence empirical) or passing reference articles (ie we did a 
two-step filter on many more articles), but were forced to only do journal 
articles for updates during the (long) revision process.  I regret that 
necessity, it decreases the utility of the work.

Given the publication venues of choice for many academics in this community I do wonder if you 
aren't shooting yourself in the foot by excluding peer-reviewed conferences and restricting to 
journals.  Personally I'd rather read a review that included the top journals and top conferences 
than one that included all journals.  Or even rather read a review over a shorter time period that 
included publications over journals and conferences, or on more specified topics. The interesting 
question is, "what do we know about wikipedia" not "what did we publish in journals 
about wikipedia".  In particular you will find you have systematically excluded the 
contribution of HCI authors.

Given the commendable and massive effort you are providing (and your approach 
to coverage below is really interesting), getting that wrong at the outset 
seems a shame.

Best regards,
James Howison

[1] Crowston, K., Wei, K., Howison, J., and Wiggins, A. (2012). Free (libre) 
open source software development: What we know and what we do not know. ACM 
Computing Surveys, 44(2):
http://floss.syr.edu/content/freelibre-open-source-software-development-what-we-know-and-what-we-do-not-know


On Mar 14, 2011, at 13:58, Chitu Okoli wrote:


Hi everyone,

We are a research group conducting a systematic literature review on 
Wikipedia-related peer-reviewed academic studies published in the English 
language. (Although there are many excellent studies in other languages, we 
unfortunately do not have the resources to systematically review these at any 
kind of acceptable scholarly level. Also, our study is about Wikipedia only, 
not about other Wikimedia Foundation projects. However, we do include studies 
about other language Wikipedias, as long as the studies are published in 
English.) We have completed a search using many major databases of scholarly 
research. In a separate thread, we will also talk about research questions 
related to our review.

Re: [Wiki-research-l] Request for feedback on research questions for Wikipedia literature review

[Chitu Okoli]

Wow, Joseph, we totally missed that. This is exactly the kind of thing we're 
looking for, a current bank of research questions. We hope our lit review will 
add a lot of new knowledge points (that is, identify lots of knowledge points 
that have some answers)!

Thanks,
Chitu


 Message original 
Sujet: Re: [Wiki-research-l] Request for feedback on research questions for 
Wikipedia literature review
De : Joseph Reagle 
Pour : Chitu Okoli 
Date : March-15-11 3:24:19 PM

On Tuesday, March 15, 2011, Chitu Okoli wrote:

Do you have any links you could give us that lists ongoing projects, from which 
we could draw the research questions?

http://strategy.wikimedia.org/wiki/Wikimedia-pedia

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Wikipedia literature review - conference articles

[Chitu Okoli]

Hi Jodi,

We're using Zotero. We like it because it's open source, but we've found that 
its syncing capabilities are simply not yet capable of handling a literature 
review as large as ours. But that's what we've got for now. Zotero references 
can easily be exported into many other formats.

Do you have some ideas?

~ Chitu


 Message original 
Sujet: Re: [Wiki-research-l] Wikipedia literature review - conference
articles
De : Jodi Schneider 
Pour : Research into Wikimedia content and communities

Date : 14/03/2011 2:28 PM

Hi Chitu,

On 14 Mar 2011, at 18:25, Chitu Okoli wrote:



Our question here is, what do we do with these conference articles? There is 
already a list of conference papers at 
http://en.wikipedia.org/wiki/Wikipedia:Academic_studies_of_Wikipedia#Conference_presentations_and_papers
 (WP:ACST), which currently lists around 230 conference articles. Here are some 
thoughts of what we could do:

* Merge the two lists. This would take too much time and effort, and since 
we're not going to actually review the conference articles, for us it's just 
not worth it. Of course, if someone else would like to do that, that would be 
great. The problem is that it's not a bit-by-bit job; since it involves merging 
tables, it seems to be an all-or-nothing operation.
What reference management software are you using? Perhaps there's a way to do 
this besides merging tables, or only merging 1 table with what you have...


-Jodi




___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Request for feedback on research questions for Wikipedia literature review

[Chitu Okoli]

Hi Joseph,

Do you have any links you could give us that lists ongoing projects, from which 
we could draw the research questions?

Thanks,
Chitu

 Message original 
Sujet: Re: [Wiki-research-l] Request for feedback on research questions for 
Wikipedia literature review
De : Joseph Reagle 
Pour : Chitu Okoli 
Copie à : wiki-research-l@lists.wikimedia.org
Date : March-14-11 3:46:24 PM

fyi: As a follow-up to the Foundation's Strategy project, I believe there is a 
lot of research going on that is not reflected on that page.


___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Wikipedia literature review - conference articles

[Chitu Okoli]

Hi everyone,

We sent a separate e-mail introducing our systematic literature review on 
Wikipedia-related peer-reviewed academic studies published in English. As we 
mentioned, we have identified over 2,100 peer-reviewed studies. This number of 
studies is far too large for conducting a review synthesis, and so we have 
decided to focus only on peer-reviewed journal publications and doctoral 
theses; we identified 638 such studies.

That leaves us with around 1,500 peer-reviewed conference articles, which we 
gathered from the ACM Digital Library (http://portal.acm.org) and IEEE 
Engineering Village (http://www.engineeringvillage.com). We have posted the 
full list at http://en.wikipedia.org/wiki/User:Moudy83/conference_papers. 
Unfortunately, the only criteria we have applied on selecting these articles is 
that "Wikipedia", "wikipedian" or "wikipedians" appears in the title, abstract 
or keywords. Thus, there are very likely some papers there that are only 
marginally related to Wikipedia. For the journal articles and doctoral theses 
we discuss in the other thread, we have verified each one to make sure that 
they are really substantially about Wikipedia; however, we haven't done this 
for these conference articles. We estimate that 5 to 20% of the articles may 
not actually be relevant.

Our question here is, what do we do with these conference articles? There is 
already a list of conference papers at 
http://en.wikipedia.org/wiki/Wikipedia:Academic_studies_of_Wikipedia#Conference_presentations_and_papers
 (WP:ACST), which currently lists around 230 conference articles. Here are some 
thoughts of what we could do:

* Merge the two lists. This would take too much time and effort, and since 
we're not going to actually review the conference articles, for us it's just 
not worth it. Of course, if someone else would like to do that, that would be 
great. The problem is that it's not a bit-by-bit job; since it involves merging 
tables, it seems to be an all-or-nothing operation.

* Add our list to the end of the WP:ACST list. This would leave lots of 
duplicates (probably between 100 and 200).

* Replace the WP:ACST list with our more complete list. This would lose the 
extra information in many of the current WP:ACST article listings.

Another significant problem is that adding these 1,500 conference articles 
would greatly lengthen an already extremely long page. Should the WP:ACST be 
subdivided into multiple pages?

What do think? We're really not sure the best way to put this useful 
information out, while retaining the value of what's already there.

Thanks for your help.

Chitu Okoli, Concordia University, Montreal, Canada
(http://chitu.okoli.org/professional/open-content/wikipedia-and-open-content.html)
Arto Lanamäki, University of Agder, Kristiansand, Norway
Mohamad Mehdi, Concordia University, Montreal, Canada
Mostafa Mesgari, Concordia University, Montreal, Canada

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Request for feedback on research questions for Wikipedia literature review

[Chitu Okoli]

Hi everyone,

We sent a separate e-mail introducing our systematic literature review on 
Wikipedia-related peer-reviewed academic studies published in English.

We hope that our review would provide useful insights for the research 
community. Thus, we would like to ask your help in reviewing the research 
questions we have developed for the data extraction and synthesis phases of our 
review. Currently, we address the following review research questions:

1. What high-quality research has been conducted with Wikipedia as a major 
topic or data source? As mentioned in the introductory e-mail, we have already 
identified over 2,100 studies, though we will only analyze the journal articles 
and doctoral theses in detail. We will group the articles by field of study.

2. What research questions have been asked by various sources, both academic 
scholarly and practitioner? We want to know both the subjects that the existing 
research has covered, and also catalogue key questions that practitioners would 
like to be answered, whether or not academic research has broached these 
questions. Also, we categorize the research questions based on their purposes. 
We have more comments on this research question below.

3. What theoretical frameworks and reference theories have been used to study 
the topic? We are very interested in theory-driven research on Wikipedia, and 
would like to identify and categorize such work.

4. What research designs have been employed to answer research questions? By 
"research design", we include all that is commonly called research 
"methodologies" or "approaches".

5. What kinds of data have been collected for research purposes? Specifically, 
we note the data collection techniques, the time dimension (one-time snapshot 
or longitudinal observations over time), the unit of analysis, the technique 
used for extracting Wikipedia data (e.g. live Wikipedia or Wikipedia clone 
server), the Wikipedia page type, and the Wikipedia language.

6. What conclusions have been made from existing research? That is, what 
questions from RQ2 have been answered, and what are these answers?

7. What questions from RQ2 are left unanswered? (These present directions for 
future research.)


Do you have any comments or feedback on these questions? On one hand, we want 
to extract useful data from the studies that could be helpful to researchers. 
On the other hand, we have to be pragmatic, considering what we can cover when 
we're dealing with over 600 peer-reviewed studies.


Beyond these basic questions, we have a special note regarding our RQ2, on the 
research questions that have been asked. In addition to the research questions 
that we extract from the articles, we want to know what questions are of 
interest that have not been studied. For this, we have identified a few banks 
of Wikipedia-related research questions. Of note to academics and researchers 
is the collection at 
http://en.wikipedia.org/wiki/Wikipedia:WikiProject_Wikidemia#Research_Questions.
 Could you please review this list and update that page directly with any 
additional questions?  Alternately, you could reply us directly, and we could 
update the list.

In addition, we are even more interested in questions that practitioners are 
asking, other than what researchers are asking. (Although we know that most 
Wikipedia researchers are also Wikipedia "practitioners", we define 
practitioner here as someone involved in the Wikipedia project who is not also 
a scholarly researcher.) Thus, we are sending a separate e-mail to wikipedia-l, 
foundation-l, and wikiEN-l asking them to update the list of Foundation 
research questions at 
http://meta.wikimedia.org/wiki/Wikimedia_Foundation_Research_Goals.

Thanks for your help.

Chitu Okoli, Concordia University, Montreal, Canada
(http://chitu.okoli.org/professional/open-content/wikipedia-and-open-content.html)
Arto Lanamäki, University of Agder, Kristiansand, Norway
Mohamad Mehdi, Concordia University, Montreal, Canada
Mostafa Mesgari, Concordia University, Montreal, Canada

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Request to verify articles for Wikipedia literature review

[Chitu Okoli]

Hi everyone,

We are a research group conducting a systematic literature review on 
Wikipedia-related peer-reviewed academic studies published in the English 
language. (Although there are many excellent studies in other languages, we 
unfortunately do not have the resources to systematically review these at any 
kind of acceptable scholarly level. Also, our study is about Wikipedia only, 
not about other Wikimedia Foundation projects. However, we do include studies 
about other language Wikipedias, as long as the studies are published in 
English.) We have completed a search using many major databases of scholarly 
research. In a separate thread, we will also talk about research questions 
related to our review.

As of the end of November 2010, when we stopped searching, we had identified 
over 2,100 peer-reviewed studies that have "wikipedia", "wikipedian", or 
"wikipedians" in their title, abstract or keywords. As this number of studies 
is far too large for conducting a review synthesis, we have decided to focus 
only on peer-reviewed journal publications and doctoral theses; we identified 
625 such studies. In addition, we identified around 1,500 peer-reviewed 
conference articles; we will discuss these in a separate thread.

In addition to the scholarly databases that we searched, we have very carefully 
compared the lists of studies from the following Wikimedia pages to verify what 
we may have missed:
* http://en.wikipedia.org/wiki/Wikipedia:Academic_studies_of_Wikipedia
* http://meta.wikimedia.org/wiki/Wiki_Research_Bibliography
* http://en.wikipedia.org/wiki/Academic_studies_about_Wikipedia
* http://en.wikipedia.org/wiki/Wikipedia:Wikipedia_in_research
* http://meta.wikimedia.org/wiki/Research

 From these pages, we identified an additional 13 journal articles and 3 
doctoral theses that we had not previously identified. These were either 
articles published after November 2010, articles in journals indexed in very 
few scholarly databases, a few European journals, and doctoral theses from 
outside North America. After adding these, we have identified a total of 638 
publications, of which 610 journal articles and 28 doctoral theses.  (However, 
as we begin to read these, we will remove some from our lists if we find that 
they are really not about Wikipedia.)

We have now updated the following page with the peer-reviewed journal articles 
and doctoral theses we have identified: 
http://en.wikipedia.org/wiki/Wikipedia:Academic_studies_of_Wikipedia. Please 
note that we have only updated the sections on peer-reviewed journal articles 
and on theses; we have not updated other sections with newly identified 
studies, except for correcting some misclassified items.

To help us in identifying all eligible studies, we would really appreciate it 
if you could look at the sections on peer-reviewed journal articles and theses 
in http://en.wikipedia.org/wiki/Wikipedia:Academic_studies_of_Wikipedia, and 
send us any citations (by yourself or others) that you know are missing. In 
particular,  please inform us of:
* Doctoral theses conducted outside North America
* Peer-reviewed articles in journals not well indexed by North American 
databases
* Peer-reviewed journal articles and doctoral theses published or accepted and 
forthcoming after November 2010.

Thanks for your help.

Chitu Okoli, Concordia University, Montreal, Canada
(http://chitu.okoli.org/professional/open-content/wikipedia-and-open-content.html)
Arto Lanamäki, University of Agder, Kristiansand, Norway
Mohamad Mehdi, Concordia University, Montreal, Canada
Mostafa Mesgari, Concordia University, Montreal, Canada

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] my ph. d. -- still formulating a research question

[Chitu Okoli]

  
  
Hi Joe,

Ziko van Dijk asked a very important question:
  
> And what is your faculty? Are you a mathematician, a sociologist?
In general, what is your academic background, and what is your
program, and what is your school? Without background about yourself,
it is hard to give you advice. For example, mathematics and
sociology, or computer science and education, use very different
research approaches.

Regards,
Chitu


    Chitu Okoli
Associate Professor in Management Information Systems
John Molson School of Business
Concordia University, Montréal

Phone: +1 (514) 848-2424 x2985
http://chitu.okoli.org/pro





  Message original 
  Sujet: Re: [Wiki-research-l] my ph. d. -- still formulating a
  research    question
  De : Ziko van Dijk 
  Pour : Research into Wikimedia content and communities
      
  Date : 23/11/2010 9:00 AM


  Hello Joe,

You are right, a good research question is the most important basis
for a thesis, and that it is why it is so difficult.
>From what I see in your mail, a-c is indeed like the three sides of
one and the same thing. What is overtly missing is the agent; "who" is
supposed to build?
In the Netherlands there is a project (government supported) to let
teachers create free textbooks or textbook materials so that the
government can save a lot of money ("Wikiwijs"). Is this kinda in the
direction you are thinking about?
And what is your faculty? Are you a mathematician, a sociologist? It
could be a good start to have look at the many projects to
collaboratively write textbooks, they exist also in English I am sure.
You could study them and tell what are the obstacles they are
struggling with?

Kind regards

Ziko van Dijk


2010/11/23 Joe Corneli :

  
So far, the best phrasing I've come up with is: "What stands in the
way of building and supplying low-cost, high-quality mathematics
education via the internet?"

The art of encyclopedia-building doesn't seem to carry over directly
to education.  This should be of fairly general concern (the Wikimedia
Foundation's mission is about developing and disseminating educational
content).

I think there's a knowledge gap in there, maybe more than one.  It's
much easier for me to think about "engineering solutions" than it is
to precisely specify a research problem question!!  In particular, I'm
thinking about

(a) building interactive textbooks that work for self-guided learners
(b) building technologies to support live tutorials over the web
(c) building infrastructure to help in developing good survey articles
or similar content

The faculty here might want me to "pick one", but this is hard for me
to do because I see each of these three approaches as being part of
the puzzle.  Asking how well one of them works in absence of the other
is a bit like asking how well a fish can breathe in the absence of
water.

So maybe the "research question" is about asking: What is the family
resemblance of (a)-(c)?  How do they work together as a system?  Or
maybe the question is about whether a given implementation of (a)-(c)
shows any promise?

I seem to be struggling to switch from a hacking-oriented way of
thinking about things to a research-oriented way of thinking about
things.  I'd appreciate some feedback from those of you in a position
to offer advice on these matters.


  
  




  


___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l