Re: [Wiki-research-l] Tech Talk livestream chat transcript (was Re: [Wikimedia Technical Talks] Data and Decision Science at Wikimedia with Kate Zimmerman, 26 February 2020 @ 6PM UTC)

[James Salsman]

I note now that the full chat transcript has been restored; thank you.

I am still interested in the answer to the question.

Sincerely,
Jim

On Wed, Feb 26, 2020 at 10:46 AM James Salsman  wrote:
>
> Just now I asked the following question on the Technical Talk
> livestream at https://www.youtube.com/watch?v=J-CRsiwYM9w
>
> 10:19 AM: Page 20 of Robert West's 2016 Stanford thesis, "Human
> Navigation of Information Networks" says, "We have access to
> Wikimedia’s full server logs, containing all HTTP requests to
> Wikimedia projects."
> 10:19 AM: The text is at
> http://infolab.stanford.edu/~west1/pubs/West_Dissertation-2016.pdf
> 10:19 AM: Page 19 indicates that this information includes the "IP
> address, proxy information, and user agent."
> 10:20 AM: This is confirmed by West at
> https://www.youtube.com/watch?v=jQ0NPhT-fsE&t=25m40s
> 10:20 AM: Does the Foundation still share that identifying information
> with research affiliates? If so, how many are them world-wide; if not,
> when did sharing this information stop?
> 10:25 AM: MediaWiki @James Salsman I see your question, but donfly.
> Can we reach out to you after the talk?
> 10:25 AM: MediaWiki: sorry hit enter too soon!
> 10:26 AM: MediaWiki: I don't have the full-context of the thesis to
> ask kate to answer the question on the fly. Can we reach out to you
> after?
> 10:26 AM: James Salsman: With whom am I corresponding?
> 10:27 AM: James Salsman: Sarah?
> 10:28 AM: MediaWiki: Yes! That's me!
> 10:28 AM: James Salsman: Would it be easier to ask, "how many research
> affiliates does the Foundation share server logs with IP addresses?"
> 10:29 AM: MediaWiki: Yes, I can ask that.
> 10:30 AM: James Salsman: Thank you.
>
> At 10:34, the messages with the the URLs I posted were removed from
> the chat log.
>
> At 10:36, the chat stream was removed from the video, which was
> replaced with the text, "Chat is disabled for this live stream," and
> now is completely missing.
>
> I am still interested in getting an answer to this question, but
> disturbed by the removal of links to sources. Could I please have an
> explanation?
>
> Sincerely,
> Jim

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Tech Talk livestream chat transcript (was Re: [Wikimedia Technical Talks] Data and Decision Science at Wikimedia with Kate Zimmerman, 26 February 2020 @ 6PM UTC)

[James Salsman]

Just now I asked the following question on the Technical Talk
livestream at https://www.youtube.com/watch?v=J-CRsiwYM9w

10:19 AM: Page 20 of Robert West's 2016 Stanford thesis, "Human
Navigation of Information Networks" says, "We have access to
Wikimedia’s full server logs, containing all HTTP requests to
Wikimedia projects."
10:19 AM: The text is at
http://infolab.stanford.edu/~west1/pubs/West_Dissertation-2016.pdf
10:19 AM: Page 19 indicates that this information includes the "IP
address, proxy information, and user agent."
10:20 AM: This is confirmed by West at
https://www.youtube.com/watch?v=jQ0NPhT-fsE&t=25m40s
10:20 AM: Does the Foundation still share that identifying information
with research affiliates? If so, how many are them world-wide; if not,
when did sharing this information stop?
10:25 AM: MediaWiki @James Salsman I see your question, but donfly.
Can we reach out to you after the talk?
10:25 AM: MediaWiki: sorry hit enter too soon!
10:26 AM: MediaWiki: I don't have the full-context of the thesis to
ask kate to answer the question on the fly. Can we reach out to you
after?
10:26 AM: James Salsman: With whom am I corresponding?
10:27 AM: James Salsman: Sarah?
10:28 AM: MediaWiki: Yes! That's me!
10:28 AM: James Salsman: Would it be easier to ask, "how many research
affiliates does the Foundation share server logs with IP addresses?"
10:29 AM: MediaWiki: Yes, I can ask that.
10:30 AM: James Salsman: Thank you.

At 10:34, the messages with the the URLs I posted were removed from
the chat log.

At 10:36, the chat stream was removed from the video, which was
replaced with the text, "Chat is disabled for this live stream," and
now is completely missing.

I am still interested in getting an answer to this question, but
disturbed by the removal of links to sources. Could I please have an
explanation?

Sincerely,
Jim

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Availability of hourly pagecounts files

[James Salsman]

That's fascinating, John; thank you. I'm copying this to wiki-research-l and
Fabian Suchanek, who gave the first part of the Research Showcase last month.

What do you like for coding stories? https://quanteda.io/reference/dfm.html ?
Sentiment is hard because errors are often 180 degrees away from correct.

How do you both feel about Soru et al (June 2018) "Neural Machine Translation
for Query Construction and Composition"
https://www.researchgate.net/publication/326030040 ?


On Sat, Jan 11, 2020 at 3:46 PM John Urbanik  wrote:
>
> Jim,
>
> I used to work as the chief data scientist at Collin's company.
>
> I'd suggest looking at things like relationships between the views / edits 
> for sets of pages as well as aggregating large sets of page views for 
> different pages in various ways. There isn't a lot of literature that is 
> directly applicable, and I can't disclose the precise methods being used due 
> to NDA.
>
> In general, much of the pageview data is weibull or GEV distributed on top of 
> being non-stationary, so I'd suggest looking into papers from extreme value 
> theory literature as well as literature around Hawkes/Queue-Hawkes processes. 
> Most traditional ML and signal processing is not very effective without doing 
> some pretty substantial pre-processing, and even then things are pretty 
> messy, depending on what you're trying to predict; most variables are 
> heteroskedastic w.r.t pageviews and there are a lot of real world events that 
> can cause false positives.
>
> Further, concept drift is pretty rapid in this space and structural breaks 
> happen quite frequently, so the reliability of a given predictor can change 
> extremely rapidly. Understanding how much training data to use for a given 
> prediction problem is itself a super interesting problem since there may be 
> some horizon after which the predictor loses power, but decreasing the 
> horizon too much means over fitting and loss of statistical significance.
>
> Good luck!
>
> John

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Wiki-research-l Digest, Vol 173, Issue 5

[James Salsman]

>
> https://meta.wikimedia.org/wiki/Special:ActiveUsers
>

Says about 2,570 active users on meta in the past 30 days, compared to
about 70,000 in unique editors on all projects? 3.7%?
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] open source alternatives to dbpedia?

[James Salsman]

Given some of the recent under-performance noticed by
toxicity-sniffing tools, I thought I would ask what people here think
of http://chat.dbpedia.org

Have a look: https://i.imgur.com/jKqRRTw.png

Is anyone else working on an open source text chatbot based on Wikidata?

I can offer no promises about how much it will teach you about
Wikidata or Natural Language Processing, but a good starter task would
be e.g.
 https://github.com/dbpedia/GSoC/issues/11

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] [Analytics] [Wikimedia Research Showcase] June 26, 2019 at 11:30 AM PST, 19:30 UTC

[James Salsman]

> For those that couldn't make it, Is there are summary of what was said?

Full recording: https://www.youtube.com/watch?v=WiUfpmeJG7E

Slides:

https://www.mediawiki.org/wiki/File:Trajectories_of_Blocked_Community_Members_-_Slides.pdf

https://meta.wikimedia.org/wiki/University_of_Virginia/Automatic_Detection_of_Online_Abuse

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] please hire a CTO who wants to protect reader privacy

[James Salsman]

I noticed just now that the Foundation is soliciting applications for a new CTO:
https://www.linkedin.com/feed/update/urn:li:activity:6515003866130505729

Can we please hire a CTO who would prefer to protect reader privacy
above the interests of any State or non-state actors, whether they
have infiltrated staff, contractor, and NDA signatory ranks, and
whether it interferes with reader statistics and analytics or not,
please?

In particular, I would like to repeat my request that we should not be logging
personally identifiable information which might increase our subpoena
burden or result in privacy violation incidents. Fuzzing geolocation
is okay, but we should not be transmitting IP addresses into logs
across even a LAN, for example, and we certainly shouldn't be
purchasing hardware with backdoor coprocessors wasting electricity and
exposing us to government or similar intrusions:
https://lists.wikimedia.org/pipermail/analytics/2017-January/005696.html

Best regards,
Jim

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] research on productivity increases through anonymization

[James Salsman]

"Anonymous calling: The WikiScanner scandals and anonymity on the
Japanese Wikipedia" from yesterday's call for reviews was very good; I
like this quote:

> “If there is a user ID attached to a user, discussion tends to become a 
> criticizing game. On the other hand, under an anonymous system, even though 
> your opinion/information is criticized, you don’t know with whom to be upset. 
> Also, with a user ID, those who participate in the site for a long time end 
> to have authority, and it becomes difficult for a user to disagree with them. 
> Under a perfectly anonymous system, you can say ‘it’s boring’, if it is 
> actually boring. All information is treated equally; only an accurate 
> argument will work.” [Andrew Lih. The Wikipedia revolution: How a bunch of 
> nobodies created the world’s greatest encyclopedia. New York: Hyperion, 2009, 
> p. 145.]

Most of the research on the ways anonymity helps productivity went on
during the design of double-blind experiments, and I hope more of that
is done in the context of Wikipedia.

I am too busy this month to do any reviews, and will be disappointed
if someone else doesn't:
https://firstmonday.org/ojs/index.php/fm/article/view/9184/7694

Thanks!

Best regards,
Jim

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Towards an inclusive digital literacy framework for digital India

[James Salsman]

Of considerable interest:
 
https://www.researchgate.net/publication/326141291_Towards_an_inclusive_digital_literacy_framework_for_digital_India

"Policy makers can use this transformational model to extend the reach
and effectiveness of Digital Inclusion through the last mile enhancing
existing training and service centers that offer the traditional model
of Digital Literacy Education This education model can be
replicated and scaled by the Digital India program by extending the
reach of existing rural Common Service Centers centers to reach remote
areas that lack infrastructure, thereby reaching the last mile for a
transformative impact across the nation."

I was particularly impressed with Figure 4: https://i.imgur.com/s30xY4C.png

Best regards,
Jim

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Wiki-research-l] Country (culture...) as a factor in contributing to collective intelligence projects

[James Salsman]

> Why do you think different language Wikipedia's have different
> sizes, outside of the popularity of a given language?

Piotr, if you model organic editing production with a Poisson
distribution, which is reasonable for a first approximation, 3x+
disparities are just natural for the same population sizes:

https://en.wikipedia.org/wiki/Poisson_distribution

I'm not sure the images in that article capture the wide platykurtosis
of large Poisson distributions.

Best regards,
Jim

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] a video from the Foundation's Research Showcase

[James Salsman]

https://www.youtube.com/watch?v=Q1sSzKKoHB8

Best regards,
Jim

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Fwd: [Wikimedia Education] Evaluation report on Wikipedia Fellows pilot

[James Salsman]

LiAnna, is there any way the filter rules could be adjusted to allow
my followup questions, please?


-- Forwarded message --
From:  
Date: Tue, May 22, 2018 at 5:50 PM
Subject: Re: [Wikimedia Education] Evaluation report on Wikipedia Fellows pilot
To: jsals...@gmail.com


Message rejected by filter rule match



-- Forwarded message --
From: James Salsman 
To: Wikimedia Education 
Cc:
Bcc:
Date: Tue, 22 May 2018 17:50:48 -0600
Subject: Re: [Wikimedia Education] Evaluation report on Wikipedia Fellows pilot
LiAnna, thank you so much for sending the list of twelve partner
associations at https://wikiedu.org/partnerships/

I am copying the list here because I have more questions:

American Anthropological Association
American Chemical Society
American Society of Plant Biologists
American Sociological Association
American Studies Association
Association for Psychological Science
Association for Women in Mathematics
Linguistic Society of America
Louisiana State University, Communication across the Curriculum
Midwest Political Science Association
National Communication Association
National Women’s Studies Association
Society for Marine Mammalogy

I'd like to recommend brainstorming what a list of, say 50 such
associations would look like if you picked them specifically to
balance them against e.g.
http://blogs.lse.ac.uk/impactofsocialsciences/files/2015/09/figure-1-wikipedia-open-access1.jpg

Since I am certain you will see the merit of such work, I want to take
this time to say a few words about economics. In particular, with
reference to:

https://lists.wikimedia.org/pipermail/wiki-research-l/2018-April/006256.html

and

https://lists.wikimedia.org/pipermail/wikimedia-l/2018-May/090241.html

These are very serious questions impacting the lives, livelihoods, and
years of productive life of billions upon billions of people, and will
probably remain as weighty for years.

In the mean time, I want to take this opportunity to reiterate my
recommendation to establish an essay contest for students on topics
such as economics, or any of Wikipedia's under-represented sciences or
humanities. The last time we discussed this question I did not
understand the reasons that you suggested such an essay contest would
be inappropriate. Has your view evolved?

Best regards,
Jim


On Tue, May 22, 2018 at 3:57 PM, LiAnna Davis  wrote:
> Answers inline!
>
> On Tue, May 22, 2018 at 12:15 PM, Juliana Bastos Marques <
> domusau...@gmail.com> wrote:
>>
>> I’d like to add another question. As you and others may know, I work in a
>> particularly quarrelsome Wikipedia (PT), where there are lots of reversions
>> of edits from newbies, even when they display knowledge of WP rules. What
>> was the reception of the Fellows’ work among the community of editors?
>>
>> There were some minor disagreements with other English Wikipedia editors,
> but conversations were ultimately productive. We did have one article
> nominated for deletion, but the Fellow was able to successfully argue for
> it to not be deleted.
>
>
>
>> Em 22 de mai de 2018, à(s) 16:02, James Salsman 
>> escreveu:
>>
>> > Would you please describe how you choose the subject matter of
>> > articles and expertise for inviting Fellows?
>>
>
> Fellows chose their own articles to improve based on their interests and
> expertise. We selected the associations to participate in the pilot based
> on our relationships with them; we're expanding future Fellows cohorts to
> other subject areas.
>
>
>> > It's not clear whether the Fellows were paid or otherwise compensated;
>> > were they?
>>
>
> There's a reference to this in the "Recruiting Wikipedia Fellows" section
> (I know there's a lot in here, so I'm not surprised if you missed it!): "We
> encouraged partners to consider offering Fellows honoraria, travel
> scholarships to their conference, or conference fee waivers. Partners were
> amenable to the idea but most said they needed more time to be able to
> offer it. We hope this might be able to be built into future Fellows
> cohorts."
>
>> "In the past four years, the Wiki Education Foundation (Wiki
>> > Education) has signed formal partnership agreements with academic
>> > associations to improve Wikipedia in their topic area." -- how many?
>> > Is the list public?
>>
>
> We've signed agreements with 12 academic associations; they're listed on
> our website: https://wikiedu.org/partnerships/
>
>
>> > When you select such subjects and topics, do you consider the number
>> > of pageviews? Do you use existing WP:BACKLOG category membership?
>> > Both?
>>
>
> We encouraged Fellows to choose artic

[Wiki-research-l] weekly periodicity mode

[James Salsman]

[Crossposting to Research and Analytics lists]

Most Wikipedia articles with a weekly periodicity show more pageviews
on a typical weekday than a weekend. Some articles associated with
weekends (e.g. articles associated with a variety of hobbies) will
show relatively fewer pave views on weekdays.

Suppose I wanted to plot a heatmap with colors corresponding to the
strength of the weekly periodicity of the pageviews of articles shown
in different geographic locations.

(1) Has anyone done anything like this before?

(2) Is sufficient information available from the current logging regime?

Finally, I would also like to ask for review of this summarization, please:
 
https://www.mediawiki.org/w/index.php?title=Wikimedia_Technology%2FAnnual_Plans%2FFY2019%2FCDP3%3A_Knowledge_Integrity&type=revision&diff=2762601&oldid=2762351

Best regards,
Jim

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Fwd: nice Wikimedia Research presentation

[James Salsman]

[sharing because the URL was obscured in the Google Hangout on Air yesterday.]


-- Forwarded message --
From: Nicholas Vincent 
Date: Fri, Apr 20, 2018 at 8:00 AM
Subject: Re: nice Wikimedia Research presentation
To: James Salsman 


Hey Jim,

Really glad you enjoyed the presentation!

Here's the link to my personal website:
http://www.nickmvincent.com

and to our lab website (which I should probably add to the slides!)
http://www.psagroup.org

Best,
Nick

On Fri, Apr 20, 2018 at 2:15 AM James Salsman  wrote:
>
> Nick,
>
> I really enjoyed your presentation.
>
> What is your web address, obscured in the attached?
>
> Thank you!
>
> Best regards,
> Jim

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] bias relative to accuracy or populist centrism? (was re: The complete program of Wiki Workshop 2018 is live)

[James Salsman]

Christoph,

Thank you for your reply. I am happy to help and glad you have made
good decisions.

Are you familiar with Priyanka's work on Accuracy Review?
 https://priyankamandikal.github.io/posts/gsoc-2016-project-overview/

Fabian Floeck and

On Wed, Apr 18, 2018 at 7:07 AM, Christoph Hube  wrote:
> Hi James,
>
> thanks a lot for your interest in our work!
>
> The problem of crowdworkers being biased is a problem definitely not to be
> neglected. Majority vote can help to sort out single extremist views of
> workers but if many workers are strongly biased then I agree that this might
> not be enough. We are actually already thinking about methods to improve
> future crowdsourced bias datasets. One way to improve the quality is to have
> a very well defined task that leaves only little room for subjective
> interpretation. For example, instead of letting the workers decide whether a
> statement is biased or not, we asked more specifically whether the statement
> reflects an opinion or contains bias words. Of course, the decision if a
> statement reflects a fact or an opinion is still subjective in many cases.
>
> Given your example it is hard to make a decision (even when being unbiased)
> without having the proper background knowledge. That is why our work mostly
> focuses on language bias, i.e. bias that is introduced through the use of
> judgemental language. Since there are many cases of bias without using
> judgemental language, we are definitely interested to come up with good
> approaches that cover these cases as well. Ideas and suggestions are always
> welcome!
>
> One other thing that we are planning to do for future crowdsourcing jobs is
> to ask workers for their political opinions and to take this background
> information into account when creating ground truth data.
>
> Best regards,
> Christoph
>
>
>
> Am 4/18/2018 um 2:22 PM schrieb James Salsman:
>>>
>>> ... Accepted papers
>>> Christoph Hube and Besnik Fetahu
>>> Detecting Biased Statements in Wikipedia
>>> http://wikiworkshop.org/2018/papers/wikiworkshop2018_paper_1.pdf
>>> ...
>>
>> Hi Christoph and Besnik,
>>
>> Having worked with several thousand of Amazon Mechanical Turkers over
>> the past year, I am not convinced that their opinions of bias, even in
>> aggregate, are not biased.  Did you take any steps to measure the bias
>> against accuracy in your crowdworkers?
>>
>> Here is an example of what I expect they would get wrong:
>>
>> "Tax cuts allow consumers to increase their spending, which boosts
>> aggregate demand."
>>
>> That statement, added by en:User:Bkwillwm in 2012,[1] is still part of
>> the English Wikipedia's Economics article. However, the statement is
>> strictly inaccurate, and heavily biased in favor of trickle-down
>> economics and austerity policy.[2] It and statements like it,
>> pervasive through many if not most of the popular language Wikipedias,
>> directly support increases in income inequality, which in turn is a
>> terrible scourge affecting both health[3] and economic growth.[4]
>>
>> How can you measure whether your crowdworkers are truly unbiased
>> relative to accuracy, instead of just reflecting the
>> propaganda-influenced whims of the populist center?
>>
>> Sincerely,
>> James Salsman
>>
>> [1]
>> https://en.wikipedia.org/w/index.php?title=Economics&diff=prev&oldid=511580566
>>
>> [2]
>> https://en.wikipedia.org/wiki/Talk:Economics/Archive_7#Tax_cut_claim_in_Fiscal_policy_section
>>
>> [3] http://talknicer.com/ehip.pdf
>>
>> [4] http://talknicer.com/egma.pdf
>
>

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] bias relative to accuracy or populist centrism? (was re: The complete program of Wiki Workshop 2018 is live)

[James Salsman]

>... Accepted papers
> Christoph Hube and Besnik Fetahu
> Detecting Biased Statements in Wikipedia
> http://wikiworkshop.org/2018/papers/wikiworkshop2018_paper_1.pdf
>...

Hi Christoph and Besnik,

Having worked with several thousand of Amazon Mechanical Turkers over
the past year, I am not convinced that their opinions of bias, even in
aggregate, are not biased.  Did you take any steps to measure the bias
against accuracy in your crowdworkers?

Here is an example of what I expect they would get wrong:

"Tax cuts allow consumers to increase their spending, which boosts
aggregate demand."

That statement, added by en:User:Bkwillwm in 2012,[1] is still part of
the English Wikipedia's Economics article. However, the statement is
strictly inaccurate, and heavily biased in favor of trickle-down
economics and austerity policy.[2] It and statements like it,
pervasive through many if not most of the popular language Wikipedias,
directly support increases in income inequality, which in turn is a
terrible scourge affecting both health[3] and economic growth.[4]

How can you measure whether your crowdworkers are truly unbiased
relative to accuracy, instead of just reflecting the
propaganda-influenced whims of the populist center?

Sincerely,
James Salsman

[1] 
https://en.wikipedia.org/w/index.php?title=Economics&diff=prev&oldid=511580566

[2] 
https://en.wikipedia.org/wiki/Talk:Economics/Archive_7#Tax_cut_claim_in_Fiscal_policy_section

[3] http://talknicer.com/ehip.pdf

[4] http://talknicer.com/egma.pdf

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Growing Wikipedia Across Languages via Recommendation

[James Salsman]

Leila, since https://arxiv.org/pdf/1604.03235.pdf worked so well, is
it going to be implemented widely, or is it shelved? Does it need
advocates?

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] research on topic coverage?

[James Salsman]

Amy Bruckman wrote:

> I was just re-reading Halavais & Lackaff’s 2008 paper on topic coverage in 
> the English Wikipedia.
> Has anyone redone or extended that analysis more recently?

I've been keeping track of the length articles on
https://en.wikipedia.org/wiki/Wikipedia:Short_popular_vital_articles
every six months for the past six years. It's great news in that
improvements as measured by byte count and controlled for maintenance
templates has been growing at a constant rate, basically four bytes
per day. I've never published anything on it and don't plan to, hoping
that someone who can use the academic publication credit will some
day. Plotting ORES scores over time is easy now, and should make it
sufficiently interesting to journal editors.

My favorite article on the topic is
https://onlinelibrary.wiley.com/doi/full/10.1002/asi.23687
It has a lot of citing articles:
https://scholar.google.com/scholar?hl=en&um=1&ie=UTF-8&lr&cites=13904159020044435588

> Also, has anyone mapped comparative topic coverage for different languages?

Yes, e.g. 
https://fenix.tecnico.ulisboa.pt/downloadFile/395144380424/popculture-paper.pdf

Best regards,
Jim

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] please review this open peer review medical article; article request

[James Salsman]

Please consider reviewing this article: http://preprints.jmir.org/preprint/9683

If you have never peer reviewed a medical literature review, this is
your chance. If you know M.D.s with some spare time, please ask them
to review it, too.

Please see also: https://imgur.com/gallery/OtLil

Finally, if someone has the full text to
https://link.springer.com/chapter/10.1007/978-3-319-59084-4_7 would
you please email it to me, I'd like to read it for educational and
critique purposes. Thanks in advance.

Best regards,
Jim

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] San Francisco union rules

[James Salsman]

> you can't have volunteer resources run the video
> equipment for you. You need a professional crew.

Is there a more specific description of the specific authority,
wording, and legislative intent for this, please?

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] New policy about performing research on English WikipediaWiki-research-l Digest, Vol 149, Issue 1

[James Salsman]

Hi Jonathan,

Can you please give a concrete example of what, for example, the
http://ide.mit.edu/sites/default/files/publications/SSRN-id3039505.pdf
researchers would have had to do differently under this new policy?

Best regards,
Jim

> Date: Tue, 2 Jan 2018 15:29:03 -0800
> From: Jonathan Morgan 
> To: Wiki Research-l 
> Subject: [Wiki-research-l] New policy about performing research on English 
> Wikipedia
>
> Hi there wiki-research folks,
>
> This is just a heads-up that English Wikipedia has adopted a new policy[1]
> about research on that project. The policy codifies some new requirements
> for community notification and disclosure that potentially apply to all
> research projects (regardless of the affiliation of the researcher).
>
> You can read more about the policy on WP:NOT[1], but I've included the
> major points below for your convenience:
>
>- any research project that involves directly changing article content,
>surveying a large number of editors, or asking editors sensitive questions
>about their real-life identities needs to be discussed on Wikipedia's
>Village Pump[2] before it is begun[3]
>- researchers should disclose who they are on their user pages,
>including their institutional affiliation, sources of research funding (if
>applicable), and the intentions behind their research[4]
>
> Many aspects of this policy boil down to either common sense, existing
> ethical standards for human subjects research, or both. However, this
> policy also leaves certain definitions and thresholds undefined. What is a
> "large number" of surveyed users? What is a "sensitive question"?
> There are no concrete answer to these questions yet, and that's probably a
> good thing. The best way to keep this policy from becoming overly
> restrictive[5] is for researchers to follow its guidance in good faith, and
> ask questions when they're uncertain.
>
> Projects that are deemed to be in violation of these guidelines may lose
> editing privileges. If the violations are deemed particularly frequent or
> severe, the EnWiki community may decide to make even more rules, which
> could have a chilling effect on wikiresearch in general. Nobody wants
> that.
>
> If you have general questions about this policy or its application, the
> best place to ask is the WP:NOT talkpage.[6]
>
> If you have questions related to a specific planned research project, the
> best thing to do is to err on the side of caution and open up a discussion
> on the Village Pump before you begin.
>
> You are also welcome to post your project plan to this list, where we, your
> friendly peers, will hopefully offer constructive feedback and links to
> relevant resources.
>
> Wikimedia Foundation research staff are not in charge of these guidelines,
> but are happy to offer advice "from the trenches" so to speak if asked. We
> are on this list too.
>
> As always, if you are currently researching Wikipedia, or plan to do so,
> please create a Research Project page on MetaWiki[7] (example[8], tips[9]),
> keep it up to date, and link to it from your userpage[10]. That way
> interested parties can follow your research and ask questions, and you
> won't need to constantly re-explain what you're doing every time someone
> asks.
>
> Happy researching,
>
> Jonathan
>
>
>
>1.
>
> https://en.wikipedia.org/wiki/Wikipedia:What_Wikipedia_is_not#Wikipedia_is_not_a_laboratory
>2. https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(proposals)
>3.
>https://en.wikipedia.org/wiki/Wikipedia:What_Wikipedia_is_not#cite_note-7
>4.
>https://en.wikipedia.org/wiki/Wikipedia:What_Wikipedia_is_not#cite_note-8
>5. https://meta.wikimedia.org/wiki/Instruction_creep
>6. https://en.wikipedia.org/wiki/Wikipedia_talk:What_Wikipedia_is_not
>7. https://meta.wikimedia.org/wiki/Research:Projects
>8.
>
> https://meta.wikimedia.org/wiki/Research:Supporting_Commons_contribution_by_GLAM_institutions
>9.
>
> https://meta.wikimedia.org/wiki/Research:Project_documentation_best_practices
>10. https://meta.wikimedia.org/wiki/User:LZia_(WMF)
>
>
>
>
> --
> Jonathan T. Morgan
> Senior Design Researcher
> Wikimedia Foundation
> User:Jmorgan (WMF) 

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] December 2017 Research Showcase

[James Salsman]

Leila and Lani,

The Article Expansion Recommendation System is an absolutely
spectacular project, which will clearly very substantially improve the
encyclopedia in ways that perhaps no other single effort has come near
to being able, so I can't wait to learn more about it. But I might not
be able to make the live-stream time, so I want to get in this
question in advance:

Are you using or do you plan to use ORES quality predictions, the
upcoming article importance predictions, and pageview statistics to
rank article expansion recommendations?

> The next Research Showcase will be live-streamed this Wednesday, December
> 13, 2017 at 11:15 AM (PST) 18:15 UTC.
>
> YouTube stream: https://www.youtube.com/watch?v=OoVwus1Owtk
>
> As usual, you can join the conversation on IRC at #wikimedia-research. And,
> you can watch our past research showcases here.
>
> This month's presentation:
> "The State of the Article Expansion Recommendation System"
>
> By Leila Zia
>
> Only 1% of English Wikipedia articles are labeled with quality class Good
> or better, and 37% of the articles are stubs. We are building an article
> expansion recommendation system to change this in Wikipedia, across many
> languages. In this presentation, I will talk with you about our current
> thinking of the vision and direction of the research that can help us build
> such a recommendation system, and share more about one specific area of
> research we have heavily focused on in the past months: building a
> recommendation system that can help editors identify what sections to add
> to an already existing article. I present some of the challenges we faced,
> the methods we devised or used to overcome them, and the result of the
> first line of experiments on the quality of such recommendations (teaser:
> the results are really promising. The precision and recall at 10 is 80%.)

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] feedback appreciated

[James Salsman]

>... what does this post have to do with wikis?

FRSbot is a very prominent bot on Wikipedia crucial to obtaining neutral
feedback for less-prominent RFCs, but it doesn't work the way people think
it does, or the way it's authors have implied it does, or the way it should
if it was going to be neutral.

Take a look at its code and see how it distributes requests. They aren't
automated, just automatically prepared for a completely obscured step
requiring manual intervention which, in my opinion, gives the person doing
that manual step a whole lot more power over the controversies in the
encyclopedia than any other role.

Who actually does that manual distribution step? Legotkm or James Hare?
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] feedback appreciated

[James Salsman]

Dr. Heather Ford wrote:

>... You may want to read Angele Christin's paper that just came
> out in Big Data and Society that complicates the notion of judges
> accepting algorithmic reasoning wholesale in making decisions.
>
> http://journals.sagepub.com/eprint/SPgDYyisV8mAJn4fm7Xi/full

I am in Australia right now, working today to save its would-be
immigrants from stupid robot AI pronunciation assessments:

https://www.theguardian.com/australia-news/2017/aug/08/computer-says-no-irish-vet-fails-oral-english-test-needed-to-stay-in-australia

http://www.smh.com.au/technology/technology-news/australian-exnews-reader-with-english-degree-fails-robots-english-test-20170809-gxsjv2.html

I clearly remember the day in 1996 when the guy who since
wrote the Pearson Spoken English test rejected my attempts
at accent adaptation.

The fight isn't against robots, it's against their lazy creators.

Best regards,
James

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] share research without paywalls or requests for personal information?

[James Salsman]

However, I am not comfortable seeing research papers being shared on
this list in manners which ask the readers to disclose their personal
information:

http://imgur.com/a/qtzRS

Can we please have some baseline standards for sharing research
without any paywalls or requests for personal information?

On Thu, Aug 24, 2017 at 9:04 PM, James Salsman  wrote:
> I have to recuse myself from the reviews until further notice because
> I am publishing on the topic:
> https://raw.githubusercontent.com/jsalsman/featex/master/Spoken-English-Intelligibility-Remediation.pdf
>
> On Thu, Aug 24, 2017 at 8:00 PM,
>  wrote:
>> Send Wiki-research-l mailing list submissions to
>> wiki-research-l@lists.wikimedia.org
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>> or, via email, send a message with subject or body 'help' to
>> wiki-research-l-requ...@lists.wikimedia.org
>>
>> You can reach the person managing the list at
>> wiki-research-l-ow...@lists.wikimedia.org
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of Wiki-research-l digest..."
>>
>>
>> Today's Topics:
>>
>>1. Upcoming research newsletter: new papers open for review
>>   (mass...@ymail.com)
>>
>>
>> --
>>
>> Message: 1
>> Date: Wed, 23 Aug 2017 13:39:41 +
>> From: 
>> To: Wikimedia Research Mailing List
>> 
>> Subject: [Wiki-research-l] Upcoming research newsletter: new papers
>> open forreview
>> Message-ID: <971883.20493...@smtp140.mail.ir2.yahoo.com>
>> Content-Type: text/plain; charset="utf-8"
>>
>> Hi,
>>
>> We’re preparing for the June 2017 research newsletter and looking for 
>> contributors. Please take a look at:
>> https://etherpad.wikimedia.org/p/WRN201706 and add your name next to any 
>> paper you are interested in covering. Our target publication date is Friday 
>> August 25 UTC although actual publication may take place several days later. 
>> As usual, short notes and one-paragraph reviews are most welcome.
>>
>> Highlights from this month:
>>
>> • ‘What are these researchers doing in my Wikipedia?’: ethical premises and 
>> practical judgment in internet-based ethnography
>> • 280 Birds with One Stone: Inducing Multilingual Taxonomies from Wikipedia 
>> using Character-level Classification
>> • Analysing Timelines of National Histories across Wikipedia Editions: A 
>> Comparative Computational Approach
>> • Assessing and Improving Domain Knowledge Representation in Dbpedia
>> • Chaudron: Extending DBpedia with measurement
>> • Cultural diversity of quality of information on Wikipedias
>> • Digging Wikipedia: The Online Encyclopedia As a Digital Cultural Heritage 
>> Gateway and Site
>> • Evaluation of Metadata Representations in RDF stores
>> • High-Throughput and Language-Agnostic Entity Disambiguation and Linking on 
>> User Generated Data
>> • Measuring Global Disease with Wikipedia: Success, Failure, and a Research 
>> Agenda
>> • Measuring Global Disease with Wikipedia: Success, Failure, and a Research 
>> Agenda
>> • Predicting Member Productivity and Withdrawal from Pre-Joining Attachments 
>> in Online Production Groups
>> • Problematizing and Addressing the Article-as-Concept Assumption in 
>> Wikipedia
>> • Scholia and scientometrics with Wikidata
>> • Shocking the Crowd: The Effect of Censorship Shocks on Chinese Wikipedia
>> • Spammer Users Identification in Wikipedia via Editing Behavior
>> • Spatio-Temporal Analysis of Reverted Wikipedia Edits
>> • The Evolution and Consequences of Peer Producing Wikipedia's Rules
>> • The Substantial Interdependence of Wikipedia and Google: A Case Study on 
>> the Relationship Between Peer Production Communities and Information 
>> Technologies
>> • The Unusual Suspects: Deep Learning Based Mining of Interesting Entity 
>> Trivia from Knowledge Graphs
>> • The Wikipedia Adventure: Field Evaluation of an Interactive Tutorial for 
>> New Users
>> • TokTrack: A Complete Token Provenance and Change Tracking Dataset for the 
>> English Wikipedia
>> • Visualizing Rank Time Series of Wikipedia Top-Viewed Pages
>> • Who Wants to Read This?: A Method for Measuring Topical Representativeness 
>> in User Generated Content Systems
>> • WikiSeq: Mining Maximally Informative Simple Sequ

Re: [Wiki-research-l] Wiki-research-l Digest, Vol 144, Issue 15

[James Salsman]

I have to recuse myself from the reviews until further notice because
I am publishing on the topic:
https://raw.githubusercontent.com/jsalsman/featex/master/Spoken-English-Intelligibility-Remediation.pdf

On Thu, Aug 24, 2017 at 8:00 PM,
 wrote:
> Send Wiki-research-l mailing list submissions to
> wiki-research-l@lists.wikimedia.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
> or, via email, send a message with subject or body 'help' to
> wiki-research-l-requ...@lists.wikimedia.org
>
> You can reach the person managing the list at
> wiki-research-l-ow...@lists.wikimedia.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Wiki-research-l digest..."
>
>
> Today's Topics:
>
>1. Upcoming research newsletter: new papers open for review
>   (mass...@ymail.com)
>
>
> --
>
> Message: 1
> Date: Wed, 23 Aug 2017 13:39:41 +
> From: 
> To: Wikimedia Research Mailing List
> 
> Subject: [Wiki-research-l] Upcoming research newsletter: new papers
> open forreview
> Message-ID: <971883.20493...@smtp140.mail.ir2.yahoo.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi,
>
> We’re preparing for the June 2017 research newsletter and looking for 
> contributors. Please take a look at:
> https://etherpad.wikimedia.org/p/WRN201706 and add your name next to any 
> paper you are interested in covering. Our target publication date is Friday 
> August 25 UTC although actual publication may take place several days later. 
> As usual, short notes and one-paragraph reviews are most welcome.
>
> Highlights from this month:
>
> • ‘What are these researchers doing in my Wikipedia?’: ethical premises and 
> practical judgment in internet-based ethnography
> • 280 Birds with One Stone: Inducing Multilingual Taxonomies from Wikipedia 
> using Character-level Classification
> • Analysing Timelines of National Histories across Wikipedia Editions: A 
> Comparative Computational Approach
> • Assessing and Improving Domain Knowledge Representation in Dbpedia
> • Chaudron: Extending DBpedia with measurement
> • Cultural diversity of quality of information on Wikipedias
> • Digging Wikipedia: The Online Encyclopedia As a Digital Cultural Heritage 
> Gateway and Site
> • Evaluation of Metadata Representations in RDF stores
> • High-Throughput and Language-Agnostic Entity Disambiguation and Linking on 
> User Generated Data
> • Measuring Global Disease with Wikipedia: Success, Failure, and a Research 
> Agenda
> • Measuring Global Disease with Wikipedia: Success, Failure, and a Research 
> Agenda
> • Predicting Member Productivity and Withdrawal from Pre-Joining Attachments 
> in Online Production Groups
> • Problematizing and Addressing the Article-as-Concept Assumption in Wikipedia
> • Scholia and scientometrics with Wikidata
> • Shocking the Crowd: The Effect of Censorship Shocks on Chinese Wikipedia
> • Spammer Users Identification in Wikipedia via Editing Behavior
> • Spatio-Temporal Analysis of Reverted Wikipedia Edits
> • The Evolution and Consequences of Peer Producing Wikipedia's Rules
> • The Substantial Interdependence of Wikipedia and Google: A Case Study on 
> the Relationship Between Peer Production Communities and Information 
> Technologies
> • The Unusual Suspects: Deep Learning Based Mining of Interesting Entity 
> Trivia from Knowledge Graphs
> • The Wikipedia Adventure: Field Evaluation of an Interactive Tutorial for 
> New Users
> • TokTrack: A Complete Token Provenance and Change Tracking Dataset for the 
> English Wikipedia
> • Visualizing Rank Time Series of Wikipedia Top-Viewed Pages
> • Who Wants to Read This?: A Method for Measuring Topical Representativeness 
> in User Generated Content Systems
> • WikiSeq: Mining Maximally Informative Simple Sequences from Wikipedia
> • Wikum: Bridging Discussion Forums and Wikis Using Recursive Summarization
>
> If you have any question about the format or process feel free to get in 
> touch off-list.
>
> Masssly, Tilman Bayer and Dario Taraborelli
>
> [1] http://meta.wikimedia.org/ wiki/Research:Newsletter
>
>
> --
>
> Subject: Digest Footer
>
> ___
> Wiki-research-l mailing list
> Wiki-research-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
>
> --
>
> End of Wiki-research-l Digest, Vol 144, Issue 15
> 

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Fwd: research trying to influence real-world outcomes by editing Wikipedia

[James Salsman]

Is there any other research studying whether editing wikipedia(s)
produces real-world changes?


-- Forwarded message --
From: James Salsman 
Date: Mon, Jul 17, 2017 at 6:10 PM
Subject: Re: research trying to influence real-world outcomes by
editing Wikipedia
To: "wiki-research-l@lists.wikimedia.org" 


A followup by the same authors reviewed in today's Signpost reverses
their opinion on causality, asserting that I improvements to articles
about places increases tourism:

http://marit.hinnosaar.net/wikipediamatters.pdf



On Mon, May 29, 2017 at 3:52 PM, James Salsman  wrote:
> This was in the recent Research Newsletter:
>
> https://www.econstor.eu/bitstream/10419/127472/1/847290360.pdf
>
> They found a correlation between the length of articles about tourist
> destinations and the number of tourists visiting them. They tried to
> influence other destinations by adding content and did not find a
> correlation in the subsequent number of tourists, suggesting that the
> causation flows from tourism to article length instead.
>
> But I was taken aback by the last line of their paper, "using the
> suggested research design to study other areas of information
> acquisition, such as medicine or school choices could be fruitful
> directions."
>
> Are there any ethical guidelines concerning whether this is
> reasonable? Should there be?

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] research trying to influence real-world outcomes by editing Wikipedia

[James Salsman]

A followup by the same authors reviewed in today's Signpost reverses
their opinion on causality, asserting that I improvements to articles
about places increases tourism:

http://marit.hinnosaar.net/wikipediamatters.pdf



On Mon, May 29, 2017 at 3:52 PM, James Salsman  wrote:
> This was in the recent Research Newsletter:
>
> https://www.econstor.eu/bitstream/10419/127472/1/847290360.pdf
>
> They found a correlation between the length of articles about tourist
> destinations and the number of tourists visiting them. They tried to
> influence other destinations by adding content and did not find a
> correlation in the subsequent number of tourists, suggesting that the
> causation flows from tourism to article length instead.
>
> But I was taken aback by the last line of their paper, "using the
> suggested research design to study other areas of information
> acquisition, such as medicine or school choices could be fruitful
> directions."
>
> Are there any ethical guidelines concerning whether this is
> reasonable? Should there be?

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] link rot

[James Salsman]

Is anyone studying the rate at which external links become unavailable
on Wikipedia projects?

I just did a quick tally and less than 40% of the external links cited
in the introductions of L1-vital enwiki health and social science
articles I sampled were good, and that's only counting those which
didn't already have a {{dead link}} tag.

I thought that the bots were doing a better job of replacing dead
links with archive copies than they apparently are. Do we need to fund
this as an official effort?

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] research trying to influence real-world outcomes by editing Wikipedia

[James Salsman]

This was in the recent Research Newsletter:

https://www.econstor.eu/bitstream/10419/127472/1/847290360.pdf

They found a correlation between the length of articles about tourist
destinations and the number of tourists visiting them. They tried to
influence other destinations by adding content and did not find a
correlation in the subsequent number of tourists, suggesting that the
causation flows from tourism to article length instead.

But I was taken aback by the last line of their paper, "using the
suggested research design to study other areas of information
acquisition, such as medicine or school choices could be fruitful
directions."

Are there any ethical guidelines concerning whether this is
reasonable? Should there be?

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] [Wikimedia Education] Essay about fake news, algorithms, social bots, and more

[James Salsman]

I've studied this question using the same framework I use to track the
WP:SPVA changes. I'm convinced that the English Wikipedia can, given
enough time, handle every kind of controversy except:

(1) religious disputes (e.g., "Historicity of Jesus... Not to be
confused with Historical Jesus."

(2) international political disputes (any number of disputed borders
and islands, Israel/Palestine etc.),

(3) economic disputes pertaining to http://talknicer.com/ehip.pdf and
http://talknicer/egma.pdf

The issues regarding (1) don't have a material (world) impact; (2) are
intractable outside of Wikipedia, so why even bother; but (3) has
profound real-world political and economic impacts which affect the
Foundation's Mission by altering the extent to which free educational
content can be created and effectively disseminated. However, if
assertions that the issues pertaining to (3) are a result of systemic
bias are met with ridicule.

So what we have is Wikipedia perpetuating the "fake news" promoted by
trickle down economics that tax cuts for the rich are good. How might
that affect electoral outcomes, for example?

The best example at present is at
https://en.wikipedia.org/wiki/Talk:Economics#Tax_cut_claim_in_Fiscal_policy_section
which has stood for months with no interest expressed by any
Wikipedians in addressing the problem.


On Sun, May 14, 2017 at 8:22 AM, Pine W  wrote:
> Agreed that what we're seeing are Internet-enabled implementations of old
> practices. I think that there has been a recent renewal of awareness of how
> effective these dark arts can be at generating revenue and perhaps
> affecting political systems.
>
> Over the years, a number of people and organizations have tried to
> manipulate the neutrality of Wikipedia content for political, financial, or
> PR advantage. I have the impression that the community's human resources
> capacity and technical tools are currently insufficient in comparison to
> the scale of the problems. I'm hoping that some of the tools that are being
> developed as a part of the anti-harassment initiative will help a little.
> I'm also thinking that a good exercise for students in Wikipedia in
> Education classes would be to identify content that is noncompliant with
> neutrality and verifiability standards, and either change that content
> themselves or flag it for review by more experienced editors.
>
> Pine
>
>
> On Sat, May 13, 2017 at 5:53 AM, James Salsman  wrote:
>
>> Pine wrote:
>> >
>> > I'm finding it encouraging to see that a number of researchers and
>> > journalists are taking these problems seriously, trying to understand
>> them,
>> > and trying to improve the situation.
>> > http://www.pbs.org/wgbh/nova/next/tech/misinformation-on-
>> social-media-could-outfox-technical-solutions-for-now
>>
>> I'm encouraged by the studies, but confused about why the fake news
>> phenomenon is considered novel, rather than continuations of age-old
>> disinformation, yellow journalism, aggressive public relations,
>> manufactured consent, astroturfing, propaganda, and deceptive
>> marketing. There's nothing new about it other than the term.
>>
>> ___
>> Wiki-research-l mailing list
>> Wiki-research-l@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>>
> ___
> Education mailing list
> educat...@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/education

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Essay about fake news, algorithms, social bots, and more

[James Salsman]

Pine wrote:
>
> I'm finding it encouraging to see that a number of researchers and
> journalists are taking these problems seriously, trying to understand them,
> and trying to improve the situation.
> http://www.pbs.org/wgbh/nova/next/tech/misinformation-on-social-media-could-outfox-technical-solutions-for-now

I'm encouraged by the studies, but confused about why the fake news
phenomenon is considered novel, rather than continuations of age-old
disinformation, yellow journalism, aggressive public relations,
manufactured consent, astroturfing, propaganda, and deceptive
marketing. There's nothing new about it other than the term.

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] please critique my grant application

[James Salsman]

Please critique and endorse my grant application, especially after Doc
James replaces his name as the applicant so I can be the adviser and
my Google Summer of Code co-mentor and student can be co-grantees:
https://meta.wikimedia.org/wiki/Grants:Project/Intelligibility_Transcriptions

Thank you!

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] crowdsource question

[James Salsman]

Thank you for your questions, Jan.

> Is this on questions on Wikipedia Articles which ask for an
> estimate of good, neutral or bad assertions (or generally
> sentiments) about a subject?

After the Signpost ran a blurb last month on research successfully
predicting company stock price changes using pageviews (confirming
similar work from 2013), I tried to find anyone using the textual
substance of edits to do the same thing. I found this:

http://community.wolfram.com/groups/-/m/t/882612

It produces small but consistently positive correlations between
companies' article edit summaries classified by the text sentiment
model which ships with Wolfram Mathematica and their daily stock price
changes. The significance is low, in part because using sentiment of
edit summaries is a very naive approach. So I wonder if anyone has
tried to train a sentiment analysis model to address the task directly
with full diffs.

> Or are you more interested in the subject of lobbyism and
> company directed edits and the like?

I'm more interested in identifying organized advocacy, and I suspect
such models would help with that, too, especially if brand product
articles are included along with companies.

2016-12-01 4:12 GMT+01:00 James Salsman :
>
> Who, if anyone, is examining crowdsource survey
> questions such as, "Look at the text added or
> removed in this edit to [Company]'s Wikipedia
> article. Was the editor saying [ ] good things, [ ]
> bad things, or [ ] was neutral about [Company]'s
> financial prospects?"?

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] crowdsource question

[James Salsman]

Who, if anyone, is examining crowdsource survey
questions such as, "Look at the text added or
removed in this edit to [Company]'s Wikipedia
article. Was the editor saying [ ] good things, [ ]
bad things, or [ ] was neutral about [Company]'s
financial prospects?"?

Best regards,
Jim

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] [Publicpolicy] supporting Wikipedia citations as formal scholarly reputation in tenure committees

[James Salsman]

Hi Dario,

Since the document specifies "Reproduction is authorised provided the
source is acknowledged," I put it here:

https://drive.google.com/file/d/0B73LgocyHQnfam51TnN3dlVqaVE/view

It has only two explicit mentions of Wikipedia, in the discussion of
ImpactStory on pp. 58-9, but this document is the only official
government (European Commission) discussion of altmetrics for formal
academic reputation assessment I have been able to find anywhere. You
will probably find the discussion in the Forward and Introduction more
pertinent than the in-passing mentions of Wikipedia, and I suggest
reaching out to the authors in person for their recommended official
contacts at the Institute for Prospective Technological Studies and
their government supporters could be even more productive. There seems
to be a real opening to give academia and society a great gift implied
by the other three references, if they can accept it. Thank you so
much for your interest!

Best regards,
Jim

On Thu, Nov 17, 2016 at 5:10 PM, Dario Taraborelli
 wrote:
> James – I'm interested in reading [1] but the PDF is behind a login screen,
> can I read this somewhere else (or do you have the full reference so I can
> search it)?
>
> Thanks,
> Dario
>
> On Mon, Oct 31, 2016 at 9:43 PM, James Salsman  wrote:
>>
>> Can anyone familiar with European Commission procedure please explain
>> how to support the Wikipedia-associated proposals in [1] based on the
>> statistics in [2] please? Very recent publications such as [3] in
>> Nature along with what appears to be a relatively sudden groundswell
>> of frankness and support e.g. [4] suggests to me that the time is
>> right to get out in front of these proposals.
>>
>> [1]
>> https://www.researchgate.net/profile/David_Nicholas5/publication/275349828_Emerging_reputation_mechanisms_for_scholars/links/553a22a60cf2c415bb06e6b7.pdf
>>
>> [2] http://www.cs.indiana.edu/~xshuai/papers/jcdl240-shuai.pdf
>>
>> [3] http://www.nature.com/news/fewer-numbers-better-science-1.20858
>>
>> [4]
>> http://blog.scielo.org/en/2016/10/14/is-it-possible-to-normalize-citation-metrics/
>>
>> ___
>> Wiki-research-l mailing list
>> Wiki-research-l@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
>
>
>
> --
>
> Dario Taraborelli  Head of Research, Wikimedia Foundation
> wikimediafoundation.org • nitens.org • @readermeter
>
>
> ___
> Publicpolicy mailing list
> publicpol...@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/publicpolicy
>

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] [Wikimedia-l] "Can Wikipedia save the internet?": Wikipedia and political neutrality

[James Salsman]

Jon,

At the October Metrics and Activities meeting, you asked for feedback
on the Related Articles feature. Please have a look at this:
http://i.imgur.com/2aujFL7.png

The Making Work Pay Tax Credit was a provision of the Emergency
Economic Stabilization Act of 2008; possibly the most effective
provision responsible for ending and reversing the recession. And
perhaps for that reason, Republicans in Congress refused to renew it
in 2010 during the same series of negotiations in which they refused
to advance the infrastructure bills which did not pass from 2010 until
this year. I believe that the Making Work Pay Tax Credit is fully in
line with the Foundation's Mission and stated public policy
objectives, and so I have recommended that the Foundation endorse it:
https://lists.wikimedia.org/pipermail/publicpolicy/2016-September/001527.html

The FairTax is a flat tax which is considered very popular among
Libertarians and Objectivists. However, it has never had much
congressional support, likely because nearly all economists think it
would lead to ruin because it is so regressive. Libertarian
presidential candidate Gary Johnson recently became very upset when
asked by a British newspaper reporter about this subject:
https://youtu.be/vvULsrjLdI4?t=3m12s

Do you think FairTax is a useful related article to suggest to people
interested in the Making Work Pay Tax Credit?

To what extent does systemic bias towards fringe Libertarian and
Objectivist economics influence the Related Articles feature?


On Wed, Nov 2, 2016 at 1:11 PM, FRED BAUDER  wrote:
> The subject affected in this way are "hot," in the news, sometimes hourly,
> or involve major financial interests. Austerity economics is as good an
> example as major political candidates. I think statistics would show a
> relationship between news mentions and editing conflict, and, also, the
> amount of profit associated with marketing of a product.
>
> Fred
>
>
> On Wed, 2 Nov 2016 12:52:58 -0600
>  James Salsman  wrote:
>>
>> Fred and Craig,
>>
>> Do you think a comparison of the effects of bias in individual
>> candidates' articles to the effects systemic bias towards trickle-down
>> austerity economics and the social implications thereof in light of
>> the WP:MEDRS-grade source at http://talknicer.com/ehip.pdf might
>> produce a helpful indication of where counter-advocacy efforts would
>> best be focused?
>>
>> I'm un-crossposting this reply to just wiki-research-l and the
>> Education list because I've been told to not crosspost to more than
>> two lists.
>>
>> On Wed, 2 Nov 2016 12:59:09 -0400 Craig Newmark > gmail.com> wrote:
>>>
>>>
>>> Fred, thanks!
>>>
>>> Worth reviewing, after people have recovered from the election. How
>>> about...reminding me two weeks from today. I might've recovered by
>>> then,
>>> seriously...
>>>
>>> Thanks!
>>>
>>> Craig Newmark
>>>
>>> founder, craigslist
>>>
>>> On Nov 2, 2016 12:44 PM, "FRED BAUDER" 
>>> wrote:
>>>
>>>> Craig,
>>>>
>>>> I don't expect you to do anything about it, but Hillary Clinton
>>>> presidential campaign, 2016 has been so much an object of political
>>>> editing
>>>> by Clinton supporters that it looks more like an ad for Hillary than
>>>> a
>>>> Wikipedia article.
>>>>
>>>> Fred Bauder
>>>>
>>>> On Wed, 2 Nov 2016 11:43:32 -0400
>>>>  Craig Newmark  wrote:
>>>>
>>>>> Wikipedia is where facts go to live.
>>>>>
>>>>>
>>>>> It helps that folks on US Capitol Hill are receptive to quiet
>>>>> suggestions
>>>>> that Wikipedia avoid becoming a partisan battleground.
>>>>>
>>>>> Craig Newmark
>>>>>
>>>>> founder, craigslist
>>>>>
>>>>> On Nov 1, 2016 7:35 PM, "Olatunde Isaac" 
>>>>> wrote:
>>>>>
>>>>> Hoi,
>>>>>>
>>>>>>
>>>>>> Pine, thanks for sharing this article. I found the entire article
>>>>>> very
>>>>>> interesting. I am glad that Wikipedia is not seen as a vehicle for
>>>>>> political campaign. Sometimes, people create account on Wikipedia
>>>>>> with
>>>>>> the
>>>>>> aim to use the encyclopedia for political campaign and a good number
>>>>>> of
>>>>>> them end up gett

Re: [Wiki-research-l] [Wikimedia-l] "Can Wikipedia save the internet?": Wikipedia and political neutrality

[James Salsman]

Fred and Craig,

Do you think a comparison of the effects of bias in individual
candidates' articles to the effects systemic bias towards trickle-down
austerity economics and the social implications thereof in light of
the WP:MEDRS-grade source at http://talknicer.com/ehip.pdf might
produce a helpful indication of where counter-advocacy efforts would
best be focused?

I'm un-crossposting this reply to just wiki-research-l and the
Education list because I've been told to not crosspost to more than
two lists.

On Wed, 2 Nov 2016 12:59:09 -0400 Craig Newmark  wrote:
>
>Fred, thanks!
>
> Worth reviewing, after people have recovered from the election. How
> about...reminding me two weeks from today. I might've recovered by
>then,
> seriously...
>
> Thanks!
>
> Craig Newmark
>
> founder, craigslist
>
> On Nov 2, 2016 12:44 PM, "FRED BAUDER" 
>wrote:
>
>> Craig,
>>
>> I don't expect you to do anything about it, but Hillary Clinton
>> presidential campaign, 2016 has been so much an object of political
>>editing
>> by Clinton supporters that it looks more like an ad for Hillary than
>>a
>> Wikipedia article.
>>
>> Fred Bauder
>>
>> On Wed, 2 Nov 2016 11:43:32 -0400
>>  Craig Newmark  wrote:
>>
>>> Wikipedia is where facts go to live.
>>>
>>>
>>> It helps that folks on US Capitol Hill are receptive to quiet
>>>suggestions
>>> that Wikipedia avoid becoming a partisan battleground.
>>>
>>> Craig Newmark
>>>
>>> founder, craigslist
>>>
>>> On Nov 1, 2016 7:35 PM, "Olatunde Isaac" 
>>> wrote:
>>>
>>> Hoi,

 Pine, thanks for sharing this article. I found the entire article
very
 interesting. I am glad that Wikipedia is not seen as a vehicle for
 political campaign. Sometimes, people create account on Wikipedia
with
 the
 aim to use the encyclopedia for political campaign and a good number
of
 them end up getting blocked either for POV pushing or other
disruptive
 editing/behavior.

 BTWI have a few question. Is it a good idea to protect a page
from
 creation if there are indications that the overall intention of the
 creator
 is to use Wikipedia as a platform for political campaign? If yes,
how is
 such protection necessary if the page is neutrally written?

 There was an incident that happened sometimes last year when an
article
 on
 "Akinwunmi Ambode"  was protected from creation and unprotected
after his
 election.

 Is this really a good idea?

 Best,

 Isaac
 Sent from my BlackBerry® wireless handheld from Glo Mobile.

 -Original Message-
 From: Pine W 
 Sender: "Wikimedia-l" Date:
 Wed,
 26 Oct 2016 20:31:59
 To: Wikimedia Mailing List;
Wikimedia
 Education; Wiki
 Research-l>>> lists.wikimedia.org>
 Reply-To: Wikimedia Mailing List 
 Subject: [Wikimedia-l] "Can Wikipedia save the internet?": Wikipedia
and
 political neutrality

 Hello colleagues,

 Some of you might be interested in this news article:
 http://www.independent.co.uk/life-style/gadgets-and-tech/
 can-wikipedia-save-the-internet-a7380786.html

 When we know that we have countless shortcomings in Wikimedia, I
found it
 refreshing to hear that some aspects of our content and community
are
 performing well and, on the whole, are serving the public interest.

 Regards,

 Pine

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] supporting Wikipedia citations as formal scholarly reputation in tenure committees

[James Salsman]

Can anyone familiar with European Commission procedure please explain
how to support the Wikipedia-associated proposals in [1] based on the
statistics in [2] please? Very recent publications such as [3] in
Nature along with what appears to be a relatively sudden groundswell
of frankness and support e.g. [4] suggests to me that the time is
right to get out in front of these proposals.

[1] 
https://www.researchgate.net/profile/David_Nicholas5/publication/275349828_Emerging_reputation_mechanisms_for_scholars/links/553a22a60cf2c415bb06e6b7.pdf

[2] http://www.cs.indiana.edu/~xshuai/papers/jcdl240-shuai.pdf

[3] http://www.nature.com/news/fewer-numbers-better-science-1.20858

[4] 
http://blog.scielo.org/en/2016/10/14/is-it-possible-to-normalize-citation-metrics/

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] article of note

[James Salsman]

I enjoyed 
https://scholar.google.com/scholar?cluster=4061737011161182722&hl=en&as_sdt=0,6

CELL (BIOLOGY)-WIKIPEDIA LEARNING PERFORMANCE IN
RELATION TO COGNITIVE STYLES, LEARNING STYLES, AND
SCIENCE ABILITY OF STUDENTS: A HIERARCHICAL MULTIPLE
REGRESSION ANALYSIS

"It is resulted that Wikipedia learning performance was better over
traditional approach."

Best regards,
Jim

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] measuring time to proofread wikipedias and the Making Work Pay Tax Credit

[James Salsman]

I am pleased to announce that, thanks to Google Summer of Code student
Priyanka Mandikal, the project for the Accuracy Review of Wikipedias
project has delivered a working demonstration of open source code and
data available here:

https://github.com/priyankamandikal/arowf/

Please try it out at:

http://tools.wmflabs.org/arowf/

We need your help to test it and try it out and send us comments. You
can read more about the project here:

https://priyankamandikal.github.io/posts/gsoc-2016-project-overview/

The formal project report, still in progress (Google docs comments
from anyone are most welcome) is at:

https://docs.google.com/document/d/1_AiOyVn9Qf5ne1qCHIygUU3OTJcbpkb14N3rItyjaVQ/edit

This allows experiments to measure, for example, how long it would
take to complete proofreading of the wikipedias with and without
paying editors to work alongside volunteers. I am sure everyone agrees
that is an interesting question which bears directly on budget
expectations. I hope multiple organizations use the published methods
and their Python implementations to make such measurements. I would
also like to suggest a proposal related to the questions in both of
the following reviews:

http://unotes.hartford.edu/announcements/images/2014_03_04_Cerasoli_and_Nicklin_publish_in_Psychological_Bulletin_.pdf

http://onlinelibrary.wiley.com/doi/10./1748-8583.12080/abstract

The most recent solicitation of community input for the Foundation's
Public Policy team I've seen said that they would like suggestions for
specific issues as long as the suggestions did not involve
endorsements of or opposition to any specific candidates. My support
for adjusting copyright royalties on a sliding scale to transfer
wealth from larger to smaller artists has been made clear, and I do
not believe there are any concerns that I have not addressed
concerning alignment to mission or effectiveness. I would also like to
propose a related endorsement.

The Making Work Pay tax credit (MWPTC) is a negative payroll tax that
expired in 2010. It has all the advantages of an expanded Earned
Income Tax Credit (EITC) but would happen with every paycheck.
Reinstating the Making Work Pay tax credit would serve to reduce
economic inequality.

This proposal is within the scope of the Foundation's mission because
reducing economic inequality should serve to empower people to develop
educational content for the projects because of the increased levels
of support for artistic production among a broader set of potential
editors with additional discretionary free time due to increased
wealth. This proposal is needed because economic inequality produces
more excess avoidable deaths and leads to fewer years of productive
life than global warming. This proposal would provide substantial
benefits to the movement, the community, the Foundation, the US and
the world if it were to be successfully adopted. For the reasons
stated above, this proposal will be seen as positive.

Here is some background and supporting information:

* MWPTC overview: https://en.wikipedia.org/wiki/Making_Work_Pay_tax_credit

* MWPTC details: http://tpcprod.urban.org/taxtopics/2011_work.cfm

* Problems with expanding the EITC:
http://www.taxpolicycenter.org/taxvox/eitc-expansion-backed-obama-and-ryan-could-penalize-marriage-many-low-income-workers

* Educational advantages of expanding the EITC:
https://www.brookings.edu/opinions/this-policy-would-help-poor-kids-more-than-universal-pre-k-does/

* Financial advantages of expanding the EITC:
http://www.cbpp.org/research/federal-tax/strengthening-the-eitc-for-childless-workers-would-promote-work-and-reduce

* The working class has lost half their wealth over the past two
decades: https://www.nerdwallet.com/blog/finance/why-people-are-angry/

* Health effects of addressing economic inequality:
http://talknicer.com/ehlr.pdf

* Economic growth effects of addressing economic inequality:
http://talknicer.com/egma.pdf

* Unemployment and underemployment effects of addressing economic
inequality: http://diposit.ub.edu/dspace/bitstream/2445/33140/1/617293.pdf

For an example of how a campaign on this issue could be conducted
based on the issues identified in the sources above, please see:
http://bit.ly/mwptc

Please share your thoughts on the wikipedias proofreading time
measurement effort and this related public policy proposal.

I expect that some people will say that they do not understand how the
public policy proposal relates to the project to measure the amount of
time it would take to proofread the wikipedias. I am happy to explain
that in detail if and when needed. On a related note, I would like to
point out that the project report Google doc suggests future work
involving a peer learning system for speaking skills using the same
architecture as we derived from the constraints for successfully
performing simultaneous paid and volunteer proofreading. I would like
people to keep that in mind when evaluating the utility of these
proposals.

Si

[Wiki-research-l] Institutional review board exceptions in the United States

[James Salsman]

According to https://en.wikipedia.org/wiki/Institutional_review_board#Exceptions

"Research ... involving the study of instructional strategies or
effectiveness of various techniques [and] curricula ... [or] intended
to assess the performance or effectiveness of public benefit or
service programs" are exempt from IRB oversight.

Source: http://www.hhs.gov/ohrp/humansubjects/guidance/45cfr46.html

Best regards,
Jim

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] WMF Open Access Policy and Independent Researchers

[James Salsman]

Max, this advise is very good:

> On 6/29/2016 11:01, Stuart A. Yeates wrote:
>
> There are many open access journals which do not charge fees or any
> description.  See http://www.opendoar.org/ or talk to a friendly
> librarian to find a journal that meets your needs.

Please see also Table 5 on p. 1320 (PDF p. 6) in
http://octavia.zoology.washington.edu/publications/WestEtAl14.pdf

That suggests the Journal of Machine Learning Research may be a good
high-impact, zero-fee choice, if appropriate.

There are alternatives by the same authors at
http://www.eigenfactor.org/openaccess/oa.php?catid=50&pt=0
in particular, College and Research Libraries and the Journal of the
Medical Library Association, if appropriate.

http://www.eigenfactor.org/openaccess/oa.php?catid=29&pt=0
has another bunch, such as the Journal of Educational Research.

http://www.eigenfactor.org/openaccess/oa.php?catid=129&pt=0
suggests Computational Linguistics.

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] WMF initiative: Community Capacity Development

[James Salsman]

That reminds me, did en:User:Dispenser ever get the 20 terabytes he
wanted for a reflinks cache? The Foundation should be hosting it, not
a volunteer, because the Foundation can easily afford to defend the
(what I believe is very clearly) fair use but if a volunteer has to it
could ruin multiple years.

Refs.: 
https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(technical)/Archive_128#Working_again.3F
and 
https://en.wikipedia.org/wiki/Wikipedia_talk:Wikipedia_Signpost/Newsroom/Suggestions/Archive_23#Reflinks_is_dead

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] "identity disclosure hurt the reliability of review systems, but not necessarily efforts provision"

[James Salsman]

Kerry Raymond wrote:
>...
> we could use big data to try to pro-actively find patterns of undesirable 
> behaviour

I agree, including with the specific examples given, most if not all
of which could be implemented within a general accuracy review system.
Please consider the three-level (actually four-level, if you count the
community) review system depicted at http://i.imgur.com/NhvyfXc.png

It can be gamed to secure an unfair advantage if the reviewer
reputation database (which includes reviewer identity) is open, but if
if the identities, reputation measurements, and algorithms are kept
confidential, then it can be very robust against several kinds of
bias, vandalism, incompetence, and other quality deficits. That is in
part because of the same reasons which make disclosing reviewer
identity less accurate.

I am trying to figure out how the reviewer reputation database can be
audited without disclosing reviewer identities. It's very difficult to
imagine the release any information about the reviewers and their
actions, even in aggregate, without exposing information which could
potentially be used to game an unfair advantage, but if the identities
are double-blinded and other related information is coded, it's still
possible to audit what seem be some fairly important aspects of the
system's operation without any obvious vulnerabilities.

Cost is a huge part of the system's operation, and there are so many
aspects of cost that need to be measured in many different ways. The
system should be open to volunteers as well as paid reviewers in order
to make sure it has throughput guarantees, but setting the payment
schedule is difficult in an international context where the cost of
living varies so widely. My initial set of paid reviewers will be
around 96% in the US if I remember right.

If anyone wants to try the same thing simultaneously, please do.
Eliminating duplication of effort would be a nice problem to have down
the road.

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] "identity disclosure hurt the reliability of review systems, but not necessarily efforts provision"

[James Salsman]

Jonathan, I am so sorry
http://aisel.aisnet.org/amcis2015/e-Biz/GeneralPresentations/11/
is behind a paywall. It wasn't when I first found it, and that version is
miles away from me at the moment. It describes a truly fascinating
empirical simulation laboratory participation experiment, which shows that
anonymous review is more accurate than review with identity disclosure,
which is actually very easy to find literally centuries of replication, but
it also found that the costs were more similar than conventional wisdom.

I want everyone to see it because of what the specific
experiment says about ways to detect bias at the lowest possible cost. I
have a feeling that you will quickly think of ways to extend it to study
projects' editing.

Can someone who has access to that paper please share the method
and results as fair use?
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] "identity disclosure hurt the reliability of review systems, but not necessarily efforts provision"

[James Salsman]

Has anyone replicated the experiment described in
http://aisel.aisnet.org/amcis2015/e-Biz/GeneralPresentations/11/
yet?
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] The Wikimedia Research Newsletter 5(7) is out

[James Salsman]

> To answer your point about "basic categorisation of the nature of edits" I
> have two words for you: Revision Scoring

As Adam Wight pointed out at
https://meta.wikimedia.org/wiki/Research_talk:Revision_scoring_as_a_service#Perpetuating_bias
the Mediawiki system doesn't allow the editor to categorize their
reason for reverting, so currently revision scoring as a service will
not actually categorize the nature of what it is learning.

Supervised learning tasks have the ability to include such categories,
and although something derived from the ontology at
https://en.wikipedia.org/wiki/Wikipedia:Edit_summary_legend
will be selectable, probably from radio buttons or a pull-down menu,
during the http://mediawiki.org/wiki/Accuracy_review
pilot, there will still be an "other" catch-all option.

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Wikipedia Natural Disaster Monitor (was Re: "Wikipedia live monitor" for identifying breaking news on Wikipedia)

[James Salsman]

P.S. Would someone on Facebook please ask https://www.facebook.com/EarthWindMap
if they would consider integrating http://earth.nullschool.net
with Wikipedia Natural Disaster Monitor?

>> The "Wikipedia live monitor" tool was designed by Thomas Steiner
>>... this is the first that I can recall seeing it
>
> If you like WLM you will love the Wikipedia Natural Disaster Monitor:
>
> http://disaster-monitor.herokuapp.com/
>
> http://ceur-ws.org/Vol-1268/paper15.pdf
>
> https://docs.google.com/presentation/d/1b563mkeP91PMWJ6B_Fy3DdKK-SjM06gPVNu2s1uAxBk/edit#slide=id.i0
>
> https://github.com/tomayac/postdoc/tree/master/demos/disaster-monitor
>
> Best regards,
> Jim

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] "Wikipedia live monitor" for identifying breaking news on Wikipedia

[James Salsman]

Pine wrote:
>...
> The "Wikipedia live monitor" tool was designed by Thomas Steiner
>... this is the first that I can recall seeing it

If you like WLM you will love the Wikipedia Natural Disaster Monitor:

http://disaster-monitor.herokuapp.com/

http://ceur-ws.org/Vol-1268/paper15.pdf

https://docs.google.com/presentation/d/1b563mkeP91PMWJ6B_Fy3DdKK-SjM06gPVNu2s1uAxBk/edit#slide=id.i0

https://github.com/tomayac/postdoc/tree/master/demos/disaster-monitor

Best regards,
Jim

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Fwd: [opensource] Call for participation in OpenSym 2015, Aug 19-20, San Francisco!

[James Salsman]

-- Forwarded message --
From: *Dirk Riehle* 
Date: Saturday, July 4, 2015
Subject: [opensource] Call for participation in OpenSym 2015, Aug 19-20,
San Francisco!
To: opensou...@lists.stanford.edu


Call for participation in OpenSym 2015!

Aug 19-20, 2015, San Francisco, http://opensym.org



FOUR FANTASTIC KEYNOTES

Richard Gabriel (IBM) on Using Machines to Manage Public Sentiment on
Social Media

Peter Norvig (GOOGLE) on Applying Machine Learning to Programs

Robert Glushko (UC BERKELEY) on Collaborative Authoring, Evolution, and
Personalization

Anthony Wassermann (CMU SV) on Barriers and Pathways to Successful
Collaboration

More at
http://www.opensym.org/category/conference-contributions/keynotes-invited-talks/



GREAT RESEARCH PROGRAM

All core open collaboration tracks, including

- free/libre/open source
- open data
- Wikipedia
- wikis and open collaboration, and
- open innovation

More at
http://www.opensym.org/2015/06/25/preliminary-opensym-2015-program-announced/



INCLUDING OPEN SPACE

The facilities provide room and space for your own working groups.



AT A WONDERFUL LOCATION

OpenSym 2015 takes place from Aug 19-20 at the Golden Gate Club of San
Francisco, smack in the middle of the Presidio, with a wonderful view of
the Golden Gate Bridge.

More at http://www.opensym.org/os2015/location/



REGISTRATION

Is simple, subsidized, and all-encompassing.

Find it here: http://www.opensym.org/os2015/registration/

Prices will go up after July 12th, so be sure to register early!



We would like to thank our sponsors Wikimedia Foundation, Google, TJEF, and
the ACM.




___
opensource mailing list
opensou...@lists.stanford.edu
https://mailman.stanford.edu/mailman/listinfo/opensource
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Delph-in LOGON vs. traditional sentence complexity for isolating confusing passages

[James Salsman]

Has anyone studied comparing the use of http://erg.delph-in.net/logon
with traditional measures of text reading level for locating potentially
confusing sentences?
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Altmetric.com now tracks Wikipedia citations

[James Salsman]

Laura Hale wrote:
>...
> [Avoiding SEO spamming is] a Wikipedia thing, by putting
>
> 
> in the article source code

Of the mirrors that come and go from time to time, it always seems to
be about even odds as to whether they keep rel=nofollow in external
and citation links. So, some months being linked to from Wikipedia
does increase search engine ranking, if such mirrors are reliably
catching up, and other months not so much.

I'm enthusiastic about the academic establishment using Wikipedia
citations as performance metrics, but agree with everyone who wants to
keep an eye on gaming. At present, commercial gaming is thousands of
times more prevalent than academics trying to enhance their CVs. Let
us know when Altmetric.com or Wikipedia is mentioned in a decision to
grant or deny tenure (track hires.)

I would love to see this become an easier way to find the secondary
peer reviewed literature reviews, because they aren't always where
people expect them to be published at all. If we attract academics in
whose interest it is to help with that, and they understand how and
actually do help, that would be great.

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Endowment perpetuity

[James Salsman]

Speaking of fundraising far over budget, did the question about an
endowment perpetuity make it on to the last donor survey? If so, what was
the result? I seem to remember a favorable response, from somewhere, but
can't find anything either way. Was it a Board poll?
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Fwd: $55 million raised in 2014

[James Salsman]

Han-Teng Liao wrote:
>...
> I hope that the Wikimedia foundation budget grows in proportion
> with the number of Internet users, and the average donations
> remains the same (inflation-adjusted).

Do you think donations will grow in proportion to the median income of
internet users? That measure is likely to continue to grow for 15
years or more after the total number of users' growth substantially
slows.

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Fwd: $55 million raised in 2014

[James Salsman]

>> I wish someone would please replicate my measurement of
>> the variance in the distribution of fundraising results using
>> the editor-submitted banners from 2008-9, and explain to the
>> fundraising team that distribution implies they can do a whole
>> lot better When are they going to test the remainder of the
>> editors' submissions?
>
> Given that you've been asking for that analysis for four years,
> and it's never been done, and you've been repeatedly told that
> it's not going to happen, could youtake those hints? And by
> hints, I mean explicit statements

Which statements? I've been told on at least two occasions that the
remainder of the volunteer submissions *will* be tested, with
multivariate analysis as I've suggested (instead of much more lengthy
rounds of A/B testing, which still seem to be the norm for some
reason) and have never once been told that it's not going to happen,
as far as I know. Who ruled it out and why? Is there any evidence that
my measurement of the distribution's kurtosis is flawed?

I'll raise the issue as to whether and how much the Foundation should
pay to crowdsource revision scoring to help transition from building
new content to updating existing articles when the appropriate
infrastructure to measure the extent of volunteer effort devoted to it
is in place. If there is any reason for refraining from discussion of
the fact that revision scoring can be equivalent to computer-aided
instruction and the ways that it can be implemented to maximize its
usefulness as such, then please bring it to my attention.

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Fwd: $55 million raised in 2014

[James Salsman]

>>... here are a couple illustrations of some reasons I
> believe a ten year extrapolation of Foundation fundraising
> is completely reasonable: http://imgur.com/a/mV72T
>
> Words tend to be more useful than contextless images.

I meant that the very sharply declining cost of solar (and wind)
energy, and the extent to which renewable energy is becoming fungible
-- see e.g. 
https://en.wikipedia.org/w/index.php?title=Wikipedia:Reference_desk/Science&oldid=640703658#Using_the_night-time_electricity_system_as_a_means_of_dealing_with_excess_electricity_generation_due_to_renewables.
-- is likely to have a profoundly positive economic effect in both the
developed and developing world, especially over the next ten years;
and that the rate at which the most populous areas of the world are
growing in terms of income per capita will combine to support a simple
extrapolation of the Foundation's fundraising success. The very rapid
per capita income growth in Asia and Africa should last for 15 to 20
years at least.

And I don't see any downward pressure on the ability of the Foundation
to raise money; especially if transitioning to maintaining existing
content is successful. That is why I think this is so important:

>>> https://meta.wikimedia.org/wiki/Grants:IEG/Revision_scoring_as_a_service
>
>> That is equivalent to a general computer-aided instruction
>> system, with the side effects of both improving the encyclopedia
>> and making counter-vandalism bots more accurate. As an
>> anonymous crowdsourced review system based on consensus
>> voting instead of editorial judgement, it leaves the Foundation
>> immunized with their safe harbor provisions regarding content
>> control intact.
>
>  It's also not worth 3 billion dollars (no offence, Aaron!) as
> evidenced by the fact that it can be established with <20k.

I agree it can be set up with very little money, and I am completely
thrilled beyond words that work is proceeding on it.

However, once it is established, it's impossible to say whether
volunteers can sustain it at useful levels. I think it's almost
impossible that volunteers will keep it up with even half of major
edits. However, again, paying people to score revisions (including
trial null revisions against existing content, for example, that
editors could flag as being out of date, for example) would be like
paying them to enrich their own education and improve the encyclopedia
and anti-vandalism bots all at the same time. That is a fantastic
opportunity for research and development.

>... This is not a discussion for research-l

On the contrary, please see e.g.
http://www.wikisym.org/os2014-files/proceedings/p609.pdf
this Foundation-sponsored IEG effort can serve as a confirmatory
replication of that prior work.

>... time is better spent doing research with the resources
> we have now

I wish someone would please replicate my measurement of the variance
in the distribution of fundraising results using the editor-submitted
banners from 2008-9, and explain to the fundraising team that
distribution implies they can do a whole lot better than sticking with
the spiel which degrades Foundation employees by implying they
typically spend $3 or £3 on coffee. (Although I wouldn't discount the
possibility that some donors feel good about sending Foundation
employers to boutique coffee shops.)

We know donor message- and banner-fatigue exists as a strong effect
which limits the useful life of fundraising approaches in some cases,
so they have to keep trying to keep up. When are they going to test
the remainder of the editors' submissions?

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Fwd: $55 million raised in 2014

[James Salsman]

Oliver Keyes wrote:
>
>... Extrapolation is not a particularly useful method to use for
> the budget, because it assumes endless exponential growth.

I agree. Formal budgeting usually shouldn't extend further than three
to five years in the nonprofit sector (long-term budgeting is
unavoidable in government and some industry.)  However, here are a
couple illustrations of some reasons I believe a ten year
extrapolation of Foundation fundraising is completely reasonable:
http://imgur.com/a/mV72T

>... I can't see what we'd actually /do/ with 3 billion dollars

I used to be in favor of a establishing an endowment with a sufficient
perpetuity, and then halting fundraising forever, but I have changed
my mind. I think the Foundation should continue to raise money
indefinitely to pay people for this task:
https://meta.wikimedia.org/wiki/Grants:IEG/Revision_scoring_as_a_service

That is equivalent to a general computer-aided instruction system,
with the side effects of both improving the encyclopedia and making
counter-vandalism bots more accurate. As an anonymous crowdsourced
review system based on consensus voting instead of editorial
judgement, it leaves the Foundation immunized with their safe harbor
provisions regarding content control intact.

Best regards,
James Salsman

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Fwd: $55 million raised in 2014

[James Salsman]

In ten years time, I predict the Foundation will raise $3 billion:
http://i.imgur.com/hdoAIan.jpg


-- Forwarded message --
From: James Salsman 
Date: Thu, Jan 1, 2015 at 9:01 PM
Subject: $55 million raised in 2014
To: Wikimedia Mailing List 


Happy new year: http://i.imgur.com/faPsI9J.jpg

Source: http://frdata.wikimedia.org/yeardata-day-vs-ytdsum.csv

I don't mind the banners, although I am still saddened that several
hundred editor-submitted banners remain untested from six years
ago, when the observed variance in the performance of those that were
tested indicates that there are likely at least 15 which would do
better than any of those which were tested. Why the heck is the
fundraising team still ignoring all those untested submissions?

But as to the intrusiveness of the banners, I would rather have
fade-in popups with fuschia  text on a epileptic
seizure-inducing background and auto-play audio than have the
fundraising director claim that donations are decreasing to help
justify "narrowing scope."

Best regards,
James Salsman

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] names of Korean supreme court justices

[James Salsman]

Regarding these articles:

> http://en.wikipedia.org/wiki/Lee_In-bok
>
> http://en.wikipedia.org/wiki/Lee_Sang-hoon
>
> http://en.wikipedia.org/wiki/Kim_Shin
>
> http://en.wikipedia.org/wiki/Kim_So-young

The problem is probably more in the transliteration between Korean and
Latin alphabets, where each name, comprised of three glyphs in an
alphabet with 1,309 glyphs, results in quite a bit of ambiguity due to
overlap with other persons' identical names. English Wikipedia editors
have certainly not maintained sufficient disambiguation headnotes on
those articles as they progressed over time. However, there are
certainly plenty of articles on the very many specific unambigious
Korean Supreme Court Justices in the Korean Wikipedia:

https://ko.wikipedia.org/wiki/%EC%96%91%EC%8A%B9%ED%83%9C

https://ko.wikipedia.org/wiki/%EC%82%AC%EB%B2%95%EC%8B%9C%ED%97%98

https://ko.wikipedia.org/wiki/%EC%96%91%EC%B0%BD%EC%88%98

https://ko.wikipedia.org/wiki/%EC%8B%A0%EC%98%81%EC%B2%A0_(1954%EB%85%84)

https://ko.wikipedia.org/wiki/%EB%AF%BC%EC%9D%BC%EC%98%81

https://ko.wikipedia.org/wiki/%EC%9D%B4%EC%9D%B8%EB%B3%B5_(%EB%B2%95%EC%A1%B0%EC%9D%B8)

https://ko.wikipedia.org/wiki/%EC%9D%B4%EC%83%81%ED%9B%88_(1956%EB%85%84)

https://ko.wikipedia.org/wiki/%EB%B0%95%EB%B3%91%EB%8C%80

https://ko.wikipedia.org/wiki/%EA%B9%80%EC%9A%A9%EB%8D%95

https://ko.wikipedia.org/wiki/%EB%B0%95%EB%B3%B4%EC%98%81_(%EB%B2%95%EC%A1%B0%EC%9D%B8)

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] StackExchange editor decline (serverfault)

[James Salsman]

Jonathan Morgan wrote:
>...
> not a single Korean Supreme Court Justice has an article.

http://en.wikipedia.org/wiki/Yang_Sung-tae

http://en.wikipedia.org/wiki/Lee_In-bok

http://en.wikipedia.org/wiki/Lee_Sang-hoon

http://en.wikipedia.org/wiki/Kim_Shin

http://en.wikipedia.org/wiki/Kim_So-young

Aaron Halfaker wrote:
>...
> It seems clear that hostility has increased.  Look at this
> graph specifically:
>
> https://meta.wikimedia.org/wiki/File:Desirable_newcomer_reverts_over_time.png

Where is the evidence that a greater proportion of reverts is
associated with increased hostility instead of higher article quality
standards?

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] StackExchange editor decline (serverfault)

[James Salsman]

Jonathan Morgan wrote:
>
> On Thu, Dec 11, 2014 at 3:15 PM, James Salsman  wrote:
>
>> I continue to maintain that editor attrition is due to the natural
>> transition from writing and completing new articles to maintaining old
>> articles, and have seen nothing to convince me otherwise or of the
>> validity of any alternative hypothesis.
>
> /me nods
>
> Sure, that's likely a huge factor. But do you really believe it's the
> *only* one?

It's certainly the only factor that I've ever seen supported by
convincing data. A larger problem is that people continue to advance
hypotheses which are easy to disprove. For example, people frequently
say that hostility became worse after 2007. I can't see any support
for that. If you don't believe me, go to a popular controversial
article, then click "history" and "oldest" e.g.

https://en.wikipedia.org/w/index.php?title=Global_warming&dir=prev&action=history

https://en.wikipedia.org/wiki/Talk:Global_warming/Archive_3#Examine_effects_of_change

What other hypotheses can be supported by any data at all?

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] StackExchange editor decline (serverfault)

[James Salsman]

I posted some data about the proportion of Wikipedia articles' edits
by year here:
https://plus.google.com/u/0/113215976889659570939/posts/5LLxgfCeaMa

I continue to maintain that editor attrition is due to the natural
transition from writing and completing new articles to maintaining old
articles, and have seen nothing to convince me otherwise or of the
validity of any alternative hypothesis.

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Tool to find poorly written articles

[James Salsman]

Ditty,

Article quality is inherently subjective in the hard-AI sense. A panel of
judges will consider accurate articles full of spelling, grammar, and
formatting errors superior in quality to hoax, biased, spam, or out-of-date
articles with perfect grammar, impeccable spelling, and immaculate
formatting.

In my studies of the short popular vital articles (WP:SPVA) the closest
correlation with subjective mean opinion score quality I've found so far is
sentence length. But it has diminishing returns and the raw correlation is
+0.2 at best.

The entirely subjective nature of article quality is additional support for
automating accuracy review.

Best regards,
James
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] FW: What works for increasing editor engagement?

[James Salsman]

Luca wrote:
>
> Re. the edit conflicts happening when a new user is editing:
>
> Can't one add some AJAX to the editor that notifies that one
> still has the editing window open? Maybe editors could wait to
> modify work in progress, if they had that indication, and if the
> content does not seem vandalism?

Instead of asking editors to wait, we could improve the merge
algorithm to avoid conflicts:

https://en.wikipedia.org/wiki/Merge_(revision_control)

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] FW: What works for increasing editor engagement?

[James Salsman]

Aaron, would you please post the script you used to create
https://commons.wikimedia.org/wiki/File:Desirable_newcomer_survival_over_time.png
?

I would be happy to modify it to also collect the number of extant
non-redirect articles each desirable user created.

> Aaron wrote:
> >... You'll find the hand-coded set of users here
> > http://datasets.wikimedia.org/public-datasets/enwiki/rise-and-decline
> >...
> > Categories:
> >
> >   1. Vandals - Purposefully malicious, out to cause harm
> >   2. Bad-faith - Trying to be funny, not here to help or harm
> >   3. Good-faith - Trying to be productive, but failing
> >   4. Golden - Successfully contributing productively

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] FW: What works for increasing editor engagement?

[James Salsman]

Thanks, Aaron, that is very helpful!

Do you also have data sets for number of still-extant non-redirect
articles created for the good-faith and golden users?

I can figure that out from the API, but I'm busy. Also, I have
previously hypothesized that creating an article is strongly
correlated with survival of desirable newcomers, and you probably have
people working for you who have no preconceived notions on the
question, or if you don't you can find them.

Best regards,
James Salsman

Aaron wrote:
>... You'll find the hand-coded set of users here
> http://datasets.wikimedia.org/public-datasets/enwiki/rise-and-decline
>...
> Categories:
>
>   1. Vandals - Purposefully malicious, out to cause harm
>   2. Bad-faith - Trying to be funny, not here to help or harm
>   3. Good-faith - Trying to be productive, but failing
>   4. Golden - Successfully contributing productively

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] FW: What works for increasing editor engagement?

[James Salsman]

Hi Aaron,

Is the data set from
https://commons.wikimedia.org/wiki/File:Desirable_newcomer_survival_over_time.png
available for correlation with the number of new articles each user created?

Aaron Halfaker  wrote:
>...
> I propose a project where we work together to generate
> a summary so that I can call it "work." I've started a stub here:
> https://meta.wikimedia.org/wiki/Research:New_editor_engagement_strategies
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] What works for increasing editor engagement?

[James Salsman]

Oliver Keyes wrote:
>...
> the reason Mobile is going to have an impact is not that it will
> have an impact on the delta, but because there are additional
> factors to juggle when working on solutions to said delta.

Are you saying that the mobile skin will affect editor attrition once
it is able to display and edit templates?

If so, why do you think that template display will be added to the
mobile skin when there was a conscious decision to remove it which has
stood for years? Why do you think template editing will ever be part
of the mobile skin?

And even if the mobile skin could edit templates, what data is there
supporting the idea that it would have any impact on editor
engagement? Pageviews when mobile views are included have remained on
their current trend for longer than editor attrition rate has held
steady. I don't see a shred of evidence that platform proportions have
had or ever will have any influence on editor engagement.

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] What works for increasing editor engagement?

[James Salsman]

Gerard Meijssen wrote:
>...
> Please define "just worked fine"... Really ?? !!
> Try editing a page that starts with a template..

Editing pages with or without templates works under the Vector skin on both
iOS and Android, although scrolling in the textarea can be difficult if you
aren't used to it.

Are referring to the fact that the mobile skin silently omits many if not
most templates, and prevents users from editing them? The thought that
active editors will ever take a skin which does that seriously is absurd.
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] What works for increasing editor engagement?

[James Salsman]

Pine wrote:
>...
> The data you show in that table indicates that
> there is a negative correlation between active
> editors and mobile pageviews

No, it does not. The rate of editor attrition has been constant since 2007,
while mobile views have increased from zero to billions. Mobile pageviews
have has absolutely no correlation with editor engagement whatsoever.

If there is a quantification of civility issues per editor somewhere,
please bring it to my attention. I suggest that editors who think
incivility has increased since 2006 are not familiar with incivility issues
prior to 2006.

Gerard Meijssen wrote:
>...
> all efforts intending to enable mobile editors
> enable a latent potential of editors.

Editing under the Vector skin has worked just fine on Android since 2010
and on iOS since 2012. There is no evidence that the edits under mobile
device specialty skins or apps will ever approach the proportion of editing
under the Vector skin.

Nor is there any evidence that the increasing proportion of mobile
pageviews has had any impact on the number of active editors, who again
have been declining along a constant trend since 2007 to the present, even
as mobile pageviews have displaced a very substantial and growing
proportion of desktop pageviews.

> The visual editor has as a side benefit that we
> will be moving away from Wiki editing.

Are you aware of the proportion of active editors who have enabled the
visual editor? It is miniscule, judging from tags in recent changes.

Oliver Keyes wrote:
>...
> the increase of mobile traffic is going to have an impact
> on efforts to reverse the negative trend in active editors.

Why? Mobile pageviews are now 30%. Editing is enabled and relatively easy
on mobile devices. The rate of editor attrition is unchanged from 2007.
Where is there any evidence that the trend in active editors will change at
all if and when mobile pageviews reach 50% or 75%?
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] What works for increasing editor engagement?

[James Salsman]

Oliver Keyes wrote:
> ...
> Mobile now makes up 30% of our page views and its
> users display divergent behavioural patterns; you
> don't think a group that makes up 30% of pageviews
> is a user group that is a 'big deal' for engagement?

For the English Wikipedia:

  >100Million
active mobile
Date   editors  Change  pageviews  Change
July 20093,795 -7%
July 20103,517 -7%   278
July 20113,374 -4%   571 105%
July 20123,360  0% 1,210 112%
July 20133,135 -7% 1,880  55%
July 20143,037 -3% 3,010  60%

Where is the evidence that mobile use has any influence on editor engagement?

If you want to predict how long editors will stay, compare how many
new articles they were successfully creating in their first 500 edits
in 2004-2006 versus 2008-present.

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] What works for increasing editor engagement?

[James Salsman]

Pine wrote:
>
> I agree that the shift to mobile is a big deal;

I do not agree: Active editor attrition began on its present trend in
2007, far before any mobile use was significant.

> I remain concerned that tech-centric approaches
> to editor engagement like VE and Flow, while
> perhaps having a modest positive impact, do little
> to fix the incivility problem that is so frequently
> cited as a reason for people to leave.

I agree that VE has already proven that it is ineffective in
significantly increasing editor engagement. And I agree that Flow has
no hope of achieving any substantial improvements. There are good
reasons to believe that Flow will make things worse. For example,
using wikitext on talk pages acts as a pervasive sandbox substitute
for practicing the use of wikitext in article editing.

And I do not agree that civility issues have any substantial
correlation with editor attrition. There have been huge civility
problems affecting most editors on controversial subjects since 2002,
and I do not see any evidence that they have become any worse or
better on a per-editor basis since.

My opinion is that the transition from the need to create new articles
to maintaining the accuracy and quality of existing articles has been
the primary cause of editor attrition, and my studies of Short Popular
Vital Articles (WP:SPVA) have supported this hypothesis.

Therefore, I strongly urge implementation of accuracy review systems:
https://strategy.wikimedia.org/wiki/Proposal:Develop_systems_for_accuracy_review

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Free open knowledge MOOC, accuracy review, strategy survey

[James Salsman]

Please forgive the crossposting. If you or someone you know who doesn't
understand enough about OERs have a handful of hours per week this fall,
please consider:

https://class.stanford.edu/courses/Education/OpenKnowledge/Fall2014/about

Stanford is trying to flip undergraduate instruction. My standard by which
I judge MOOCs are whether they tell students to refrain from communicating
with the instructors.

Lila Tretikov wrote:
>...
> Let's think big.

Okay, I would like to change my opinion about http://
strategy.wikimedia.org/wiki/Proposal:Develop_systems_for_accuracy_review

While I still would love to work on it if I had the spare time, I would
also like to ask the Foundation to try to do it first. I feel the same way
about a strategic objective survey for increasing participation by
nurturing social change likely to increase potential volunteer editor spare
time, and I feel both are about equally important for the future of the
projects.

Best regards,
James Salsman
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] stats.grok.se beats Google Flu Trends for predicting outbreaks

[James Salsman]

McIver, David J. and John S. Brownstein (April 17, 2014) "Wikipedia
Usage Estimates Prevalence of Influenza-Like Illness in the United
States in Near Real-Time" PLOS Computational Biology:
http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003581

"Wikipedia article view data has been demonstrated to be effective at
estimating the level of Influenza-Like Illness [ILI] activity in the
US, when compared to CDC data and can provide a reliable estimate
of ILI activity up to 2 weeks in advance of traditional ILI
reporting."

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] community survey

[James Salsman]

Dear Federico:

Do you have time to re-do the European community survey, or know any
trusted community members who do? I am nowhere near Europe. Any
trusted community member can update the questionnaire, and I am sure
you could get volunteers to help translate it:
http://meta.wikimedia.org/wiki/General_User_Survey/Questionnaire

Furthermore, there are some pretty obvious reasons why anyone at the
Foundation or any Foundation-funded entity has a conflict of interest
when trying to administer such surveys, which manifests in ways which
we should try to reduce. The last time I tried to run a survey I was
accused of violating a proposal. That makes it much harder for me to
run another one than it would if you did. I am confident that the
permissions involved will be restored, because I am confident that the
Foundation will try to make amends for their mistake someday. It took
me about 60 hours for 330 enwiki administrators, but there were
complicated issues and questions to which I still have not received a
response.

Original: http://meta.wikimedia.org/wiki/EU_policy/Survey

16 proposed additions followed by the earlier list of 24:

1. Labor rights, e.g., linking to fixmyjob.com

2. Support the ratification of the Convention on the Rights of the
Child and its protocols without reservation

3. Increase infrastructure spending

4. Increase education spending

5. Public school class size reduction

6. College subsidy with income-based repayment terms

7. More steeply progressive taxation

8. Negative interest on excess reserves

9. Telecommuting

10. Workweek length reduction

11. Single-payer health care

12. Renewable power purchase

13. Increased data center hardware power efficiency

14. Increased security against eavesdropping

15. Metropolitan broadband

16. Oppose monopolization of software, communications, publishing, and
finance industries

---

A. Open Access (Scientific Research)

B. Database Rights

C. Freedom of Information

D. Orphan Works

E. Broadband Internet Access

F. Data Protection

G. Human Rights

H. Freedom of Panorama

I. Open (Government) Data

J. Censorship

K. Copyright on Government Works

L. Internet Neutrality

M. Three-strikes laws

N. Cultural Heritage

O. Data Retention

P. Provider/Hoster Liability

Q. Copyright Enforcement

R. Geodata

S. Open Educational Resources

T. Software Patents

U. Research Funding

V. Surveillance

W. Public Broadcasting

X. Frequency Allocation

Justification from previous messages:

> Our advocacy orientation isn't well aligned with the issues that most
> affect Wikimedians at present, because previous surveys had ... flaws.

Would quantitative measures of how various proposed actions counter
threats to building and sharing free knowledge help?

For example, if someone makes a case that acting successfully on some
issue is likely to cause X additional hours of productive editor
contribution time than failing to act on it, and nobody disagrees with
the analysis, or, if the analysis is supported by reliable sources,
nobody is able to counter those sources or show that they aren't
applicable, then the Foundation could be obligated to at least open a
formal RFC on the topic, and at larger thresholds of X, for example,
point people to it with CentralNotice or watchlist notices etc.

A good specific example is the Comcast-Time Warner Cable issue. I
think we should act to avoid monopoly consolidation of internet
resources, and there are sources which measure the extent to which
monopolies result in additional rent-seeking which would tend to
exclude editors. But I'm not particularly motivated to ask for action
on it without some expectation of whether it is even worth it to try
to persuade people.

see also:
http://lists.wikimedia.org/pipermail/advocacy_advisors/2014-February/000394.html

Best regards,
James Salsman

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Notes from the wiki research session at CSCW '14

[James Salsman]

>... There’s also a related ... project that was piloted a few months
> ago to try and gauge gender gap in specific segments of the editor population
> or editor lifecycle via microsurveys I’d love to hear from other parties
> interested in using this model, which I think is promising

Having run a mildly successful retention-oriented survey of about 330
enwiki administrators which caused me to be accused of violating a
proposed but unadopted research policy, I am still undeterred.

>> Were there any follow ups to the "annual" editor survey from 2011? A blog 
>> post
>> says the survey was anticipated to be annual. There is a page about a 2012
>> annual survey on Meta but no results are posted and it appears no follow up
>> surveys were completed in 2012 or 2013
>
> As Tilman noted in the section of the report about surveys, at this stage 
> it’s not
> clear if there’s bandwidth to run these surveys on an annual basis.

Is there any reason to think that the Foundation is more competent at
administering such surveys than the community?

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] General user survey: future plans?

[James Salsman]

>> Is anyone working on a general user survey?

Sorry I left the 'l' off the end of this link:

http://lists.wikimedia.org/pipermail/advocacy_advisors/2014-February/000394.html

Here are some related links:

http://meta.wikimedia.org/wiki/EU_policy/Survey

http://lists.wikimedia.org/pipermail/advocacy_advisors/2014-February/000388.html

http://lists.wikimedia.org/pipermail/advocacy_advisors/2014-February/000390.html

> Our advocacy orientation isn't well aligned with the issues that most
> affect Wikimedians at present, because previous surveys had ... flaws.

Here are 16 proposed additions followed by the earlier list of 24 from
the EU policy survey:

1. Labor rights, e.g., linking to fixmyjob.com

2. Support the ratification of the Convention on the Rights of the
Child and its protocols without reservation

3. Increase infrastructure spending

4. Increase education spending

5. Public school class size reduction

6. College subsidy with income-based repayment terms

7. More steeply progressive taxation

8. Negative interest on excess reserves

9. Telecommuting

10. Workweek length reduction

11. Single-payer health care

12. Renewable power purchase

13. Increased data center hardware power efficiency

14. Increased security against eavesdropping

15. Metropolitan broadband

16. Oppose monopolization of software, communications, publishing, and
finance industries

---

A. Open Access (Scientific Research)

B. Database Rights

C. Freedom of Information

D. Orphan Works

E. Broadband Internet Access

F. Data Protection

G. Human Rights

H. Freedom of Panorama

I. Open (Government) Data

J. Censorship

K. Copyright on Government Works

L. Internet Neutrality

M. Three-strikes laws

N. Cultural Heritage

O. Data Retention

P. Provider/Hoster Liability

Q. Copyright Enforcement

R. Geodata

S. Open Educational Resources

T. Software Patents

U. Research Funding

V. Surveillance

W. Public Broadcasting

X. Frequency Allocation

Best regards,
James Salsman

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] General user survey: future plans?

[James Salsman]

> Is anyone working on a general user survey?

http://lists.wikimedia.org/pipermail/advocacy_advisors/2014-February/000394.htm

Our advocacy orientation isn't well aligned with the issues that most
affect Wikimedians at present, because previous surveys had some huge bias
flaws. We need to cover about twelve categories previously under "human
rights." For example, class size and negative interest on excess reserves.
A lot of our copyright advocacy skirts blatant plagiarism which anyone
skilled at summarization would wonder why to ever care about.
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] generating QA review questions?

[James Salsman]

Has anyone been working on automatic generation of QA review questions
weighted by their real-world outcome preference implications?

I recently noticed a controversy about when rebar for reinforced
concrete was invented. I had looked the fact up earlier out of idle
curiosity, but I realize that many insurance plans are struggling with
the end-of-life claims on reinforced concrete foundations, so there is
a potential financial incentive to mislead people about the date that
rebar was invented: it would be easier to terminate insurance
contracts and total more buildings, which has substantial dollar
figure financial implications.

I am not suggesting the process of ranking questions by their dollar
value implications could be automated without human support, but has
anyone explored the extent to which automation can assist? Updating is
a core, if not the most core as of last year, editing activity
necessary to maintain quality. Wikidata can help, but QA review needs
to be independent and relatively anonymous.

Best regards,
James Salsman

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Existitng Research on Article Quality Heuristics?

[James Salsman]

Maximilian Klein wrote:

>... Can you also think of any other dimensions or heuristics
> to programatically rate?

Ref tags per article text bytes works pretty well, even by itself.

Also, please consider readability metrics. I would say that at this point
on enwiki, about a third of our real reader-impeding quallity issues have
more to do with overly technical jargon-laden articles, which usually also
have word and sentence length issues, than underdeveloped exposition.
Especially our math articles, many of which are almost useless for
undergraduates, let alone students at the earlier grade levels where the
corresponding concepts are introduced.

The good news is that doesn't seem to be happening in other topic areas
like biology, physics, or medicine. But math is kind of a disaster area
that way and it's not getting better with time.
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] gastroenterology and hepatology articles (was Re: Fwd: the Helsinki Times evaluates...)

[James Salsman]

Has there ever been a general purpose encyclopedia which was found
suitable for medical student instruction?

What are our median level readers going to do if we suddenly start
including enough pathophysiology images to please the med school
instructors? I'm not entirely sure it will help them, although on the
other hand it might encourage them to see a professional which is what
they often should be doing instead of reading Wikipedia. (But if
wishes were horses, beggars would ride)

>... Daniel Mietchen wrote:
>>
>> A similar paper on 39 gastroenterology/ hepatology articles on the
>> English Wikipedia came to different conclusions:
>> https://en.wikipedia.org/wiki/Wikipedia_talk:WikiProject_Medicine#Paper:_.22Evaluation_of_gastroenterology_and_hepatology_articles_on_Wikipedia:_Are_they_suitable_as_learning_resources_for_medical_students.3F.22

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Principal component analysis for multivariate testing

[James Salsman]

I have long wondered why the WMF Fundraising department seems stuck on
A/B testing instead of multivariate analysis.[1] Does anyone know?

In any case, modern multivariate analysis depends on principal
component analysis[2] (PCA) and it really does work great. I have been
told for years that it is planned and can't wait to see what happens
to fundraising when implemented.

So the reason why I am writing is because I just noticed that
EIGENSOFT[3] version 5 has "a new option for PCA projection with large
amounts of missing data" in pca.c[4] which is really clean code, too.
I hope this helps.

Best regards,
James Salsman

[1] http://en.wikipedia.org/wiki/Multivariate_analysis
[2] http://en.wikipedia.org/wiki/Principal_component_analysis
[3] http://www.hsph.harvard.edu/alkes-price/software/
[4] http://www.hsph.harvard.edu/alkes-price/files/2013/08/EIG5.0.1.tar.gz

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] *** Extended deadline *** CfP: 4th International Workshop on Model-driven Approaches for Simulation Engineering part of the Symposium on Theory of Modeling and Simulation (SCS Sp

[James Salsman]

Dear Drs. Gianni and Cuccuru:

Are you aware that you posted a solicitation of papers for publication
in non-open access journals to wiki-research-l? Ref.:
http://www.scs.org/publications

Would you please find a publisher adhering to the open access principles?

Thank you.

Regards,
James Salsman


> Date: Sun, 27 Oct 2013 22:10:22 +0100
> From: Daniele Gianni
> To: wiki-research-l@lists.wikimedia.org
> Subject: [Wiki-research-l] *** Extended deadline *** CfP: 4th
> International Workshop on Model-driven Approaches for Simulation
> Engineering part of the Symposium on Theory of Modeling and Simulation
> (SCS SpringSim 2014)
> Message-ID:
> 
> Content-Type: text/plain; charset="windows-1252"
>
> (Please accept our apologies if you receive multiple copies of this CFP)
>
> #
>  CALL FOR PAPERS
>
>  4th International Workshop on
> Model-driven Approaches for Simulation Engineering
> part of the Symposium on Theory of Modeling and Simulation
>  (SCS SpringSim 2014)
>
>
> #
>
> April 13-16, 2014, Tampa, FL (USA)
> http://www.sel.uniroma2.it/Mod4Sim14
>
> #
> # Papers Due: *** November 22, 2013 *** extended
> # Accepted papers will be published in the conference proceedings
> # and archived in the ACM Digital Library.
> #
>
> The workshop aims to bring together experts in model-based, model-driven
> and software engineering with experts in simulation methods and simulation
> practitioners, with the objective to advance the state of the art in
> model-driven simulation engineering.
>
> Model-driven engineering approaches provide considerable advantages to
> software systems engineering activities through the provision of consistent
> and coherent models at different abstraction levels. As these models are in
> a machine readable form, model-driven engineering approaches can also
> support the exploitation of computing capabilities for model reuse,
> programming code generation, and model checking, for example.
>
> The definition of a simulation model, its software implementation and its
> execution platform form what is known as simulation engineering. As
> simulation systems are mainly based on software, these systems can
> similarly benefit from model-driven approaches to support automatic
> software generation, enhance software quality, and reduce costs,
> development effort and time-to-market.
>
> Similarly to systems and software engineering, simulation engineering can
> exploit the capabilities of model-driven approaches by increasing the
> abstraction level in simulation model specifications and by automating the
> derivation of simulator code. Further advantages can be gained by using
> modeling languages, such as UML and SysML – but not exclusively those. For
> example, modeling languages can be used for descriptive modeling (to
> describe the system to be simulated), for analytical modeling (to specify
> analytically the simulation of the same system), and for implementation
> modeling (to define the respective simulator).
>
> A partial list of topics of interest includes:
>
> * model-driven simulation engineering processes
> * requirements modeling for simulation
> * domain specific languages for modeling and simulation
> * model transformations for simulation model building
> * model transformations for simulation model implementation
> * model-driven engineering of distributed simulation systems
> * relationship between metamodeling standards (e.g., MOF, Ecore) and
> distributed simulation standards (e.g., HLA, DIS)
> * metamodels for simulation reuse and interoperability
> * model-driven technologies for different simulation paradigms (discrete
> event simulation, multi-agent simulation, sketch-based * simulation, etc.)
> * model-driven methods and tools for performance engineering of simulation
> systems
> * simulation tools for model-driven software performance engineering
> * model-driven technologies for simulation verification and validation
> * model-driven technologies for data collection and analysis
> * model-driven technologies for simulation visualization
> * Executable UML
> * Executable Architectures
> * SysML / Modelica integration
> * Simulation Model Portability and reuse
> * model-based systems verification and validation
> * simulation for model-based systems engineering
>
> To stimulate creativity, however, the workshop maintains a wider scope and
> welcomes contribution

[Wiki-research-l] sustainability of conferences (was Re: Wiki Research Journal?)

[James Salsman]

>... WikiSym is changing is for the same reason.  People are
> not going to the conference!  I think the attendance has been below
> 100 for some time now.  That's not a sustainable number for the amount
> of work that goes into organizing a conference.

I would like to see an honest comparison of, for example, the reported
benefits of in-person conferences compared to their social and
economic costs. Meeting people in person is valuable, but I think it
happens more often than it needs to in most fields. Until people get
serious about organizing workflow around teleconferencing, huge and
expensive inefficiencies will persist. People love deductible junkets,
but where is the cost-benefit analysis?

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Access2research petition = bad idea

[James Salsman]

Dr. Jensen,

You ask who will pay for publication of journals under the open access model.

Closed access journals are supported primarily by university libraries
which pay subscription fees to publishers.  Very rarely do the
publishers pay anything to the editors and reviewers who produce the
journals, but they pocket a continuously increasing profit margin,
which has been increasing at about 1% per year, and currently stands
at about 27%, per
http://www.reedelsevier.com/mediacentre/pressreleases/2012/Pages/reed-elsevier-2011-results-announcement.aspx
In order to achieve such continually increasing profit margins,
publishers have been forcing price increases through bundling, which
is an abuse of their monopolistic market power which lack of
competition from alternative publishing models has allowed them to
attain.

Under the open access model, universities pay to support the
publication and printing of the journals, but do not pay subscription
fees.  Because there is no profit margin charged, these costs are less
to the university than commercial subscription fees, and the resulting
readership is not limited to a tiny fraction of the population.
(Because costs to the universities are less, they can keep more of the
money for university official perks and salaries, tax deductible
junkets for the faculty, and athletic salaries.  Sadly, universities
hardly ever pass any savings on to tuition payers.  Every subsidy and
loan guarantee supporting tuition in the postwar era has been matched
by tuition increases above the cost of living, sadly, while university
administrative official salaries have kept pace with CEO salaries
generally, exacerbating income inequality, and increases in faculty
salaries, perks, and expenses have also exceeded the inflation rate.)
As you point out, this situation often results in greater charges to
graduate students, unless their sponsors and grant investigators are
kind enough to include the journal production fees in their department
budget.  How often does that happen?

Your example of journals charging per-paper open access fees is an
example of subtle extortion in order to cause professors such as
yourself and other authors to take the position that you have, opposed
to open access.  Are there any reasons to the contrary?  Are there any
reasons that participation in such market manipulation schemes could
be seen as ethical?

Sincere regards,
James Salsman

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] [Foundation-l] Motivations to Contribute to Wikipedia

[James Salsman]

Yaroslav M. Blanter  wrote:

> actually, I am pretty sure we did discuss the procedure which requires
> endorsement (we did not call it approval), either at the extraordinary
> meeting in December (related to the survey banner story) or in the RCom
> mailing list in the thread related to the same story

The discussion in the RCom mailing list archives is fairly summarized by
http://lists.wikimedia.org/pipermail/rcom-l/2011-December/000506.html
"I rather suspect that there isn't consensus on this committee to restrict
researchers in their requesting community members to complete research
questionnaires."

The subsequent etherpad minutes for the December 22, 2011 meeting,
discuss a "future subject recruitment policy" which the participants
state they do not understand how to craft.

> do not call other people liars unless you have very strong proofs

Is there any way to interpret these two statements such that one of
them is not a lie?

"this is a policy that we're enforcing ... approval is required"
http://meta.wikimedia.org/w/index.php?title=Research_talk%3AFAQ&diff=3441309&oldid=3440848

"due to the lack of a formal policy, the RCom has never been in a
position to grant any kind of "definitive approval" to recruit
participants"
http://lists.wikimedia.org/pipermail/wiki-research-l/2012-March/001896.html

The first of those two statements was made in an effort to accuse me
of misconduct. I stand by my statements, and I am certain that I have
acted ethically.

It would be best if this issue were addressed as a mistake on the part
of those who have contradicted themselves.

Sincerely,
James Salsman

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] [Foundation-l] Motivations to Contribute to Wikipedia

[James Salsman]

Dario Taraborelli  wrote:

> James, I think I have replied consistently to your requests, both on wiki
> and by mail

Anyone can judge for themselves whether this is true by looking at
http://meta.wikimedia.org/wiki/Research_talk:FAQ

There you claimed that research approval was a mandatory policy, and
much more recently you thanked a third party for an edit which clearly
implies that it is strictly mandatory. But in September 2010 you
agreed with the rest of the RCom that research subject recruitment
approval should not be mandatory in favor of published guidelines
instead. And when called on the inconsistency, you wrote that approval
is not in fact mandatory. Both can not be true.

Instead of apologizing for your lie with which you attempted to impugn
my integrity, you have been trying to cover it up with rhetoric.

Is that behavior considered acceptable at the Wikimedia Foundation?

Sincerely,
James Salsman

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Motivations to Contribute to Wikipedia

[James Salsman]

Lane,

Thanks for your message:

> James: I made the edit stating the research should get approval,
> and I did that by jumping into the game and just making the edit
> based on what I read in discussion boards. I did not consider it
> to be a new requirement

For the benefit of those who haven't clicked on the link, you edited
[[meta:Research:Subject recruitment]] to read, at the top:

"If you are doing research which involves contacting Wikimedia project
editors or users then you must first notify the Wikimedia Research
Committee by describing your project. After your project gets approval
then you may begin."

How could that not be seen as a requirement?  Do you think there is a
way to phrase it so that it would not be seen as a requirement?

Certainly this is not your fault.  As you read, Dario Taraborelli
stated on February 15, "this is a policy that we're enforcing ...
approval is required"
http://meta.wikimedia.org/w/index.php?title=Research_talk%3AFAQ&diff=3441309&oldid=3440848

And after you made that edit, Dario thanked you for it, saying, "I
appreciate the documentation on the review procedure" -- even though
the Research Committee had explicitly rejected an approval policy
requirement in September 2010, has not discussed it since, and neither
the community or the Foundation has ever endorsed any of the earlier
policy proposals.

I would not be so upset about this if I hadn't been repeatedly accused
of misconduct in failing to obtain RCom approval.

Given the ease and lack of remorse with which Dr. Taraborelli, Mr.
Walling, and Mr. Beaudette have all repeatedly lied about me while
accusing me of misconduct, I have lost all confidence in the ability
of Foundation staff to adhere to basic ethics. I intend to continue to
raise this issue until it is addressed sufficiently.

Sincerely,
James Salsman

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Motivations to Contribute to Wikipedia

[James Salsman]

Dario Taraborelli  wrote:
>
>... due to the lack of a formal policy, the RCom has never been
> in a position to grant any kind of "definitive approval" to recruit
> participants

I appreciate that clarification, but it strictly contradicts this edit from
11 days ago:

http://meta.wikimedia.org/w/index.php?title=Research:Subject_recruitment&diff=3546474&oldid=2703471

about which Dario said, "I appreciate the documentation on the
review procedure" at
http://meta.wikimedia.org/wiki/Research_talk:Subject_recruitment

I think there are some very serious ethical issues here.  Requiring
Research Committee approval to contact editors or users was
explicitly rejected by the Research Committee:

http://meta.wikimedia.org/wiki/Research:Committee/Meetings/Meeting_2010-09-18/Log

As far as I can tell, the Research Committee has not discussed the
topic since.

I wonder what the community thinks of this new requirement.

Sincerely,
James Salsman

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Motivations to Contribute to Wikipedia

[James Salsman]

En Pine  wrote:
>
>... It is my understanding that Research Committee approval is
> required before soliciting Wikipedia subjects for surveys

Although Dario Taraborelli has claimed that such Committee approval is
required, he has refused to say where such a policy statement exists:

http://meta.wikimedia.org/wiki/Research_talk:FAQ

The only minutes I was able to find wherein the Research Committee
discussed the possibility of reviewing and approving research, doing
so was explicitly rejected, including by Dario, in favor of issuing
guidelines instead:

http://meta.wikimedia.org/wiki/Research:Committee/Meetings/Meeting_2010-09-18/Log

A Wikipedia Research Policy was announced in April 2010, but never
formally approved by the English Wikipedia community or the Foundation:

http://lists.wikimedia.org/pipermail/wiki-research-l/2010-April/000955.html

At one point it included a proposal for a "Subject Recruitment Approvals
Group" but that was removed in September 2010:

http://en.wikipedia.org/w/index.php?title=Wikipedia%3AResearch&action=historysubmit&diff=382382612&oldid=382338985

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] wikitrends

[James Salsman]

Ed,

Please consider asking Johan Gunnarsson 
to collaborate with you. He did something similar (with the same name)
in 2008 which you can see at: http://toolserver.org/~johang/wikitrends

Best regards,
James Salsman

> Date: Tue, 21 Feb 2012 12:01:28 -0500
> From: Ed Summers 
> To: Research into Wikimedia content and communities
>
> Subject: [Wiki-research-l] wikitrends
>
> I imagine something like this has already been done before, but I thought I
> would mention it as a curiosity:
>
> Wikitrends
> http://inkdroid.org/wikitrends/
>
> Wikitrends is a display of the top 25 view articles on English Wikipedia in
> the latest hour. It relies on stats that Wikimedia make available [1]. If
> you hover over the article you should get the article summary (courtesy of
> the MediaWiki API), and there are canned search links of realtime Google and
> Twitter and Facebook search if you want to look at what people might be
> saying about the topic.
>
> I put the code up on Github [2] and wrote a brief blog entry about the
> process of putting the app together. The punchline that I was trying to work
> up to is that it is truly wonderful that Wikimedia makes an effort to make
> its data assets available on the Web, both via an API and as bulk downloads.
> It is a great role model for other organizations and institutions.
>
> Thanks!
> //Ed
>
> [1] http://dumps.wikimedia.org/other/pagecounts-raw/
> [2] http://inkdroid.org/edsu/wikitrends/
> [3] http://inkdroid.org/journal/2012/02/21/nodb/

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Re: [Wiki-research-l] Contact details

[James Salsman]

Dario,

I have replied at http://meta.wikimedia.org/wiki/Research_talk:FAQ

Sincerely,
James Salsman


On Tue, Feb 14, 2012 at 3:55 PM, Dario Taraborelli
 wrote:
> James,
>
> I replied here so other RCom members can chime in  (rcom-l is publicly 
> indexed but members-only)
>
> http://meta.wikimedia.org/wiki/Research_talk:FAQ
>
>> That seems to be for people who are unable to contact subjects without help.
>
> no, that's only part of the reason why we set up these instructions, we 
> certainly want to support but also control who runs surveys (and how and what 
> for)
>
>> I have looked through
>> http://meta.wikimedia.org/wiki/Notes_on_good_practices_on_Wikipedia_research
>> -- and as far as I can tell, that is the closest thing to a research
>> policy which exists.  Do you think there was some way which I had not
>> adhered to it?
>
> that page unfortunately predates the creation of the RCom and the standard 
> review procedure that we introduced for SR (I added a note to the talk page 
> to mark it as obsolete for this reason). I appreciate that many people may 
> have never heard about the new SR process and that's ok, you are not the 
> first person to be blocked and invited to file a proposal for RCom review.
>
> Hope this helps, let me know if you have any question
>
> Dario

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Fwd: [RCom-l] Fwd: Actual Inactive Wikipedia administrator survey (swalling at wikimedia.org)

[James Salsman]

-- Forwarded message --
From: James Salsman 
Date: Tue, Feb 14, 2012 at 12:19 AM
Subject: Re: [RCom-l] Fwd: Actual Inactive Wikipedia administrator
survey (swalling at wikimedia.org)
To: rco...@lists.wikimedia.org


On 12 February 2012, Steven Walling wrote:

> I did not receive any prior contact from Salsman about this before
> he began to send emails out.

That is blatantly untrue. See this IRC Office Hours log from
http://meta.wikimedia.org/wiki/IRC_office_hours/Office_hours_2012-02-10

[09:41am] jsalsman okay, well I guess the first thing I need is to
know who in Zack's department will be point of contact for editor
recruitment efforts
[09:41am] Philippe jsalsman: that hasn't changed.
[09:41am] jsalsman who then?
[09:41am] StevenW jsalsman: you can talk to me and Maryana
[09:41am] jsalsman okay
...
[09:59am] jsalsman StevenW: I'm going to go ahead with the three-year
old inactive admins survey and send you access to the results
spreadsheet


On 13 February 2012, Philippe Beaudette wrote at
https://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard#Heads_up:_unapproved_survey_of_inactive_administrators

> I was unaware of distribution mechanism and was certainly unaware that it 
> would
> list a Foundation staff member as the contact.

That is also blatantly untrue, as is clear from the plain language of
the IRC log above, as well as
http://meta.wikimedia.org/w/index.php?title=Legal_and_Community_Advocacy/Community_Advocacy&oldid=3422700#Supporting_the_Wikimedia_community


On 14 February 2012, Dario Taraborelli wrote at
http://meta.wikimedia.org/wiki/Talk:Inactive_administrators_survey#Lack_of_RCom_review

> This survey has not been reviewed by the Research Committee and
> as such it's in violation of our Research:Subject recruitment.

The page linked to there says, "Until an official policy is approved
by the Wikimedia Research Committee regarding subject recruitment,
individual requests can be submitted following these instructions."
The only Wikipedia Research Policy was announced at
http://lists.wikimedia.org/pipermail/wiki-research-l/2010-April/000955.html
and all of the provisions for subject recruitment approvals were
removed from that policy as described in
http://lists.wikimedia.org/pipermail/wiki-research-l/2012-February/001839.html


Moreover, Steven Walling has stated twice so far that I continued
sending email surveys after he asked me to stop.  That is easily
disproven.  I stopped sending them four hours before he asked me to
stop.

As for the survey, it's been an enormous success, with several
formerly inactive admins returning to editing so far, and profoundly
helpful data for resolving issues surrounding admin and editor
attrition.

It is abundantly clear from
http://strategy.wikimedia.org/wiki/Thread:Talk:Task_force/Community_Health/Former_administrators_survey/Status_update%3F/reply_(3)
that this survey had been approved in September 2010 but never acted
on. I brought this up repeatedly in the years since and was ignored.
It took me three days to administer the survey.

The intentional lies about my conduct are *not* ethical, and I am owed
an apology.

Sincerely,
James Salsman

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

[Wiki-research-l] Wikipedia Research Policy

[James Salsman]

After careful review of the wiki-research-l archives, I have found that the
original Wikipedia Research Policy, which was announced in April 2010:

http://lists.wikimedia.org/pipermail/wiki-research-l/2010-April/000955.html

-- at http://en.wikipedia.org/wiki/Wikipedia:Research -- was modified in 2010
to remove mention of the "Subject Recruitment Approvals Group" (SRAG):

http://en.wikipedia.org/w/index.php?title=Wikipedia%3AResearch&action=historysubmit&diff=382382612&oldid=382338985

Presumably this was done because technical features to support the SRAG
had not been, and are still not implemented:

http://en.wikipedia.org/w/index.php?title=Wikipedia:Research&diff=382263699&oldid=382076735

If I am mistaken about this conclusion, please let me know immediately.

Sincerely,
James Salsman

___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l