Re: [Wiki-research-l] effects of vandalism and abuse on editors and readers

[Aaron Halfaker]

+1 WSC.   When I thought about replicating it, I expected to see a dramatic
decline in the impact of vandalism with the advent of counter-vandalism
tools and abuse filter.

It would be interesting to see that on a cross-wiki basis as different
wikis employ different strategies (or seemingly none at all) for
counter-vandalism over time.

On Tue, Jan 19, 2021 at 10:58 AM WereSpielChequers <
werespielchequ...@gmail.com> wrote:

> Hi Aaron,
>
> That was an interesting read and a bit of a time capsule. 2002-2006 is a
> bit before I started editing Wikipedia. Before many of the tools such as
> huggle that give vandalfighters such an advantage over vandals, I think
> before the era of bot reversion of vandalism when vandalism had to be
> reverted by humans rather than computers, and certainly before the edit
> filters that prevent much, possibly most vandalism from even being saved.
> It also seems to predate the whole panoply of page protection that stops
> vandals even editing many common vandalism targets (they do say that every
> single article is available for anyone to edit).
>
> It would be interesting to see a study now when recent changes patrollers
> boast of the times they have got to some vandalism faster than Cluebot.
>
> I know there were predictions in the early years that eventually the tidal
> wave of vandalism would overwhelm the defenders of the wiki, that study
> seems to have been part of that. I wonder if anyone in 2004 predicted that
> we would get to the current situation where adolescent vandalism has turned
> out to be so predictable that dealing with it has been mostly automated and
> now we are more worried about spam than vandalism.
>
> WSC
>
> On Mon, 18 Jan 2021 at 23:52, Aaron Halfaker 
> wrote:
>
> > See page 7 of Priedhorsky, R., Chen, J., Lam, S. T. K., Panciera, K.,
> > Terveen, L., & Riedl, J. (2007, November). Creating, destroying, and
> > restoring value in Wikipedia. In *Proceedings of the 2007 international
> ACM
> > conference on Supporting group work* (pp. 259-268).
> > http://reidster.net/pubs/group282-priedhorsky.pdf
> >
> > They discuss the probability of a page view of Wikipedia containing
> > vandalism rising over time.  I wanted to replicate this analysis and
> extend
> > it past 2007 but I never got the chance.  I think the methodology is
> really
> > interesting though.
> >
> > It doesn't directly answer the question but it does get at the *impact*
> of
> > vandalism.
> >
> > On Mon, Jan 18, 2021 at 12:13 PM Isaac Johnson 
> > wrote:
> >
> > > To WSC's point about the difficulty of detecting such behavior or
> > surveying
> > > at a point in which it would still be salient, I'd add that in general
> we
> > > have a large gap in our knowledge about why people choose to stop
> editing
> > > because almost all of our survey mechanisms depend on existing
> logged-in
> > > usage of the wikis. This is a challenge with many other websites too
> but
> > > it's generally easier to find and survey who, for instance, has left
> > > Facebook (example
> > > <
> > >
> >
> http://socialmedia.soc.northwestern.edu/wp-content/uploads/2013/05/CHI2013-FBLL.pdf
> > > >)
> > > by collecting a random sample of people than it is to find and survey
> > > someone who was a former editor of Wikipedia. There were surveys that
> did
> > > ask about major barriers to editing (which presumably contribute to
> > > burnout) such as the 2012 survey:
> > >
> > >
> >
> https://upload.wikimedia.org/wikipedia/commons/8/81/Editor_Survey_2012_-_Wikipedia_editing_experience.pdf#page=17
> > > (see the editor survey category
> > >  if you're
> > > looking
> > > for others)
> > >
> > > Some things that come to mind though:
> > >
> > >- I suspect very few readers see vandalism in their daily browsing
> > (as a
> > >very frequent, long-term reader of English Wikipedia, I have trouble
> > >recalling encountering any clear vandalism in the course of normal
> > >reading). That said, I do suspect that most people have seen plenty
> of
> > >stories of outlandish vandalism to Wikipedia -- some legitimate but
> > many
> > >more about vandalism that literally lasted minutes -- that may lead
> to
> > >lower trust. Whether or not lower trust in Wikipedia leads to lower
> > >readership is a separate question though. Jonathan Morgan ran some
> > > recent
> > >surveys on reader trust and what factors affected it that might be
> > >relevant:
> > >
> > >
> >
> https://meta.wikimedia.org/wiki/Research:The_role_of_citations_in_how_readers_evaluate_Wikipedia_articles#Second_round_survey
> > >- Specifically in the context of harassment and gender equity:
> > >   - Harassment as barrier:
> > >
> > >
> >
> https://meta.wikimedia.org/wiki/Gender_equity_report_2018/Barriers_to_equity
> > >   - Edit summaries in particular as harassment:
> > >   https://www.elizabethwhittaker.net/wmf-internship (more details
> > >   

Re: [Wiki-research-l] effects of vandalism and abuse on editors and readers

[WereSpielChequers]

Hi Aaron,

That was an interesting read and a bit of a time capsule. 2002-2006 is a
bit before I started editing Wikipedia. Before many of the tools such as
huggle that give vandalfighters such an advantage over vandals, I think
before the era of bot reversion of vandalism when vandalism had to be
reverted by humans rather than computers, and certainly before the edit
filters that prevent much, possibly most vandalism from even being saved.
It also seems to predate the whole panoply of page protection that stops
vandals even editing many common vandalism targets (they do say that every
single article is available for anyone to edit).

It would be interesting to see a study now when recent changes patrollers
boast of the times they have got to some vandalism faster than Cluebot.

I know there were predictions in the early years that eventually the tidal
wave of vandalism would overwhelm the defenders of the wiki, that study
seems to have been part of that. I wonder if anyone in 2004 predicted that
we would get to the current situation where adolescent vandalism has turned
out to be so predictable that dealing with it has been mostly automated and
now we are more worried about spam than vandalism.

WSC

On Mon, 18 Jan 2021 at 23:52, Aaron Halfaker 
wrote:

> See page 7 of Priedhorsky, R., Chen, J., Lam, S. T. K., Panciera, K.,
> Terveen, L., & Riedl, J. (2007, November). Creating, destroying, and
> restoring value in Wikipedia. In *Proceedings of the 2007 international ACM
> conference on Supporting group work* (pp. 259-268).
> http://reidster.net/pubs/group282-priedhorsky.pdf
>
> They discuss the probability of a page view of Wikipedia containing
> vandalism rising over time.  I wanted to replicate this analysis and extend
> it past 2007 but I never got the chance.  I think the methodology is really
> interesting though.
>
> It doesn't directly answer the question but it does get at the *impact* of
> vandalism.
>
> On Mon, Jan 18, 2021 at 12:13 PM Isaac Johnson 
> wrote:
>
> > To WSC's point about the difficulty of detecting such behavior or
> surveying
> > at a point in which it would still be salient, I'd add that in general we
> > have a large gap in our knowledge about why people choose to stop editing
> > because almost all of our survey mechanisms depend on existing logged-in
> > usage of the wikis. This is a challenge with many other websites too but
> > it's generally easier to find and survey who, for instance, has left
> > Facebook (example
> > <
> >
> http://socialmedia.soc.northwestern.edu/wp-content/uploads/2013/05/CHI2013-FBLL.pdf
> > >)
> > by collecting a random sample of people than it is to find and survey
> > someone who was a former editor of Wikipedia. There were surveys that did
> > ask about major barriers to editing (which presumably contribute to
> > burnout) such as the 2012 survey:
> >
> >
> https://upload.wikimedia.org/wikipedia/commons/8/81/Editor_Survey_2012_-_Wikipedia_editing_experience.pdf#page=17
> > (see the editor survey category
> >  if you're
> > looking
> > for others)
> >
> > Some things that come to mind though:
> >
> >- I suspect very few readers see vandalism in their daily browsing
> (as a
> >very frequent, long-term reader of English Wikipedia, I have trouble
> >recalling encountering any clear vandalism in the course of normal
> >reading). That said, I do suspect that most people have seen plenty of
> >stories of outlandish vandalism to Wikipedia -- some legitimate but
> many
> >more about vandalism that literally lasted minutes -- that may lead to
> >lower trust. Whether or not lower trust in Wikipedia leads to lower
> >readership is a separate question though. Jonathan Morgan ran some
> > recent
> >surveys on reader trust and what factors affected it that might be
> >relevant:
> >
> >
> https://meta.wikimedia.org/wiki/Research:The_role_of_citations_in_how_readers_evaluate_Wikipedia_articles#Second_round_survey
> >- Specifically in the context of harassment and gender equity:
> >   - Harassment as barrier:
> >
> >
> https://meta.wikimedia.org/wiki/Gender_equity_report_2018/Barriers_to_equity
> >   - Edit summaries in particular as harassment:
> >   https://www.elizabethwhittaker.net/wmf-internship (more details
> >   <
> > https://www.mediawiki.org/wiki/Wikimedia_Research/Showcase#July_2019>
> >   )
> >   - Annual Community Insights Reports often have a section on this --
> >   e.g.,
> >
> >
> https://meta.wikimedia.org/wiki/Community_Insights/Community_Insights_2020_Report/Thriving_Movement#Safe_and_Secure_Spaces
> >   - 2015 Harassment Survey:
> >   https://meta.wikimedia.org/wiki/Research:Harassment_survey_2015
> >- The body of work around barriers to newcomers might have some good
> >insights too -- e.g.,
> >
> >
> https://www-users.cs.umn.edu/~halfaker/publications/The_Rise_and_Decline/
> >
> >
> > On Sun, Jan 

Re: [Wiki-research-l] effects of vandalism and abuse on editors and readers

[WereSpielChequers]

I'm sure there has been a survey of former editors done using the email
this user function (as I remember it one of the more common responses was I
haven't left yet). However this would not be a great way to survey re
harassment as harassed people are more likely to close an email account or
disable the email future.

As for how many readers saw vandalism in the era before edit filters etc,
it didn't need to be many readers who saw it, and many of those to remove
it for this to be an important way to recruit editors. We have such a huge
imbalance between readers and editors that even if only 1% of readers saw
vandalism and only 1% of those fixed it, that would still be an extra
hundred editors for every million readers.

WSC

On Mon, 18 Jan 2021 at 20:13, Isaac Johnson  wrote:

> To WSC's point about the difficulty of detecting such behavior or surveying
> at a point in which it would still be salient, I'd add that in general we
> have a large gap in our knowledge about why people choose to stop editing
> because almost all of our survey mechanisms depend on existing logged-in
> usage of the wikis. This is a challenge with many other websites too but
> it's generally easier to find and survey who, for instance, has left
> Facebook (example
> <
> http://socialmedia.soc.northwestern.edu/wp-content/uploads/2013/05/CHI2013-FBLL.pdf
> >)
> by collecting a random sample of people than it is to find and survey
> someone who was a former editor of Wikipedia. There were surveys that did
> ask about major barriers to editing (which presumably contribute to
> burnout) such as the 2012 survey:
>
> https://upload.wikimedia.org/wikipedia/commons/8/81/Editor_Survey_2012_-_Wikipedia_editing_experience.pdf#page=17
> (see the editor survey category
>  if you're
> looking
> for others)
>
> Some things that come to mind though:
>
>- I suspect very few readers see vandalism in their daily browsing (as a
>very frequent, long-term reader of English Wikipedia, I have trouble
>recalling encountering any clear vandalism in the course of normal
>reading). That said, I do suspect that most people have seen plenty of
>stories of outlandish vandalism to Wikipedia -- some legitimate but many
>more about vandalism that literally lasted minutes -- that may lead to
>lower trust. Whether or not lower trust in Wikipedia leads to lower
>readership is a separate question though. Jonathan Morgan ran some
> recent
>surveys on reader trust and what factors affected it that might be
>relevant:
>
> https://meta.wikimedia.org/wiki/Research:The_role_of_citations_in_how_readers_evaluate_Wikipedia_articles#Second_round_survey
>- Specifically in the context of harassment and gender equity:
>   - Harassment as barrier:
>
> https://meta.wikimedia.org/wiki/Gender_equity_report_2018/Barriers_to_equity
>   - Edit summaries in particular as harassment:
>   https://www.elizabethwhittaker.net/wmf-internship (more details
>   <
> https://www.mediawiki.org/wiki/Wikimedia_Research/Showcase#July_2019>
>   )
>   - Annual Community Insights Reports often have a section on this --
>   e.g.,
>
> https://meta.wikimedia.org/wiki/Community_Insights/Community_Insights_2020_Report/Thriving_Movement#Safe_and_Secure_Spaces
>   - 2015 Harassment Survey:
>   https://meta.wikimedia.org/wiki/Research:Harassment_survey_2015
>- The body of work around barriers to newcomers might have some good
>insights too -- e.g.,
>
> https://www-users.cs.umn.edu/~halfaker/publications/The_Rise_and_Decline/
>
>
> On Sun, Jan 17, 2021 at 5:44 AM WereSpielChequers <
> werespielchequ...@gmail.com> wrote:
>
> > Hi Amir,
> >
> > This is one of those areas of research where we really need the annual
> > editor survey. I think it ran once after the 2009/10 Strategy process,
> and
> > I don't know if the best questions got included.
> >
> > But the best  time to ask editors what prompted them to  start editing
> has
> > to be fairly soon after they started as memories fade. I once went back
> to
> > my early edits and the edit I remembered starting me editing barely made
> it
> > into my first 50.
> >
> > There is a longstanding theory that a lot of new editors start or started
> > to fix some vandalism that they saw, and that this group went into steep
> > decline a decade ago with the rise of Cluebot and other antivandalism
> tools
> > that work faster than a newbie could. But without an annual survey to ask
> > editors what prompted them to edit you are going to struggle to research
> > this. Of course you could look at the early logged in edits of
> > active/prolific wikipedians, but if it is true that many/most Wikipedians
> > start with some IP edits, the earliest edits of many Wikipedians won't be
> > available.
> >
> > Abuse one assumes has a differential effect on the targets of abuse,
> > disproportionately women, gays and ethnic minorities. But I'd 

[Wiki-research-l] [Wikimedia Research Showcase] January 20, 2021: Macro-level organizational analysis of peer production communities

[Martin Gerlach]

Hi all,
The next Research Showcase will be live-streamed on Wednesday, January 20,
at 9:30 AM PST/17:30 UTC.  In this month’s showcase, Aaron Shaw will
present ongoing research illustrating the values and challenges of
macro-level organizational analysis of peer production and social computing
systems. Specifically, he will give an overview on different studies
showing convergent trends of formalization in large Wikipedias; divergent
editor engagement in small Wikipedias; and commensal patterns of ecological
interdependence across communities.

Youtube stream: 
https://www.youtube.com/watch?v=ujd8S82YfmA

As usual, you can join the conversation on IRC at #wikimedia-research. You
can also watch our past research showcases here:
https://www.mediawiki.org/wiki/Wikimedia_Research/Showcase



*Speaker*:
Aaron Shaw (Northwestern University)
*Title*: The importance of thinking big. Convergence, divergence, and
interdependence among wikis and peer production communities
*Abstract*: Designing and governing collaborative, peer production
communities can benefit from large-scale, macro-level thinking that focuses
on communities as the units of analysis. For example, understanding how and
why seemingly comparable communities may follow convergent, divergent,
and/or interdependent patterns of behavior can inform more parsimonious
theoretical and empirical insights as well as more effective strategic
action. This talk gives a sneak peak at research-in-progress by members of
the Community Data Science Collective  to
illustrate these points. In particular, I focus on studies of (1)
convergent trends of formalization in several large Wikipedias; (2)
divergent editor engagement among three small Wikipedias; and (3) commensal
patterns of ecological interdependence across communities. Together, the
studies underscore the value and challenges of macro-level organizational
analysis of peer production and social computing systems.


-- 
Martin Gerlach
Research Scientist
Wikimedia Foundation
___
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l