Re: [Wiki-research-l] effects of vandalism and abuse on editors and readers

[Aaron Halfaker]

See page 7 of Priedhorsky, R., Chen, J., Lam, S. T. K., Panciera, K.,
Terveen, L., & Riedl, J. (2007, November). Creating, destroying, and
restoring value in Wikipedia. In *Proceedings of the 2007 international ACM
conference on Supporting group work* (pp. 259-268).
http://reidster.net/pubs/group282-priedhorsky.pdf

They discuss the probability of a page view of Wikipedia containing
vandalism rising over time.  I wanted to replicate this analysis and extend
it past 2007 but I never got the chance.  I think the methodology is really
interesting though.

It doesn't directly answer the question but it does get at the *impact* of
vandalism.

On Mon, Jan 18, 2021 at 12:13 PM Isaac Johnson  wrote:

> To WSC's point about the difficulty of detecting such behavior or surveying
> at a point in which it would still be salient, I'd add that in general we
> have a large gap in our knowledge about why people choose to stop editing
> because almost all of our survey mechanisms depend on existing logged-in
> usage of the wikis. This is a challenge with many other websites too but
> it's generally easier to find and survey who, for instance, has left
> Facebook (example
> <
> http://socialmedia.soc.northwestern.edu/wp-content/uploads/2013/05/CHI2013-FBLL.pdf
> >)
> by collecting a random sample of people than it is to find and survey
> someone who was a former editor of Wikipedia. There were surveys that did
> ask about major barriers to editing (which presumably contribute to
> burnout) such as the 2012 survey:
>
> https://upload.wikimedia.org/wikipedia/commons/8/81/Editor_Survey_2012_-_Wikipedia_editing_experience.pdf#page=17
> (see the editor survey category
>  if you're
> looking
> for others)
>
> Some things that come to mind though:
>
>- I suspect very few readers see vandalism in their daily browsing (as a
>very frequent, long-term reader of English Wikipedia, I have trouble
>recalling encountering any clear vandalism in the course of normal
>reading). That said, I do suspect that most people have seen plenty of
>stories of outlandish vandalism to Wikipedia -- some legitimate but many
>more about vandalism that literally lasted minutes -- that may lead to
>lower trust. Whether or not lower trust in Wikipedia leads to lower
>readership is a separate question though. Jonathan Morgan ran some
> recent
>surveys on reader trust and what factors affected it that might be
>relevant:
>
> https://meta.wikimedia.org/wiki/Research:The_role_of_citations_in_how_readers_evaluate_Wikipedia_articles#Second_round_survey
>- Specifically in the context of harassment and gender equity:
>   - Harassment as barrier:
>
> https://meta.wikimedia.org/wiki/Gender_equity_report_2018/Barriers_to_equity
>   - Edit summaries in particular as harassment:
>   https://www.elizabethwhittaker.net/wmf-internship (more details
>   <
> https://www.mediawiki.org/wiki/Wikimedia_Research/Showcase#July_2019>
>   )
>   - Annual Community Insights Reports often have a section on this --
>   e.g.,
>
> https://meta.wikimedia.org/wiki/Community_Insights/Community_Insights_2020_Report/Thriving_Movement#Safe_and_Secure_Spaces
>   - 2015 Harassment Survey:
>   https://meta.wikimedia.org/wiki/Research:Harassment_survey_2015
>- The body of work around barriers to newcomers might have some good
>insights too -- e.g.,
>
> https://www-users.cs.umn.edu/~halfaker/publications/The_Rise_and_Decline/
>
>
> On Sun, Jan 17, 2021 at 5:44 AM WereSpielChequers <
> werespielchequ...@gmail.com> wrote:
>
> > Hi Amir,
> >
> > This is one of those areas of research where we really need the annual
> > editor survey. I think it ran once after the 2009/10 Strategy process,
> and
> > I don't know if the best questions got included.
> >
> > But the best  time to ask editors what prompted them to  start editing
> has
> > to be fairly soon after they started as memories fade. I once went back
> to
> > my early edits and the edit I remembered starting me editing barely made
> it
> > into my first 50.
> >
> > There is a longstanding theory that a lot of new editors start or started
> > to fix some vandalism that they saw, and that this group went into steep
> > decline a decade ago with the rise of Cluebot and other antivandalism
> tools
> > that work faster than a newbie could. But without an annual survey to ask
> > editors what prompted them to edit you are going to struggle to research
> > this. Of course you could look at the early logged in edits of
> > active/prolific wikipedians, but if it is true that many/most Wikipedians
> > start with some IP edits, the earliest edits of many Wikipedians won't be
> > available.
> >
> > Abuse one assumes has a differential effect on the targets of abuse,
> > disproportionately women, gays and ethnic minorities. But I'd be inclined
> > to look at stuff targeted at their user and usertalkpages rather than
> >

Re: [Wiki-research-l] effects of vandalism and abuse on editors and readers

[Isaac Johnson]

To WSC's point about the difficulty of detecting such behavior or surveying
at a point in which it would still be salient, I'd add that in general we
have a large gap in our knowledge about why people choose to stop editing
because almost all of our survey mechanisms depend on existing logged-in
usage of the wikis. This is a challenge with many other websites too but
it's generally easier to find and survey who, for instance, has left
Facebook (example
)
by collecting a random sample of people than it is to find and survey
someone who was a former editor of Wikipedia. There were surveys that did
ask about major barriers to editing (which presumably contribute to
burnout) such as the 2012 survey:
https://upload.wikimedia.org/wikipedia/commons/8/81/Editor_Survey_2012_-_Wikipedia_editing_experience.pdf#page=17
(see the editor survey category
 if you're looking
for others)

Some things that come to mind though:

   - I suspect very few readers see vandalism in their daily browsing (as a
   very frequent, long-term reader of English Wikipedia, I have trouble
   recalling encountering any clear vandalism in the course of normal
   reading). That said, I do suspect that most people have seen plenty of
   stories of outlandish vandalism to Wikipedia -- some legitimate but many
   more about vandalism that literally lasted minutes -- that may lead to
   lower trust. Whether or not lower trust in Wikipedia leads to lower
   readership is a separate question though. Jonathan Morgan ran some recent
   surveys on reader trust and what factors affected it that might be
   relevant:
   
https://meta.wikimedia.org/wiki/Research:The_role_of_citations_in_how_readers_evaluate_Wikipedia_articles#Second_round_survey
   - Specifically in the context of harassment and gender equity:
  - Harassment as barrier:
  
https://meta.wikimedia.org/wiki/Gender_equity_report_2018/Barriers_to_equity
  - Edit summaries in particular as harassment:
  https://www.elizabethwhittaker.net/wmf-internship (more details
  
  )
  - Annual Community Insights Reports often have a section on this --
  e.g.,
  
https://meta.wikimedia.org/wiki/Community_Insights/Community_Insights_2020_Report/Thriving_Movement#Safe_and_Secure_Spaces
  - 2015 Harassment Survey:
  https://meta.wikimedia.org/wiki/Research:Harassment_survey_2015
   - The body of work around barriers to newcomers might have some good
   insights too -- e.g.,
   https://www-users.cs.umn.edu/~halfaker/publications/The_Rise_and_Decline/


On Sun, Jan 17, 2021 at 5:44 AM WereSpielChequers <
werespielchequ...@gmail.com> wrote:

> Hi Amir,
>
> This is one of those areas of research where we really need the annual
> editor survey. I think it ran once after the 2009/10 Strategy process, and
> I don't know if the best questions got included.
>
> But the best  time to ask editors what prompted them to  start editing has
> to be fairly soon after they started as memories fade. I once went back to
> my early edits and the edit I remembered starting me editing barely made it
> into my first 50.
>
> There is a longstanding theory that a lot of new editors start or started
> to fix some vandalism that they saw, and that this group went into steep
> decline a decade ago with the rise of Cluebot and other antivandalism tools
> that work faster than a newbie could. But without an annual survey to ask
> editors what prompted them to edit you are going to struggle to research
> this. Of course you could look at the early logged in edits of
> active/prolific wikipedians, but if it is true that many/most Wikipedians
> start with some IP edits, the earliest edits of many Wikipedians won't be
> available.
>
> Abuse one assumes has a differential effect on the targets of abuse,
> disproportionately women, gays and ethnic minorities. But I'd be inclined
> to look at stuff targeted at their user and usertalkpages rather than
> talkpages and edit summaries, though an email survey of former editors
> would be useful.
>
> My suspicion is that when we revert, block and maybe even revdel or
> oversight abuse we assume that fixes the problem, and if we want to tackle
> abuse we need more edit filters to prevent such abuse from going live.
>
> WSC
>
> On Sat, 16 Jan 2021 at 15:16, Amir E. Aharoni <
> amir.ahar...@mail.huji.ac.il>
> wrote:
>
> > Hi,
> >
> > Is there any research about the effect of vandalism in wiki content pages
> > on readers, experienced editors, and new and potential editors?
> >
> > And of abuse in discussion pages and edit summaries on experienced
> editors
> > and new and potential editors?
> >
> > Intuitively and anecdotally one could think of the following:
> > 1. Vandalism in content pages (articles) wastes editors' and patrollers'
> > time. This (pr