Hello Aaron and Other Wiki Researchers,
Thank you for responding so quickly and thoroughly to my recent proposal! Many
of your concerns align with issues I’ve been discussing with my research team,
so I’m glad to hear that we’re overlapping in that sense. Apologies in advance
for the length of the following:
- - Sampling: I completely agree with your concerns in response to the
(relatively) recent revisit to the original Gender Gap results. As an
exploratory study, I don’t think we could accurately represent the entire
Wikipedia community or make causal inferences about the community as a whole
due to the voluntary nature of the survey and the potential for inaccuracies in
self-reporting. However, I’m hoping that this preliminary project could reveal
a few new patterns that might be explored in greater depth at a later date.
Based on the Wikipedia editor rankings, I’d planned to pull the top 20% of
editors and post on their Talk Pages, giving us the “super-editor” sample.
Since the two remaining samples are more difficult to recruit, I’m currently
exploring the most effective way to obtain a randomized sample of the active
(moderate) and inactive editors (infrequent edits) – this will likely be
developed with the assistance of someone more skilled in programming than
myself. I’ve also been speaking with a statistician about alternative methods,
beyond propensity-matching, where we might account for response biases that are
likely to occur. However, I’d be very open to suggestions from this community
about effectively sampling from Wikipedia and methods you’ve used to account
for biases common in these surveys.
- - Self-Report Measures of Edit History: This would only serve to
verify the editor ranking and provide a more thorough context by which the
editor feels he/she makes contributions to the Wikipedia community. Since
we’ll have usernames – via Talk Pages – as you suggested, I’d like to explore
actual editing behaviors given that we’d have the resources to do so.
- - Collaboration: Participant fatigue is a huge concern with all of
these online surveys targeting active editors. I believe you’re correct that
the WMF is planning another editor survey, but I had hoped to provide some
foundation for other themes that might be explored in these larger surveys.
The prior WMF surveys didn’t provide as much depth as we might need to reveal
any patterns in editing behaviors. I’ve also reached out to a couple of other
proposals, with similar interests, to determine whether we can compliment each
other’s efforts. I think these types of collaborations are very do-able and
may help us to limit the frequency of Wikipedia editor surveys.
- Missing Measures and People: I was able to access your article, so
thank you for linking it! I’ve been reviewing the literature to clarify
variables (such as the web use you identify) to determine which should be
included in the survey. In order to keep the survey at a reasonable length,
I’d hoped to capture some of these editing barriers via themes captured in the
open-ended responses. This might be particularly relevant in the context of
editors’ perceived barriers, which might vary based on the aforementioned
traits. However, I agree that the study would likely benefit form some further
questioning about editing experiences and I’ll be adding this into the proposal.
- Missing People and Sampling: Your main concern also parallels the
concerns of my research team. I’ve been speaking with my team about
potentially recruiting a passive Wikipedia user sample that would serve as a
comparison. It was my original hope that a small incentive would encourage
even the infrequent editors to complete the survey measure, but in the event
that they don’t we’ll need that comparison group. Our greatest barrier would
be matching the “pertinent” comparison sample characteristics with our
super-editors. I’m not sure that we can achieve this yet, but more to come as
I explore this option.
Thank you again Aaron for your thorough feedback! As I’ve been following this
listserv, I’m incredibly grateful that we have developed such a strong
research-oriented Wikipedia community.
Sincerely,
Christina
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
