<quote who="Pine W" date="Thu, Sep 18, 2014 at 01:49:13PM -0700"> > Yes, but supposedly phone survey companies are able to get > representative samples of broad populations despite many people > refusing to respond to phone surveys. If opt-in users were chosen > using similar methods, could arguably representative data be > obtained?
The way that people build representative surveys from non-representative data is by understanding quite a lot about the nature and structure of the bias in your sample. You might want to think about how people do this as trying to create a very complicated system of weights. Folks who do this for US phone surveys, for example, have spent many decades and many millions of dollars on research to understand how to get reliable results and even then it's a quickly moving target. They still routinely sometimes miss things and get things wrong. That said, there are certainly things we can learn. Aaron Shaw and I actually did something related with one of the big Wikipedia surveys in this article: http://www.plosone.org/article/info:doi/10.1371/journal.pone.0065782 In our case, our study was only possible because (a) we had very good luck finding "ground truth" data from the right point in time, (b) we had detailed demographic data on folks from the WP survey, and (c) we make a series of untestable assumptions. After all that work, we still can't know that we've got it right. We really can only suggest that there are reasons to believe our estimates are better that pretending that the opt-in survey is unbiased. In the case of signing up for a Wikipedia toolbar, we might not even attract a sub-population that even /can/ reliably used to build representative estimates. :-( Regards, Mako -- Benjamin Mako Hill http://mako.cc/ Creativity can be a social contribution, but only in so far as society is free to use the results. --GNU Manifesto
signature.asc
Description: Digital signature
_______________________________________________ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
