On Thu, Aug 27, 2009 at 8:24 PM, Thomas Dalton<thomas.dal...@gmail.com> wrote: > 2009/8/28 Anthony <wikim...@inbox.org>: >> On Thu, Aug 27, 2009 at 7:58 PM, Stephen Bain <stephen.b...@gmail.com>wrote: >>> On Fri, Aug 28, 2009 at 4:58 AM, Anthony<wikim...@inbox.org> wrote: >>> > It seems to me to be begging the question. You don't answer the question >>> > "how bad is vandalism" by assuming that vandalism is generally reverted. >>> Can you suggest a better metric then? >> I must admit I don't understand the question. > > He means what would you measure in order to draw conclusions about the > severity of vandalism.
The obvious methodology would be to take a large random sample and hand classify it. It's not rocket science. By having multiple people perform the classification you could measure the confidence of the classification. This is somewhat labor intensive, but only somewhat as it doesn't take an inordinate number of samples to produce representative results. This should be the gold standard for this kind of measurement as it would be much closer to what people actually want to know than most machine metrics. If the results of this kind of study have good agreement with mechanical proxy metrics (such as machine detected vandalism) our confidence in those proxies will increase, if they disagree it will provide an opportunity to improve the proxies. These are techniques widely used in other fields. _______________________________________________ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
