Regaring measuerment of editor retention - this is tricky - as in fact many participant created new accounts only to join the contest. Some of them had accounts on Wikipedia (but different) - some others - abandoned their accounts and created a new ones for various reasons (the most trival - they have forgoten passwords). There are also user who are active only during contensts - also for various reasons - not only due to possibility to win attractive prizes, but also because the normal upload process is too tricky for them, or they don't know what to photograph if there is no easy to use list of objects.
In fact measurement of editor retention is tricky even for workshops if it is only based on list of nicknames. I saw this many times - that people create the accounts during the workshop and then abandon them, but create later a new ones. The only effective way to follow the retention of users after workshop is to collect their e-mails and then survey them some time after the workshop. It might produce completely different picture that studies based on following the activity of accounts created during workshops... 2015-05-07 11:34 GMT+02:00 Lodewijk <lodew...@effeietsanders.org>: > Hi Sam, > > The main misconception (which is understandable, but also often pointed out > already) is that Wiki Loves Monuments can be fundamentally different > projects from a goals-and-outcomes point of view, based on the interests > and strenghts of the local organizers and the local situation. In some > countries, the main outcome of the competition is that it brings together > organizers for a first project, that can then move on, and leverage their > collaboration in other projects. In other countries it fosters > collaborations with other organizations. > > In some countries, it is a very grassroots competition, with low budget and > big focus on getting a lot of photos. In other countries, there is a lot of > effort (and funding) going into catching editors, setting up structures or > overcoming the local challenges or making concepts better aware. > > Aside from the fact that many of these outcomes are qualitative, which > seems to get no attention in the (summaries of the) reports, but do get > described in the reports of the individual contests, the local competitions > are too diverse to try and catch as one group. > > This is a fundamental flaw (pointed out before) in the approach. The work > is appreciated of course, the numbers can be useful - the way they are > presented is however very sensitive for major misunderstandings. > > Besides this, there are several very specific flaws in the number crunching > that have been pointed out, which are for example messing up the numbers on > editor retention. > > I hope that at some point WLM organizers can be given the tools, enthusiasm > and support to create their own evaluation on a larger scale. That way I > hope that some of the flaws can be avoided thanks to a better understanding > of the collaborations, structures and the projects in general. > > All in all it is good to have something 'to shoot at' but I would prefer > that these reports are produces more in concert with the stakeholders > involved and affected, rather than 'announced' and 'presented' to the wide > community. > > Best, > Lodewijk (effeietsanders) > member of the international coordinating team 2011-2013 > > On Wed, May 6, 2015 at 4:40 PM, Samuel Klein <meta...@gmail.com> wrote: > > > Claudia, I share your concerns about reducing subtle things to a few > > numbers. Data can also be used in context-sensitive ways. So I'm > > wondering if there are any existing quantitative summaries that you find > > useful? Or qualitative descriptions that draw from more than one > project? > > > > Figuring out what ideas are repeatable, scalable, or awesome but one-time > > only, is complex. We probably need many different approaches, not one > > central approach, to understand and compare. > > > > I'm glad to see data being shared, and again it might help to have many > > different datasets, to limit conceptual bias in what sort of data is > > relevant. > > On May 6, 2015 9:59 AM, "Claudia Garád" <claudia.ga...@wikimedia.at> > > wrote: > > > > > Hi Sam, > > > > > > I am sure there are figures and stories that the various orgs collect > and > > > publish. But they are spread across different wikis and websites and/or > > > languages. E.g. many of the FDC orgs are looking into ways to > demonstrate > > > these more qualitative aspects of our work (e.g. by storytelling) in > > their > > > reports. > > > But these information does not get the same attention and publicity in > > the > > > wider community as the evaluation done by the WMF. Many WMAT volunteers > > and > > > I myself share the concerns expressed by Romaine that these > > unidimensional > > > numbers and lack of context foster misconceptions or even prejudices > > > especially in the parts of the community that are not closely involved > in > > > the work of the respective groups and orgs. > > > > > > Best > > > Claudia > > > > > > > > > > > > Am 06.05.2015 um 13:40 schrieb Sam Klein: > > > > > >> Hi Romaine, > > >> > > >> Are there other evals of WLM projects that capture the complexity you > > >> want? > > >> > > >> Perhaps single-community evaluations done by the WLM organizers there? > > >> > > >> Sam > > >> > > >> On Wed, May 6, 2015 at 7:21 AM, Romaine Wiki <romaine.w...@gmail.com> > > >> wrote: > > >> > > >> Hi all, > > >>> > > >>> In the past months the Wikimedia Foundation has been writing an > > >>> evaluation > > >>> about Wiki Loves Monuments. [1] > > >>> > > >>> At such it is fine that WMF is writing an evaluation, however they > fail > > >>> in > > >>> actual understanding Wiki Loves Monuments, and that is shown in the > > >>> evaluation report. > > >>> > > >>> As a result on the Wiki Loves Monuments mailing list a discussion > grows > > >>> about the various problems the evaluation has. > > >>> > > >>> As the Learning and Evaluation team at the Wikimedia Foundation > already > > >>> had > > >>> released the first Programs Reports for Wiki Loves Monuments, we are > > now > > >>> put as fait accompli with this evaluation report. > > >>> > > >>> Therefore I am writing here so that the rest of the worldwide > Wikimedia > > >>> community is informed that this is not going right. > > >>> > > >>> Wiki Loves Monuments is not just a bunch of uploads done in > September, > > >>> the > > >>> report is too simplified without actual understanding how the > community > > >>> is > > >>> doing this project. > > >>> > > >>> > > >>> Romaine > > >>> > > >>> > > >>> > > >>> [1] > > >>> > > >>> > > >>> > > > https://meta.wikimedia.org/wiki/Grants:Evaluation/Evaluation_reports/2015/Wiki_Loves_Monuments > > >>> _______________________________________________ > > >>> Wikimedia-l mailing list, guidelines at: > > >>> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines > > >>> Wikimedia-l@lists.wikimedia.org > > >>> Unsubscribe: > https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, > > >>> <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe> > > >>> > > >> > > >> > > >> > > >> > > > > > > _______________________________________________ > > > Wikimedia-l mailing list, guidelines at: > > > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines > > > Wikimedia-l@lists.wikimedia.org > > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, > > > <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe> > > _______________________________________________ > > Wikimedia-l mailing list, guidelines at: > > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines > > Wikimedia-l@lists.wikimedia.org > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, > > <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe> > > > _______________________________________________ > Wikimedia-l mailing list, guidelines at: > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines > Wikimedia-l@lists.wikimedia.org > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, > <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe> > -- Tomek "Polimerek" Ganicz http://pl.wikimedia.org/wiki/User:Polimerek http://www.ganicz.pl/poli/ http://www.cbmm.lodz.pl/work.php?id=29&title=tomasz-ganicz _______________________________________________ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe>
