In our case, in the very first competition WL I've organized with WMPT in 2018, we came to the clear conclusion that at this point a pre-announced jury does not work, let alone a pre-announced jury taking care of gender parity and other diversity issues. So we do it, as we say, with the "house silverware", with the very precious help of members of other affiliates and other members of the wiki community (country independent). It has been working quite well that way. As I said, all members of the jury (which can be confirmed on Montage) are usually tanked in our blog post afterwards.
We also came to the clear conclusion that the voting alone is not enough, for the reasons described by Ivo, so we set up a kind of final core jury or grand jury that has a final word on the voting results, adjusting them according to quality, scope and diversity, something that often does not came out of the voting results. The whole process, as it is, is transparent (or can be fully released in any moment). Forcing bureaucratic measures, such as the pre-announcement of the juri, would make the organization much more difficult, at least to the ones like us who are doing this uniquely on their volunteer time. Best, Paulo Ivo Kruusamägi <ivo.kruusam...@gmail.com> escreveu no dia domingo, 4/10/2020 à(s) 03:49: > In Estonia, I think we have always published the names of jury members (or > in some cases, their wiki names) in all of the article and image > competitions we've had over the last 10+ years. At least any exceptions > don't come to my mind. So I don't see a problem with the expectation that > jury members should be clearly brought out. > > It becomes even more important on the international level. Like with Wiki > Science Competition we have even published the results on how have jury > members voted. Like: > https://commons.wikimedia.org/wiki/Commons:Wiki_Science_Competition_2019/Winners/People_in_Science/round_2 > Not that it needs to be that specific, but still... With the local version > of the science photo competition, it has even been common that all the jury > physically comes together to review the files and discuss them. It really > helps to use the competence of the jury (I'm "bit" critical to relying only > on casting votes, as this clearly works against the images that are > actually more special). Occasionally it has happened that one of the jury > members can't participate, but I haven't taken the name out for that and > he/she has always had the possibility to provide feedback via email. And > with the physical gathering is also obvious, that there can't be any > numerical values to be published later except to say that "jury reached a > consensus" or to bring out what the jury members thought about some images. > > What has happened a few times (can remember 3) over the years is that the > results have been so weak or one-sided, that I didn't assemble a jury [with > smaller competitions that may take place at the same time the > articles/photos are being collected] or I not used it (as there are so > clear winners that it would a total waste of time to do so). That is: > setting up a fancy jury only makes sense if there would be results worthy > of that jury. It there would only be a few not-so-good files, then the jury > members may not want to return the next year. And when it is the first time > to organize something it may not be clear on what may be the result. As > privacy issues can easily be avoided, then the only possible edge case that > comes to my mind is the possibility that the results may end up being too > weak to justify some decent pre-announced jury... but with WLM I'd guess > there should anyway be like at least 500 images to choose from whatever > participating country, so that should not be a problem. > > Correction: when looking at the list of ca 70 competitions we've had then > there wasn't jury listed here: https://www.miljonpluss.ut.ee/24h/ (even > thou there still was something in the press release) > > Regards > Ivo Kruusamägi > > Kontakt attolippip (<attolip...@gmail.com>) kirjutas kuupäeval L, 3. > oktoober 2020 kell 20:25: > >> Last year we did publish an overview of how the evaluation/selection >> process was organised (in Ukrainian): http://wlm.org.ua/top123_2019/ >> while introducing top 123 pictures of the last round (as it was published >> before announcing our top 10). I think it also helped to engage people on >> social media, as we asked our followers to submit their guesses about top >> 10 from these 123 in the comments :) >> >> On Sat, Oct 3, 2020, 19:56 effe iets anders <effeietsand...@gmail.com> >> wrote: >> >>> Hi Mykola, >>> >>> I would encourage WLX Jury Tool to follow suit and provide a similar >>> option to give easy openness to its users. There is no obligation to use >>> either or any tool. This overview is just a convenient way to make it >>> easier for organizers to be transparent. >>> >>> Lodewijk >>> >>> >>> >>> On Sat, Oct 3, 2020 at 9:49 AM Mykola Kozlenko <mycol...@ukr.net> wrote: >>> >>>> Hi Lodewijk, >>>> >>>> As we intend to use WLX Jury Tool and not Montage in Ukraine, what is >>>> expected from us? >>>> >>>> We always publish the list of our jury members here: >>>> http://wlm.org.ua/juri/ , you can see all our jury members between >>>> 2012 and 2019. The 2020 list is still being finalised as we are still >>>> waiting for final confirmations, but we will also publish it as we did in >>>> previous years. >>>> >>>> On the other side, as we have 47.5K images in Ukraine, we will have to >>>> organise multiple rounds, including checking the images before submitting >>>> them to the jury (as no jury is able to review that many images) and >>>> possibly a final call between jury members to distribute places among the >>>> top photos. We are happy to explain our selection process but we may not >>>> have the same details as for the teams using Montage. >>>> >>>> Best regards >>>> Mykola (NickK) >>>> WLM Ukraine team >>>> >>>> --- Оригінальне повідомлення --- >>>> Від кого: "effe iets anders" <effeietsand...@gmail.com> >>>> Дата: 1 жовтня 2020, 00:49:56 >>>> >>>> Hi all, >>>> >>>> After getting feedback both on- and offlist, we arrived at the >>>> following: the Montage developers will make a page available to each jury >>>> coordinator, that they can copy and paste to a wiki page to share their >>>> process settings easily. That way it's easy to be transparent. At some >>>> point in the future, we will likely make this public by default. >>>> >>>> I do express the expectation that each national coordinator will >>>> publish their jury members (either real life name or username). It may well >>>> be that we ask you to submit this list of jurors when you submit your >>>> nominees (although we will not publish it on your behalf, that is your >>>> responsibility). >>>> >>>> Warmly, >>>> Lodewijk >>>> >>>> On Thu, Sep 3, 2020 at 11:01 PM effe iets anders < >>>> effeietsand...@gmail.com> wrote: >>>> >>>> All great suggestions. >>>> >>>> In the past the WLM international team has also maintained a database >>>> of jurors to be backup. We had very little requests in recent years though, >>>> probably because so many people know organizers form other countries. If >>>> you're stuck, I would suggest to either ask someone you already know, or to >>>> ask this list. Most likely you have a response within 48h. But don't wait >>>> until the last minute, if you can avoid it. Jurors typically like a week to >>>> get things done (if the country isn't huge). >>>> >>>> Lodewijk >>>> >>>> On Thu, Sep 3, 2020 at 3:48 AM Rebecca O'Neill <rebeccanin...@gmail.com> >>>> wrote: >>>> >>>> Thank you Yaroslav, I will send you a separate email now! >>>> >>>> Having the WLE team setting up a pool of jurors like that was so >>>> helpful Anton. Yes, I could find the names and email other WLM organisers, >>>> but this system took that extra work away from me - which was hugely >>>> appreciated, especially in our first year! >>>> >>>> On Thu, 3 Sep 2020 at 11:45, Anton Protsiuk < >>>> anton.prots...@wikimedia.org.ua> wrote: >>>> >>>> The international WLE team this year has tried to create a database of >>>> jurors from different countries for local contests. We had a Google form >>>> for these purposes (https://forms.gle/Pj61adjgYiE6Jn687) & asked local >>>> teams whether they needed help with jury. >>>> We haven't estimated the results yet, since the local selection process >>>> is still ongoing, but it seems to have worked well. >>>> >>>> Best Regards >>>> Anton Protsiuk >>>> Project Manager at Wikimedia Ukraine >>>> >>>> >>>> On Thu, Sep 3, 2020 at 11:05 AM Yaroslav Blanter <ymb...@gmail.com> >>>> wrote: >>>> >>>> If WLM Ireland is in September I can help as a juror. If it is in >>>> October this could be more difficult because I am also in the jury for >>>> Russia, and there is typically a lot of work there. >>>> >>>> Generally, asking around (may be also on Commons) typically helps. In >>>> the past I have been on jury for a few different countries. Somebody would >>>> just approach me and ask.If I can make it, I agree. >>>> >>>> Yaroslav >>>> >>>> On Thu, Sep 3, 2020 at 9:50 AM Mārtiņš Bruņenieks <marti...@gmail.com> >>>> wrote: >>>> >>>> Hi, Rebecca! >>>> >>>> I have organized multiple WLM/WLE editions in Latvia. We have used >>>> Montage in most of them. >>>> After experiencing problems with jurors dropping out or being known at >>>> the last moment, we created new Wikimedia accounts just for the jury and >>>> sent the passwords to them. >>>> >>>> Montage allows some editing of jurors after the round has started but >>>> results can be unexpected. >>>> >>>> Mārtiņš >>>> >>>> >>>> On Thu, Sep 3, 2020 at 2:13 AM Rebecca O'Neill <rebeccanin...@gmail.com> >>>> wrote: >>>> >>>> Hi Lodewijk, >>>> >>>> As a slight aside to this, and perhaps something that might help ease >>>> some of the worries around judging processes in other countries, would it >>>> be an idea to have a pool of international WLM Wikimedia jurors that could >>>> help judge other countries? Here in Ireland we have done this with Malta, >>>> exchanging jurors over the past few years, and for our first WLE this year >>>> we had Axel from Sweden be on our jury. You get the benefit of a fellow >>>> Wikimedian who understands the whole process, and who could bring some >>>> different expertise or perspective to a country's WLM. Having someone who >>>> is completely unfamiliar with your local built heritage means they can >>>> assess the images with a different take than someone who knows them very >>>> well. >>>> >>>> After 6 years, we have found it harder to recruit a jury from our pool >>>> of active Wikimedians and relevant expert judges from the arts and >>>> architecture sectors in Ireland. I know I would really appreciate it if we >>>> could "borrow" a juror from another Wikimedia group (in the past we have >>>> had some UK help with this too with jurors). We have suffered from jurors >>>> dropping out of the process at the last minute or after judging has begun >>>> (which results in having to restart rounds in Montage), generally it has >>>> been those who are not Wikimedians who perhaps did not fully understand the >>>> commitment when they agreed. Not only is it frustrating, it's very >>>> stressful. It may be less of an issue this year, given that the deadline >>>> for submitting to the international jury won't be at the end of October. >>>> >>>> Hope that helps clarify some of the issues some of the smaller >>>> countries can face over the years of WLM! >>>> Thanks, >>>> Rebecca >>>> >>>> On Wed, 2 Sep 2020 at 23:57, effe iets anders <effeietsand...@gmail.com> >>>> wrote: >>>> >>>> Hi Alexander, >>>> >>>> As for the status of Montage requests: I suggest that you start a >>>> separate thread on that, and would like to leave this to the maintainers to >>>> respond to. >>>> >>>> As for publishing the settings: I was imagining some kind of log-style >>>> publication, not a near write-up. This won't be pretty, but it will allow >>>> people to figure out how it worked out in practice. If we follow a logical >>>> naming convention, people should be able to puzzle it together. Ideally, >>>> the national organizers also publish their process on the website, but this >>>> log would be a way to verify that. But I accept your note that we may need >>>> to add a context explaining that more process may happen before/after this >>>> tool is used. >>>> >>>> Lodewijk >>>> >>>> On Wed, Sep 2, 2020 at 3:17 PM Alexander Tsirlin <altsir...@gmail.com> >>>> wrote: >>>> >>>> Dear Lodewijk, >>>> >>>> We also intend to publish by default the settings of the montage jury >>>> tool, and the number of photos in each round that the national competitions >>>> have used. We're debating whether there should be an opt-out for this year. >>>> >>>> This is unrealistic, because jury process may involve several campaigns >>>> that are later merged together (in Russia, we do it all the time in order >>>> to meet your submission deadline). Moreover, some of the photos can be >>>> accepted for the next round within Montage but excluded later on if we find >>>> that they do not depict cultural heritage. The end result is that any >>>> number you take from Montage will not match the number of photos that we >>>> publish (e.g., as a short-list). This will only lead to confusion and won't >>>> be of any use for anyone. >>>> >>>> Since you mentioned Montage, let me also ask when two important pull >>>> requests, which were done by one of our team members, are going to be >>>> merged into the code: >>>> https://github.com/hatnote/montage/pull/169 >>>> https://github.com/hatnote/montage/pull/175 >>>> These are really, really important fixes. Without them I would have a >>>> problem creating new Montage campaigns in October. >>>> >>>> Sincerely, >>>> Alexander >>>> >>>> >>>> On 9/2/2020 11:49 PM, effe iets anders wrote: >>>> >>>> Hi all, >>>> >>>> over the past years, we have had various requests to encourage national >>>> organizers to be transparent in their judging processes and who sits on >>>> their jury. Most of the national organizers are currently transparent about >>>> this already. In the past weeks/month, more conversation around this has >>>> continued with some concerns (valid or not) on certain jury processes. >>>> >>>> In this light, the international team intends to institute a new >>>> expectation for national organizers, namely to publish the members of their >>>> jury (be it their username or real life name) at some point. We have not >>>> figured out the practical details yet, but I can imagine that while we >>>> encourage publication on the website, we would ask national organizers to >>>> add a list of jury members to their submission to the international jury - >>>> which we then will publish as well. >>>> >>>> We also intend to publish by default the settings of the montage jury >>>> tool, and the number of photos in each round that the national competitions >>>> have used. We're debating whether there should be an opt-out for this year. >>>> >>>> We will of course apply at least the same level of transparency to the >>>> international jury. >>>> >>>> Before we make this decision, I would like to ask for feedback on this, >>>> and whether there are edge cases we should consider where such transparency >>>> would be harmful. I'll take 1 week to gather some feedback on this, and >>>> then we'll make a final decision. You can respond to this on this mailing >>>> list, or privately to me. >>>> >>>> Warmly, >>>> Lodewijk >>>> >>>> _______________________________________________ >>>> Wiki Loves Monuments mailing >>>> listWikiLovesMonuments@lists.wikimedia.orghttps://lists.wikimedia.org/mailman/listinfo/wikilovesmonumentshttp://www.wikilovesmonuments.org >>>> >>>> >>>> _______________________________________________ >>>> Wiki Loves Monuments mailing list >>>> WikiLovesMonuments@lists.wikimedia.org >>>> https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments >>>> http://www.wikilovesmonuments.org >>>> >>>> _______________________________________________ >>>> Wiki Loves Monuments mailing list >>>> WikiLovesMonuments@lists.wikimedia.org >>>> https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments >>>> http://www.wikilovesmonuments.org >>>> >>>> >>>> >>>> -- >>>> PhD in Digital Media >>>> Project Coordinator Wikimedia Community Ireland <http://wikimedia.ie> >>>> She/Her >>>> _______________________________________________ >>>> Wiki Loves Monuments mailing list >>>> WikiLovesMonuments@lists.wikimedia.org >>>> https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments >>>> http://www.wikilovesmonuments.org >>>> >>>> _______________________________________________ >>>> Wiki Loves Monuments mailing list >>>> WikiLovesMonuments@lists.wikimedia.org >>>> https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments >>>> http://www.wikilovesmonuments.org >>>> >>>> _______________________________________________ >>>> Wiki Loves Monuments mailing list >>>> WikiLovesMonuments@lists.wikimedia.org >>>> https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments >>>> http://www.wikilovesmonuments.org >>>> >>>> _______________________________________________ >>>> Wiki Loves Monuments mailing list >>>> WikiLovesMonuments@lists.wikimedia.org >>>> https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments >>>> http://www.wikilovesmonuments.org >>>> >>>> >>>> >>>> -- >>>> PhD in Digital Media >>>> Project Coordinator Wikimedia Community Ireland <http://wikimedia.ie> >>>> She/Her >>>> _______________________________________________ >>>> Wiki Loves Monuments mailing list >>>> WikiLovesMonuments@lists.wikimedia.org >>>> https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments >>>> http://www.wikilovesmonuments.org >>>> >>>> _______________________________________________ >>>> Wiki Loves Monuments mailing >>>> listWikiLovesMonuments@lists.wikimedia.orghttps://lists.wikimedia.org/mailman/listinfo/wikilovesmonumentshttp://www.wikilovesmonuments.org >>>> >>>> _______________________________________________ >>> Wiki Loves Monuments mailing list >>> WikiLovesMonuments@lists.wikimedia.org >>> https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments >>> http://www.wikilovesmonuments.org >>> >> _______________________________________________ >> Wiki Loves Monuments mailing list >> WikiLovesMonuments@lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments >> http://www.wikilovesmonuments.org >> > _______________________________________________ > Wiki Loves Monuments mailing list > WikiLovesMonuments@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments > http://www.wikilovesmonuments.org >
_______________________________________________ Wiki Loves Monuments mailing list WikiLovesMonuments@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments http://www.wikilovesmonuments.org
