I have been on conference committee for almost a decade, I can’t recall us ever having more than 4 proposals for the global FOSS4G. Each year there are different factors that influence our choices, I don’t recall a year when there was much doubt about who we should select. A predetermined marking system would not make our job any easier it would just lead to endless debate about the relative weighting of each of the factors ______ Steven
Unusual maps in strange places - mappery.org <http://mappery.org/> Subscribe to my weekly “Maps in the Wild <http://eepurl.com/dKStT-/>” newsletter > On 13 Jan 2022, at 15:22, María Arias de Reyna <dela...@gmail.com> wrote: > > On Thu, Jan 13, 2022 at 1:13 PM Jonathan Moules via Discuss > <discuss@lists.osgeo.org> wrote: >> I don't think there's any need to reinvent the wheel here; a number of >> open-source initiatives seem to use scoring for evaluating proposals. >> Chances are something from one of them can be borrowed. >> >> Apache use it for scoring mentee proposals for GSOC: >> https://community.apache.org/mentee-ranking-process.html >> >> Linux Foundation scores their conference proposals for example: >> https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/program/scoring-guidelines/ > > Am I understanding it wrong or this is to accept talk proposals, not > conference proposals? > > Scoring a contractor for a well defined project (as you pointed public > administrations do), choosing the right person for a specified job, or > deciding if a talk deserves to be in a schedule is more or less "easy" > compared to decide who is hosting a conference. > > If you want to propose a draft of score requirements for FOSS4G, I > think it would be interesting to go through them and try to come up > with something. Even if the scoring is not binding, it may help future > proposals see what is the path. > > My only "but" with this system (which I use almost always when I have > to review anything and I intended to use for this FOSS4G voting) is > that it is hard to come up with an objective system that counts all > the variables. And if the score does not match the final decision, it > may be difficult to process. > > I have been on the GSoC as mentor with the ASF and true, we have a > ranking process, but it helped us mostly to order the candidates and > reject those that deviate too much. The final decision was not a clear > numeric decision. When the difference is small, you do have to > consider other things. And from what I have seen these past few years > on FOSS4G, either there is one candidate that outshines obviously, or > the difference is really small between candidates and it comes down to > things that may not be even defined on the RFP. > > And there's things you have to consider that a generic scoring system > can't help you with. We used this system in FOSS4G 2021 to decide > which talks to accept on the conference, where the community voting > had a strong weight but was not binding. And we had to make some > exceptions with good talks that were experimental but didn't get a > good score and objectively numerically they were rejected. We also had > to reject some duplicated talks that had a high score but we couldn't > argue both were accepted. Which one to reject? Usually the one that > had a speaker with more talks. But what if both have a speaker with no > more talks? That's something you have to check case by case. > > Which leads us that with the scoring there is less room for > experimentation because the candidates will focus on getting high > scores on specific questions. Not on offering what is their best. For > example, the proposal we made for FOSS4G Sevilla 2019 in a pirate > amusement park to celebrate Magallanes... no score could have > predicted that. > > So I may agree on scoring, not on binding scoring. > > But first we need some draft to work on to score proposals :) > _______________________________________________ > Conference_dev mailing list > conference_...@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/conference_dev
_______________________________________________ Discuss mailing list Discuss@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/discuss