> And cognitive bias suddenly does not play a role anymore when you
score a good friend vs a hated enemy against a "list of
requirements"....? It might look transparent but is not the tiniest bit
more fair.
Sure the biases will still be there, but the justification for the score
is written down for all to see. Hence: Transparent. It'll be available
for the entire community to then read; if it's a rationalisation it'll
be there for all to see (and call out).
Suggestions for even more fairness are welcome.
On 2022-01-13 14:25, Kobben, Barend (UT-ITC) wrote:
Quoting "To work around this, with public sector contracts in the
western world you have a list of requirements and then all the bids
are scored against those requirements. The one with the highest score
wins the contract. *That* is transparent. "
Really...? And cognitive bias suddenly does not play a role anymore
when you score a good friend vs a hated enemy against a "list of
requirements"....? It might look transparent but is not the tiniest
bit more fair.
/-- /
/Barend Köbben/
*From: *Discuss <discuss-boun...@lists.osgeo.org> on behalf of
Jonathan Moules via Discuss <discuss@lists.osgeo.org>
*Organisation: *LightPear
*Reply to: *"jonathan-li...@lightpear.com" <jonathan-li...@lightpear.com>
*Date: *Thursday, 13 January 2022 at 13:13
*To: *Bruce Bannerman <bruce.bannerman.os...@gmail.com>
*Cc: *"discuss@lists.osgeo.org" <discuss@lists.osgeo.org>
*Subject: *Re: [OSGeo-Discuss] Conference selection transparency (Was
Announcement: Call for Location global FOSS4G 2023)
Excellent question Bruce!
I don't think there's any need to reinvent the wheel here; a number of
open-source initiatives seem to use scoring for evaluating proposals.
Chances are something from one of them can be borrowed.
Apache use it for scoring mentee proposals for GSOC:
https://community.apache.org/mentee-ranking-process.html
Linux Foundation scores their conference proposals for example:
https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/program/scoring-guidelines/
A comprehensive web-page with tons of suggestions and guidance for how
to do it: https://rfp360.com/rfp-weighted-scoring/
Best,
Jonathan
On 2022-01-13 11:43, Bruce Bannerman wrote:
Jonathan,
Do you have a suggestion as to how the process can be improved?
Kind regards,
Bruce
Disclosure:
I was a member of the LOC for FOSS4G-2009.
I personally don’t have a problem with the process as is, but it
may be possible to improve things. That is, provided that we don’t
make the job of our volunteers more difficult than it needs to be.
In the end the people who have stepped up to do the work will need
to make the call. We may not like the outcome, but we need to
trust that they are acting in OSGeo’s best interest and respect
their decision.
On 13 Jan 2022, at 20:58, Jonathan Moules via Discuss
<discuss@lists.osgeo.org> <mailto:discuss@lists.osgeo.org> wrote:
> Anyone can ask questions to the candidates.
Yes, they can (and yes, I have asked questions), but here's
the thing: The only people who actually matter are the people
who vote. And we have no idea what they vote (for the valid
reason stated) or what their criteria are for their vote
(which is a problem). If the committee don't read and/or care
about the questions asked/answered then said questions/answers
are meaningless.
> The only two things that are not public are:
I disagree, the third thing that's not public, and by far the
most important, is the actual scoring criteria. Each committee
member is a black-box in this regard. Not only do we not find
out *what* they voted (fine), we also never know *why* they
voted a specific way.
Did Buenos Aires win because:
* it had the shiniest brochure?
* it was cheapest?
* that's where the committee members wanted to go on holiday?
* nepotism?
* the region seemed like it'd benefit the most?
* they were feeling grumpy at the chair of the other RfP that day?
* they had the "best" bid?
... etc
Disclosure: I am definitely **NOT** stating those are the
reasons it was chosen!!! I'm highlighting them because the
lack of transparency means we can't know what the actual
reasons were. Frankly, given the absolutely huge list of
cognitive biases that exist, there's a reasonable chance that
the voters aren't voting why they think they're voting either.
That's just the human condition; we're great at deceiving
ourselves and rationalisations (me included).
To work around this, with public sector contracts in the
western world you have a list of requirements and then all the
bids are scored against those requirements. The one with the
highest score wins the contract. *That* is transparent.
TL;DR: We don't know why the voters vote as they do. The
public sector solves this by requiring scoring of bids against
a list of pre-published requirements.
I hope that clears things up. I'm not in any way suggesting
impropriety, I'm highlighting we have no way of knowing
there's no impropriety. Hence my claim as to a lack of
transparency; the votes are opaque.
Cheers,
Jonathan
On 2022-01-13 07:35, María Arias de Reyna wrote:
On Wed, Jan 12, 2022 at 10:50 PM Jonathan Moules via Discuss
<discuss@lists.osgeo.org> <mailto:discuss@lists.osgeo.org> wrote:
On the surface, this is a good idea, but unfortunately it has a
fundamental problem:
There are no "criteria for selection" of the conference beyond "the
committee members voted for this proposal". There's zero transparency in the process.
I can't let this serious accusation go unanswered.
All the process is done via public mailing lists. All the criteria
is
published on the Request For Proposals. Anyone on the community can
review the RFP and propose changes to it. Anyone on the community
can
read the proposals and interact with the candidatures.
The only two things that are not public are:
* Confidentiality issues with the proposals. For example sometimes
providers give you huge discounts in exchange of not making that
discount public. So you can't show the budget publicly, unless you
are
willing to not use the discount.
* What each member of the committee votes. And this is to ensure
they
can freely vote without fearing consequences.
Which are two very reasonable exceptions.
Anyone can ask questions to the candidates. If I am right, you
yourself have been very active on this process for the past years.
Were you not the one that asked what a GeoChica is or am I confusing
you with some other Jonathan? If I am confusing you with some other
Jonathan, my mistake. Maybe you are not aware of the transparency of
the process.
The process is transparent and public except on those two exceptions
that warrantee the process is going to be safe.
_______________________________________________
Discuss mailing list
Discuss@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/discuss
_______________________________________________
Discuss mailing list
Discuss@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/discuss
