John, This is all very insightful and I agree with all of it. One reason we posted the survey in 12 different channels was because we have all worked within OSM for a long time and we are familiar with the different cultures within the different channels.
You wrote, "So are you interested in a sample of OpenStreetMap mappers or a sample of online forum mappers who are happy with Google forms as your sample? There is a difference and as long as you don't say our survey says everyone in OSM thinks this about online forums I'm happy and content." We want the former, but there are limits to our reach and resources, so we will have to be content with something less than that, but hopefully not as limited as the latter. It reassures me that what you are looking for is for us to give appropriate context and framing, because I can absolutely promise that we will do that. Thank you for this thoughtful response. :) C On Sun, Apr 30, 2023 at 1:46 PM john whelan <jwhelan0...@gmail.com> wrote: > My background was working with surveys and my comments simply came from > that background and the steps taken to obtain accurate results. Nothing > else. > > Typically a university run survey isn't done to high standards. > > Your comment on questions from talk I think relates to the users. > OpenStreetMap roots are in open data and a desire to avoid proprietary > systems. The old fogies, if you will, tend to use mailing lists, they do a > lot of editting and background work to make it run smoothly > disproportionately so. Often they don't use the flavour of the month > online forum. > > So if you ask in an online forum are on line forums great you'll get a > positive answer. Those who don't think they are great won't use them. > > If you ask in the talk mailing list you get a different set of respondents. > > As I said before the selection of the sample is critical. There is a > story told of an interviewer who surveyed passengers at a railway station > about gambling. 100% were in favour. It was only later it was spotted > she'd interviewed people who were there to catch a special train to a race > course. > > So what exactly are you trying to measure? Are you weighting the replies > against the number of edits the person has made? Does a HOT mapper who has > made three edits count. Remember that they will almost certainly have been > recruited through social media. > > Then you get people who map quite happily by themselves or in a very small > group of two communicating one on one via email. Not everyone feels the > need to group hug in an online forum. The mapper I'm thinking of is exHOT > and prefers to quietly map parts of Africa that are basically unmapped. We > do work together and communicate via email to decide which bit to map > next. Remember HOT is very much working together and for some mappers this > doesn't work well. > > So are you interested in a sample of OpenStreetMap mappers or a sample of > online forum mappers who are happy with Google forms as your sample? > > There is a difference and as long as you don't say our survey says > everyone in OSM thinks this about online forums I'm happy and content. > > Cheerio John > > On Sun, Apr 30, 2023, 13:06 Courtney <courtney.william...@gmail.com> > wrote: > >> >> We do indeed have people with non technical backgrounds working on the >> survey, including a multilingual person with an advanced degree in language >> and technology, and a person with an advanced degree in English language. >> We have two very experienced data analysts working on it, as well. >> >> We did not run a trial survey against a random sample because, as I said >> in my previous post, this survey is an ancillary part of a larger, >> long-term study that relies on publically available data from OSM >> communication channels. We are also quite capable of framing our findings >> within the context of how the survey was distributed, with appropriate >> reference to everything from survey bias, to the difficulties of conducting >> a free survey across a global community, to the short amount of time that >> we have to do the survey. No one is claiming that we will be able to >> deliver the one true, definitive quantitative analysis of OSM communication >> behaviors to rule them all. We are attempting to uncover some >> directional behaviors, and see if we can foster >> a better conversation within the community. >> >> This conversation has opened up important new questions. Why is the main >> "Talk" channel the only one that is producing pushback? Why is it the only >> one that is producing such a negative tone? How widely is the principle of >> using only open source software adopted across the community? We already >> had a question to this effect within the survey, but we will now be able to >> learn more by adding the limesurvey. None of this is going to be >> definitive. All of it is going to be interesting and help raise new >> questions that hopefully can be studied. >> >> Can I ask--what is the fundamental objection to us trying to learn a bit >> more about OSM communication habits? I understand the impulse to give >> advice--this is welcome even when the advice is predicated on the idea that >> we lack any kind of insight or experience--there is always more to learn. >> But, I don't understand the degree of ire and frankly, incredulity that is >> being levied here. Should we wait until there is a university study that >> is fully funded and staffed, and with a perfect approach, with a year's >> worth of pre-testing, to ask these questions? Is that the standard here? >> Wait for perfection or do nothing? Is that how OSM itself was built? I >> don't understand the tone or the defensiveness of these comments. If the >> goal is to advance the OSM project, is it better to gate keep all inquiries >> to a suffocating degree? Or to try to learn and grow? >> >> On Sun, Apr 30, 2023 at 11:45 AM John Whelan <jwhelan0...@gmail.com> >> wrote: >> >>> Just a comment on Fredrick's input. Selecting the sample is one of the >>> most difficult parts of a survey to get right. The self selection part of >>> this survey makes it open to bias, as Frederick has commented this is >>> compounded by the platform. I'm not making a comment about if the platform >>> is appropriate or not just that if it affects your response then it begins >>> to cast doubt on your results. >>> >>> The second is knowing enough about your target audience so they will >>> understand your questions. Perhaps have someone non technical with an >>> English Language background, a librarian, for example check it for jargon. >>> One technique is to run a trial survey against a true random sample. I >>> don't think this was done here. >>> >>> If they don't understand what you're asking then you aren't going to get >>> a reliable answers and to be honest I didn't. >>> >>> I'm not sure if this particular survey is trying to justify a particular >>> stance or get accurate information. >>> >>> Cheerio John >>> >>> Frederik Ramm wrote on 4/30/2023 11:18 AM: >>> >>> Hi, >>> >>> On 4/28/23 15:57, Marc_marc wrote: >>> >>> I am impressed (and disappointed) that those who do these surveys >>> have still not learned that part of the active opendata community >>> does not wish to ally a closeddata based enterprise (nominally: >>> no use of google forms for some of us). >>> >>> >>> Agree. It's one thing for an OSMF working group to use a closed >>> source/siloed product internally, but quite another to attempt to engage >>> with the community via such a product. >>> >>> I am not surprised when a commercial company like Tom Tom does that >>> without a second thought, but I would expect more from an OSMF working >>> group. >>> >>> Please find a way for non-Google users to participate in this survey, or >>> your results will be biased to the point of un-suitability because they >>> will lack responses from people who'd rather not engage with Google, i.e. >>> the whole "communication behaviours" of this group of people would not be >>> represented. >>> >>> Bye >>> Frederik >>> >>> >>> -- >>> Sent from Postbox <https://www.postbox-inc.com> >>> _______________________________________________ >>> talk mailing list >>> talk@openstreetmap.org >>> https://lists.openstreetmap.org/listinfo/talk >>> >> >> >> -- >> >> --Courtney Cook Williamson >> survivalbybook.substack.com >> _______________________________________________ >> talk mailing list >> talk@openstreetmap.org >> https://lists.openstreetmap.org/listinfo/talk >> > -- --Courtney Cook Williamson survivalbybook.substack.com
_______________________________________________ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
