Hi Taylor. We're discussing this in the committee. I agree that to the extent they can accurately reflect something, language surveys are useful, and appreciate that you want to run a useful survey, and certainly want to encourage and help you in making it as broad and useful as possible. That said, I don't know if slapping a "haskell.org" label on the survey will help manage the biggest drivers of selection bias -- which is not only about who chooses to respond, but about who is reached through what mechanisms. (I don't know the relative importance of response-bias vs. outreach-bias in general, and would be curious if somebody has some good research on that to point to). I honestly don't know if we have enough channels _in general_ to do a good survey, no matter who runs it or how at all! Regardless of the decision we come to, here are a few of my personal thoughts on the questions you have thus far, and what could be added:
1) A question "how did you hear about this survey" -- this could at least help to disentangle outreach-bias, or notice it, depending on if it induces any correlations. 2) A question on how long after a new GHC release users upgrade -- both personally, and at work. 3) Distinguishing between personal and work build-systems in the relevant question. 4) I think the sorts of questions that make sense to ask in this early part can resemble those in the first part of the Go survey: https://blog.golang.org/survey2017-results (I especially like the questions about the area of development and server vs cli apps vs libraries). I also like their questions about what environments teams deploy programs to. It would also be worth asking if the apps developed are customer-facing or internal. 5) I think an interesting question would be what preferred js solution, if any, teams adopt -- i.e. ghcjs, typescript, purescript, raw js, etc. 6) for the "why did you stop" question, there are a good range of potential multi-choice answers that can be drawn from with the rust user survey: https://blog.rust-lang.org/2017/09/05/Rust-2017-Survey-Results.html Cheers, Gershom On Sun, Oct 14, 2018 at 3:32 PM Taylor Fausak <tay...@fausak.me> wrote: > > I'm not entirely sure what official support would look like. A few things > come to mind: > > 1. Simply putting "official" somewhere in the title, such as: Official 2018 > state of Haskell survey. > > 2. Putting something about Haskell.org in the description of the survey, such > as: Sponsored by Haskell.org. > > 3. Announcing the survey through channels that I may not be aware of. Or > helping me announce the survey through various channels (mailing lists, > Reddit, and so on) by mentioning Haskell.org. > > I'm sure there are more ways that I'm not thinking of. > > Perhaps it would be better to state my goal and see what, if anything, can be > to achieve that goal? My goal is for this survey to be *the* authoritative > Haskell survey and for the community to broadly accept it results. In > particular I would like to avoid reactions like these to the recent FP > Complete survey: > > > I browse r/haskell all the time and follow FPco on Twitter, and I wasn't > > aware of this survey. [1] > > > It should not be surprising to think that fp complete has much better > > outreach to Stackage users than to non Stackage users. [2] > > > Any survey hosted by FPComplete is biased towards users of stack, for > > reasons that should be self evident. [3] > > Similar sentiments have been expressed about last year's Haskell Weekly > survey: > > > Haskell Weekly's reputation is tainted as it appears to be seen as partisan > > (and I tend to agree). [4] > > > one of those surveys is from FP Complete and one of them is from someone > > who I would consider very partisan in these kind of discussions. [5] > > > For the love of god stop posting those surveys. They're not convincing and > > obviously flawed. [6] > > I don't expect to be able to make everyone happy, but I think that presenting > this year's survey as sponsored by both Haskell Weekly and Haskell.org would > go a long way toward making it acceptable to a broad range of the Haskell > community. > > I hope that helps! > > [1]: > https://np.reddit.com/r/haskell/comments/9mm05d/2018_haskell_survey_results/e7gplya/ > [2]: > https://np.reddit.com/r/haskell/comments/8tc8pr/fp_complete_launches_new_blockchain_auditing/e188ftv/?context=3 > [3]: > https://np.reddit.com/r/haskell/comments/9mm05d/2018_haskell_survey_results/e7gpfwe/ > [4]: > https://np.reddit.com/r/haskell/comments/9mm05d/2018_haskell_survey_results/e7ka8xn/ > [5]: > https://np.reddit.com/r/haskell/comments/8tc8pr/fp_complete_launches_new_blockchain_auditing/e187pzq/?context=1 > [6]: > https://np.reddit.com/r/haskell/comments/8uw9hw/psa_cabal_breaks_with_yaml0831_on_ghc_710_and/e1lfzgr/?context=1 > > > On Sun, Oct 14, 2018, at 2:21 PM, Neil Mitchell wrote: > > Hi Taylor, > > What does official support look like? I don't think there's anything above > and beyond what you're already doing. > > Thanks, Neil > > On Sun, 14 Oct 2018 at 3:49 pm, Taylor Fausak <tay...@fausak.me> wrote: > > (I have CCed <commit...@haskell.org> because the haskell-community mailing > list seems relatively quiet and I want to make sure this is seen.) > > Hello! My name is Taylor Fausak. I run the Haskell Weekly newsletter. Last > year I published a survey [1] for the Haskell community. I collected and > reported [2] on about 1,335 responses. I plan on publishing another survey > this year on the same date, November 1st. I am developing it in the open > again [3] and would love to hear from any interested parties. Please let me > know if you have any ideas about the survey! > > This year I am interested in making the survey official by seeking support > from Haskell.org. Is such a thing possible and desirable? > > Thanks for your consideration! I hope to hear from you soon. > > [1]: https://haskellweekly.news/surveys/2017.html > [2]: https://taylor.fausak.me/2017/11/15/2017-state-of-haskell-survey-results/ > [3]: https://github.com/haskellweekly/haskellweekly.github.io/issues/206 > _______________________________________________ > Haskell-community mailing list > Haskell-community@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community > > > _______________________________________________ > Haskell-community mailing list > Haskell-community@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community _______________________________________________ Haskell-community mailing list Haskell-community@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community