Re: [Haskell-community] 2018 state of Haskell survey results

[Simon Peyton Jones via Haskell-community]

Just wanted to add in: good catch Gershom on identifying the problem, and thank 
you Taylor for working to remove them from the report.

I'd like to add +1 to that.

It's a source of astonishment, and some dismay, to me that anyone would go to 
so much trouble to affect a survey about Haskell.  (Brexit, perhaps, but 
Haskell??)

But many thanks to Gershom and Taylor for dealing with it so professionally.

Simon


From: Haskell-community  On Behalf Of 
Michael Snoyman
Sent: 18 November 2018 19:32
To: Taylor Fausak 
Cc: haskell-community@haskell.org
Subject: Re: [Haskell-community] 2018 state of Haskell survey results

Just wanted to add in: good catch Gershom on identifying the problem, and thank 
you Taylor for working to remove them from the report.


On 18 Nov 2018, at 21:17, Taylor Fausak 
mailto:tay...@fausak.me>> wrote:

Great catch, Gershom! There are indeed about 300 responses that tick all the 
boxes except for disliking the new GHC release schedule. The main thing the 
attacker seemed to be interested in was over-representing Stack and Stackage. 
Also, bizarrely, Java.

That brings the number of bogus responses up to 3,735, which puts the number of 
legitimate responses at 1,361. For context, last year's survey asked far fewer 
questions and had 1,335 responses.


On Sun, Nov 18, 2018, at 1:26 PM, Imants Cekusins wrote:
What if the announcement mentioned a large number of potentially bogus 
responses, explained the grounds for this conclusion, with a new survey 
conducted early next year?

The next survey would then need to be done differently from this one somehow. 
To improve the reliability, some authentication may be necessary.


Maybe Stack, Cabal questions could be grouped as separate distinct surveys, 
conducted by their maintainers through own channels?

Not sure how much value is in exact numbers of users of Stack or Cabal. Both 
groups are large enough. The maintainers of both groups are aware about usage 
stats.

Is either library likely to be influenced by this survey?
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] Creating a new @haskell.org mailing list?

[Simon Peyton Jones via Haskell-community]

Good idea.   “k12” is rather USA specific. What about 
educat...@haskell.org?

Simon

From: Haskell-community  On Behalf Of 
Chris Smith
Sent: 22 October 2018 15:32
To: Haskell-community 
Subject: [Haskell-community] Creating a new @haskell.org mailing list?

Hey,

Is there a process to request a new mailing list on the 
haskell.org
 domain?

Here's my use case.  About 25 Haskell programmers met at ICFP to discuss uses 
of Haskell in K-12 education (for non-US readers, that means before 
university).  I'm also in touch with another half-dozen people who either have 
done, or are doing, something pre-university with Haskell, but could not be at 
ICFP.  The main result of our conversation was that we wanted a common place to 
discuss, report on our experiences, look for productive collaborations and 
common threads, etc.  There are already a few project-specific places, e.g. the 
codeworld-discuss mailing list for my own project, but we were explicitly 
looking for something general-purpose and universal.  It would be great if this 
could be, say, "k...@haskell.org" or something like 
that.

I'm pretty open in terms of how we'd administer the list.  I'm willing to do 
the work of handling obvious spam bots and things like that.  If there's a 
feeling we'd need something more than that, then let's have that discussion.  
We explicitly don't want a strict topicality enforcement, though.  For example, 
several people who attended the dinner at ICFP were also interested in 
functional programming for non-majors at the university level, or were using 
Elm and other Haskell-like languages - even a few people from the Racket 
community.  I'd hope to rely on the name of the mailing list to keep things a 
bit focused, but not really police it at all.

Thoughts?

Thanks,
Chris Smith
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] 2018 state of Haskell survey

[Simon Peyton Jones via Haskell-community]

| Hi Taylor. I like the way you pose things here: "I don't expect that
| to remove selection bias, but it will let me (us, really) say: We're
| doing this together for the benefit of all sides". I think that's a
| better place to start from.

I like this too -- and like Gershom, I'd delete "sides".  We aspire
to work together, not on different sides.

| earlier I've been thinking about a bit, where you wrote: "My goal is
| for this survey to be *the* authoritative Haskell survey and for the
| community to broadly accept it's results." 

This sounds a bit too exclusive to me, and implicitly critical of other
work.  Better to stick to the positives: you simply want the
opinions of a broad constituency on a broad range of questions.

| Anyway, this is all a long-winded way of suggesting that it might be
| good if the purpose of the survey was explicitly set out as trying to
| inform developers of haskell libraries and tools (and educational
| materials) regarding the systems their potential users work on and
| develop, and their habits and practices in doing so, and where they
| encounter difficulty. That is explicitly as a way of learning rather
| than as any sort of horse-race or popularity contest.

That sounds good to me -- but again in drafting the goals I'd stick 
to the positives, and not speak about horse-races.

Simon
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] 2018 state of Haskell survey

[Simon Peyton Jones via Haskell-community]

Taylor

On the GHC side, as I say we are going to do a 1-question GHC survey shortly, 
so you don't need to bother about that one.  (I think it'd be too buried as one 
question among many in your survey.)

| - Reaction to the new pace of GHC releases

That would be interesting, yes.

It would be interesting to know people's perceptions of the (relatively new) 
GHC Proposals process.  Do they even know about it?  Do they follow what is 
going on?  Does the greater transparency and opportunity to contribute makes 
them feel a greater sense of ownership?

I also wonder if they feel included or excluded in our shared enterprise of 
making GHC a better tool.  

Simon

| -Original Message-
| From: Taylor Fausak 
| Sent: 16 October 2018 00:43
| To: Simon Peyton Jones ; Gershom B
| ; Mathieu Boespflug ; Ben Gamari
| 
| Cc: haskell-community@haskell.org
| Subject: Re: [Haskell-community] 2018 state of Haskell survey
| 
| Thanks for the kind words, Simon! They mean a lot :)
| 
| I would be happy to include questions that would benefit the GHC team,
| including:
| 
| - Simon's question
| - Reaction to the new pace of GHC releases
| - Target of the GHC team's focus: performance, features, ergonomics, etc.
| - Average wait time before upgrading GHC
| 
| Are there any other questions the GHC team would be interested in asking?
| Perhaps I should ask on a different mailing list.
| 
| In response to Gershom's comments:
| 
| 1. Asking how people heard about the survey is a great idea. Not only would
| it let me identify the best ways to reach people, it could also be useful
| in dealing with selection bias.
| 
| 2. Addressed above.
| 
| 3. In general, distinguishing between work and home is something I would
| love to do for basically every question. Unfortunately I think that would
| balloon the size of the survey. Maybe identifying a few key questions for
| the work/home split would be the best way to go? Build systems, as you
| identified, are certainly one of those key questions. Maybe GHC versions
| used is another?
| 
| 4. I also like Go's survey and have been trying to crib as much as I can
| from it. Questions worth asking:
| - Area of development (web, embedded, etc.)
| - Type of development (server, CLI, desktop, library, etc.)
| - Deploy environments / infrastructure
| - Internal versus external
| 
| 5. Asking about JS solutions for web developers is a great idea! I like the
| choices you've given, and there are a whole slew of JS libraries to include
| as well, such as React or Vue.
| 
| 6. Giving multiple choice answers to the "why did you stop" question (and,
| in fact, as many questions as possible) is awesome and would make the
| results much easier to digest. It also makes things easier to compare
| across time, which could be used to gauge the effectiveness of various
| endeavors.
| 
| Thank you all for your feedback so far! I am very excited about this year's
| survey. I want to include as many useful questions as I can without
| overwhelming respondents. As I continue to develop the survey, I constantly
| ask myself this question: "How would I act on responses to this question?"
| For example, last year's survey asked if people had contributed to an open
| source Haskell project. I suspect I will exclude that question because it's
| not really actionable.
| 
| With regards to Haskell.org sponsorship, I still think that throwing around
| the words "official" and "Haskell.org" would do a lot in terms of
| credibility. I don't expect that to remove selection bias, but it will let
| me (us, really) say: We're doing this together for the benefit of all
| sides. And if people have problems with the survey, I want them to feel
| comfortable trying to fix those problems, even if they're not on my "side".
| 
| On Mon, Oct 15, 2018, at 3:55 AM, Simon Peyton Jones wrote:
| > The GHC team (coincidentally) is cooking up a very short survey
| > that is intended to help guide our strategic priorities.
| >
| > There is only one substantial question:
| >
| > Imagine that you had one developer working on GHC for
| > six months full-time, that you were paying for yourself.
| > What would you ask that person to do?   Keep in mind that
| > the project should be within GHC itself and tractable
| > in a six-month time-frame.
| >
| > It has a rather different purpose to Taylor's proposed survey
| > and FP Complete's, because it is not comprehensive, but instead
| > focuses on a single question about a single artefact.
| >
| > Returning to Taylor's question list, yes, it would be interesting to
| > know whether the increased release tempo is perceived as helpful or
| > unhelpful.  (It's helpful for the development /process/ because it makes
| > reduces the pressure to squeeze "just one more thing" into a release:
| > the next bus will be along in only 6 months.)
| >
| > Incidentally, I for one think that it's a Good Thing that you included
| > FP Complete's survey in the HWN, Taylor.  First, I think 

Re: [Haskell-community] 2018 state of Haskell survey

[Simon Peyton Jones via Haskell-community]

The GHC team (coincidentally) is cooking up a very short survey
that is intended to help guide our strategic priorities.

There is only one substantial question:

Imagine that you had one developer working on GHC for 
six months full-time, that you were paying for yourself.
What would you ask that person to do?   Keep in mind that
the project should be within GHC itself and tractable
in a six-month time-frame.

It has a rather different purpose to Taylor's proposed survey
and FP Complete's, because it is not comprehensive, but instead
focuses on a single question about a single artefact.

Returning to Taylor's question list, yes, it would be interesting to know 
whether the increased release tempo is perceived as helpful or unhelpful.  
(It's helpful for the development /process/ because it makes reduces the 
pressure to squeeze "just one more thing" into a release: the next bus will be 
along in only 6 months.)

Incidentally, I for one think that it's a Good Thing that you included FP 
Complete's survey in the HWN, Taylor.  First, I think it's a substantial and 
interesting piece of work -- and /any/ survey is vulnerable to response bias.  
Second, I don’t think anyone should expect you as HWN editor to play a role as 
community censor. Third, deliberately excluding it would in itself be a 
divisive act in a community that needs less division and more love.

You do a fantastic job with HWN.  Please keep doing it!

Thanks

Simon

| -Original Message-
| From: Haskell-community  On
| Behalf Of Gershom B
| Sent: 15 October 2018 01:56
| To: tay...@fausak.me; Mathieu Boespflug ; Ben Gamari
| 
| Cc: haskell-community@haskell.org
| Subject: Re: [Haskell-community] 2018 state of Haskell survey
| 
| (cc mathieu boespflug, ben gamari)
| 
| One more thought:
| 
| mathieu, ben -- do you think you would be interested in any questions
| on the frequency of ghc releases -- if people appreciate more
| frequent, smaller releases, or not, etc?
| 
| I wonder if there are any other questions as well about core libraries
| vs. performance vs. "big features" (like type system things) vs. small
| ergonomic features etc. that the core ghc team might be interested in
| sounding out people on, bearing in mind the necessary limitations of
| survey derived data.
| 
| --g
| On Sun, Oct 14, 2018 at 8:36 PM Gershom B  wrote:
| >
| > Hi Taylor.
| >
| > We're discussing this in the committee. I agree that to the extent
| > they can accurately reflect something, language surveys are useful,
| > and appreciate that you want to run a useful survey, and certainly
| > want to encourage and help you in making it as broad and useful as
| > possible. That said, I don't know if slapping a "haskell.org" label on
| > the survey will help manage the biggest drivers of selection bias --
| > which is not only about who chooses to respond, but about who is
| > reached through what mechanisms. (I don't know the relative importance
| > of response-bias vs. outreach-bias in general, and would be curious if
| > somebody has some good research on that to point to). I honestly don't
| > know if we have enough channels _in general_ to do a good survey, no
| > matter who runs it or how at all! Regardless of the decision we come
| > to, here are a few of my personal thoughts on the questions you have
| > thus far, and what could be added:
| >
| > 1) A question "how did you hear about this survey" -- this could at
| > least help to disentangle outreach-bias, or notice it, depending on if
| > it induces any correlations.
| >
| > 2) A question on how long after a new GHC release users upgrade --
| > both personally, and at work.
| >
| > 3) Distinguishing between personal and work build-systems in the
| > relevant question.
| >
| > 4) I think the sorts of questions that make sense to ask in this early
| > part can resemble those in the first part of the Go survey:
| >
| https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.go
| lang.org%2Fsurvey2017-
| resultsdata=02%7C01%7Csimonpj%40microsoft.com%7C9fbbfac1fd9e420bbf2
| 008d63238f7c4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6367516175958
| 02229sdata=2wB5Ph5%2Be6wZm0CWP7Yzx%2Fxe4dOlKHqNPKZReiJmE50%3Dr
| eserved=0 (I especially like the
| > questions about the area of development and server vs cli apps vs
| > libraries). I also like their questions about what environments teams
| > deploy programs to. It would also be worth asking if the apps
| > developed are customer-facing or internal.
| >
| > 5) I think an interesting question would be what preferred js
| > solution, if any, teams adopt -- i.e. ghcjs, typescript, purescript,
| > raw js, etc.
| >
| > 6)  for the "why did you stop" question, there are a good range of
| > potential multi-choice answers that can be drawn from with the rust
| > user survey:
| https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.ru
| st-lang.org%2F2017%2F09%2F05%2FRust-2017-Survey-
|