GHC 7.8 release?

[Simon Peyton-Jones]

Dear GHC users,

Carter: Will this RTS update make it into ghc 7.8 update thats coming up in the 
next monthish?
Andreas: We are almost there - we are now trying to sort out a problem on mac 
os x. It would be helpful to know if there is a cutoff date for getting things 
into 7.8.

Simon, Ian, and I have just been discussing 7.8, and would be interested in 
what you guys think.

At ICFP we speculated that we'd make a release of GHC soon after Christmas to 
embody tons of stuff that has been included since 7.6, specifically:

· major improvements in DPH (vectorisation avoidance, new vectoriser)

· type holes

· rebindable list syntax

· major changes to the type inference engine

· type level natural numbers

· overlapping type families

· the new code generator

· support for vector (SSE/AVX) instructions

Whenever it comes it would definitely be great to include Andreas & friends' 
work:

· Scheduler changes to the RTS to improve latency

The original major reason for proposing a post-Xmas release was to get DPH in a 
working state out into the wild.  However, making a proper release imposes 
costs on everyone else.  Library authors have to scurry around to make their 
libraries work, etc.   Some of the new stuff hasn't been in HEAD for that long, 
and hence has not been very thoroughly tested.   (But of course making a 
release unleashes a huge wave of testing that doesn't happen otherwise.)

So another alternative is to leave it all as HEAD, and wait another few months 
before making a release.  You can still use all the new stuff by compiling 
HEAD, or grabbing a snapshot distribution.  And it makes it hard for the 
Haskell platform if GHC moves too fast. Many people are still on 7.4.

There seem to be pros and cons each way.  I don't have a strong opinion.  If 
you have a view, let us know.

Simon

___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: GHC 7.8 release?

[José Pedro Magalhães]

For the record, if we decide for a release soon, I'll make sure the
new-typeable branch gets merged asap.


Cheers,
Pedro

On Thu, Feb 7, 2013 at 8:25 AM, Simon Peyton-Jones wrote:

>  Dear GHC users,  
>
> *
> *
>
> *Carter*: Will this RTS update make it into ghc 7.8 update thats coming
> up in the next monthish?
>
> *Andreas*: We are almost there - we are now trying to sort out a problem
> on mac os x. It would be helpful to know if there is a cutoff date for
> getting things into 7.8. 
>
> ** **
>
> Simon, Ian, and I have just been discussing 7.8, and would be interested
> in what you guys think.  
>
>
> At ICFP we speculated that we’d make a release of GHC soon after Christmas
> to embody tons of stuff that has been included since 7.6, specifically:***
> *
>
> **· **major improvements in DPH (vectorisation avoidance, new
> vectoriser)
>
> **· **type holes
>
> **· **rebindable list syntax
>
> **· **major changes to the type inference engine
>
> **· **type level natural numbers
>
> **· **overlapping type families
>
> **· **the new code generator
>
> **· **support for vector (SSE/AVX) instructions
>
> ** **
>
> Whenever it comes it would definitely be great to include Andreas &
> friends’ work:
>
> **· **Scheduler changes to the RTS to improve latency
>
> ** **
>
> The original major reason for proposing a post-Xmas release was to get DPH
> in a working state out into the wild.  However, making a proper release
> imposes costs on everyone else.  Library authors have to scurry around to
> make their libraries work, etc.   Some of the new stuff hasn’t been in HEAD
> for that long, and hence has not been very thoroughly tested.   (But of
> course making a release unleashes a huge wave of testing that doesn’t
> happen otherwise.)
>
> ** **
>
> So another alternative is to leave it all as HEAD, and wait another few
> months before making a release.  You can still use all the new stuff by
> compiling HEAD, or grabbing a snapshot distribution.  And it makes it hard
> for the Haskell platform if GHC moves too fast. Many people are still on
> 7.4.
>
> ** **
>
> There seem to be pros and cons each way.  I don’t have a strong opinion.
> If you have a view, let us know.
>
> ** **
>
> Simon
>
> ** **
>
> --
> You received this message because you are subscribed to the Google Groups
> "parallel-haskell" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to parallel-haskell+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: GHC 7.8 release?

[Roman Cheplyaka]

* Simon Peyton-Jones  [2013-02-07 08:25:10+]
> So another alternative is to leave it all as HEAD, and wait another
> few months before making a release.  You can still use all the new
> stuff by compiling HEAD, or grabbing a snapshot distribution.  And it
> makes it hard for the Haskell platform if GHC moves too fast. Many
> people are still on 7.4.

Maybe make a release candidate, as was done with 7.6.2?

Roman

___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: GHC 7.8 release?

[Geoffrey Mainland]

In practice the versions of GHC that are widely used are those that are
included in the platform. Maybe we should coordinate with their next
release? They are targeting a May 6 release, and the release process is
starting March 4, so it sounds like the original GHC release plan
(February release) would be a good fit for the platform as it would
allow library writers to catch up and ensure that STABLE was tested
enough for inclusion in the platform. It would be a shame to miss the
platform release.

Geoff

On 02/07/2013 08:25 AM, Simon Peyton-Jones wrote:
> Dear GHC users, 
> 
> * 
>  
> *
> 
> *Carter*: Will this RTS update make it into ghc 7.8 update thats coming
> up in the next monthish?
> 
> *Andreas*: We are almost there - we are now trying to sort out a problem
> on mac os x. It would be helpful to know if there is a cutoff date for
> getting things into 7.8. 
> 
>  
> 
> Simon, Ian, and I have just been discussing 7.8, and would be interested
> in what you guys think. 
> 
> 
> At ICFP we speculated that we’d make a release of GHC soon after
> Christmas to embody tons of stuff that has been included since 7.6,
> specifically:
> 
> · major improvements in DPH (vectorisation avoidance, new
> vectoriser)
> 
> · type holes
> 
> · rebindable list syntax
> 
> · major changes to the type inference engine
> 
> · type level natural numbers
> 
> · overlapping type families
> 
> · the new code generator
> 
> · support for vector (SSE/AVX) instructions
> 
>  
> 
> Whenever it comes it would definitely be great to include Andreas &
> friends’ work:
> 
> · Scheduler changes to the RTS to improve latency
> 
>  
> 
> The original major reason for proposing a post-Xmas release was to get
> DPH in a working state out into the wild.  However, making a proper
> release imposes costs on everyone else.  Library authors have to scurry
> around to make their libraries work, etc.   Some of the new stuff hasn’t
> been in HEAD for that long, and hence has not been very thoroughly
> tested.   (But of course making a release unleashes a huge wave of
> testing that doesn’t happen otherwise.)
> 
>  
> 
> So another alternative is to leave it all as HEAD, and wait another few
> months before making a release.  You can still use all the new stuff by
> compiling HEAD, or grabbing a snapshot distribution.  And it makes it
> hard for the Haskell platform if GHC moves too fast. Many people are
> still on 7.4.
> 
>  
> 
> There seem to be pros and cons each way.  I don’t have a strong
> opinion.  If you have a view, let us know.
> 
>  
> 
> Simon



___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: GHC 7.8 release?

[Richard Eisenberg]

Geoff's reasoning seems quite sound.
+1 for February release.

On Feb 7, 2013, at 3:50 AM, Geoffrey Mainland  wrote:

> In practice the versions of GHC that are widely used are those that are
> included in the platform. Maybe we should coordinate with their next
> release? They are targeting a May 6 release, and the release process is
> starting March 4, so it sounds like the original GHC release plan
> (February release) would be a good fit for the platform as it would
> allow library writers to catch up and ensure that STABLE was tested
> enough for inclusion in the platform. It would be a shame to miss the
> platform release.
> 
> Geoff
> 
> On 02/07/2013 08:25 AM, Simon Peyton-Jones wrote:
>> Dear GHC users, 
>> 
>> *
>>   
>> *
>> 
>> *Carter*: Will this RTS update make it into ghc 7.8 update thats coming
>> up in the next monthish?
>> 
>> *Andreas*: We are almost there - we are now trying to sort out a problem
>> on mac os x. It would be helpful to know if there is a cutoff date for
>> getting things into 7.8. 
>> 
>> 
>> 
>> Simon, Ian, and I have just been discussing 7.8, and would be interested
>> in what you guys think. 
>> 
>> 
>> At ICFP we speculated that we’d make a release of GHC soon after
>> Christmas to embody tons of stuff that has been included since 7.6,
>> specifically:
>> 
>> · major improvements in DPH (vectorisation avoidance, new
>> vectoriser)
>> 
>> · type holes
>> 
>> · rebindable list syntax
>> 
>> · major changes to the type inference engine
>> 
>> · type level natural numbers
>> 
>> · overlapping type families
>> 
>> · the new code generator
>> 
>> · support for vector (SSE/AVX) instructions
>> 
>> 
>> 
>> Whenever it comes it would definitely be great to include Andreas &
>> friends’ work:
>> 
>> · Scheduler changes to the RTS to improve latency
>> 
>> 
>> 
>> The original major reason for proposing a post-Xmas release was to get
>> DPH in a working state out into the wild.  However, making a proper
>> release imposes costs on everyone else.  Library authors have to scurry
>> around to make their libraries work, etc.   Some of the new stuff hasn’t
>> been in HEAD for that long, and hence has not been very thoroughly
>> tested.   (But of course making a release unleashes a huge wave of
>> testing that doesn’t happen otherwise.)
>> 
>> 
>> 
>> So another alternative is to leave it all as HEAD, and wait another few
>> months before making a release.  You can still use all the new stuff by
>> compiling HEAD, or grabbing a snapshot distribution.  And it makes it
>> hard for the Haskell platform if GHC moves too fast. Many people are
>> still on 7.4.
>> 
>> 
>> 
>> There seem to be pros and cons each way.  I don’t have a strong
>> opinion.  If you have a view, let us know.
>> 
>> 
>> 
>> Simon
> 
> 
> 
> ___
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users@haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
> 


___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: GHC 7.8 release?

[Ian Lynagh]

I'm not too optimistic we could actually get the final release out
during February, assuming we want to allow a couple of weeks for people
to test an RC.

Does the Haskell Platform actually want to commit to using a GHC release
with "tons of [new] stuff", that has had little testing, days or weeks
after its release? I thought the idea was that it would favour
known-good releases over the latest-and-greatest, but perhaps I
misunderstood or the philosophy has changed.


Thanks
Ian

On Thu, Feb 07, 2013 at 09:00:37AM -0500, Richard Eisenberg wrote:
> Geoff's reasoning seems quite sound.
> +1 for February release.
> 
> On Feb 7, 2013, at 3:50 AM, Geoffrey Mainland  wrote:
> 
> > In practice the versions of GHC that are widely used are those that are
> > included in the platform. Maybe we should coordinate with their next
> > release? They are targeting a May 6 release, and the release process is
> > starting March 4, so it sounds like the original GHC release plan
> > (February release) would be a good fit for the platform as it would
> > allow library writers to catch up and ensure that STABLE was tested
> > enough for inclusion in the platform. It would be a shame to miss the
> > platform release.
> > 
> > Geoff
> > 
> > On 02/07/2013 08:25 AM, Simon Peyton-Jones wrote:
> >> Dear GHC users, 
> >> 
> >> *  
> >> 
> >> *
> >> 
> >> *Carter*: Will this RTS update make it into ghc 7.8 update thats coming
> >> up in the next monthish?
> >> 
> >> *Andreas*: We are almost there - we are now trying to sort out a problem
> >> on mac os x. It would be helpful to know if there is a cutoff date for
> >> getting things into 7.8. 
> >> 
> >> 
> >> 
> >> Simon, Ian, and I have just been discussing 7.8, and would be interested
> >> in what you guys think. 
> >> 
> >> 
> >> At ICFP we speculated that we’d make a release of GHC soon after
> >> Christmas to embody tons of stuff that has been included since 7.6,
> >> specifically:
> >> 
> >> · major improvements in DPH (vectorisation avoidance, new
> >> vectoriser)
> >> 
> >> · type holes
> >> 
> >> · rebindable list syntax
> >> 
> >> · major changes to the type inference engine
> >> 
> >> · type level natural numbers
> >> 
> >> · overlapping type families
> >> 
> >> · the new code generator
> >> 
> >> · support for vector (SSE/AVX) instructions
> >> 
> >> 
> >> 
> >> Whenever it comes it would definitely be great to include Andreas &
> >> friends’ work:
> >> 
> >> · Scheduler changes to the RTS to improve latency
> >> 
> >> 
> >> 
> >> The original major reason for proposing a post-Xmas release was to get
> >> DPH in a working state out into the wild.  However, making a proper
> >> release imposes costs on everyone else.  Library authors have to scurry
> >> around to make their libraries work, etc.   Some of the new stuff hasn’t
> >> been in HEAD for that long, and hence has not been very thoroughly
> >> tested.   (But of course making a release unleashes a huge wave of
> >> testing that doesn’t happen otherwise.)
> >> 
> >> 
> >> 
> >> So another alternative is to leave it all as HEAD, and wait another few
> >> months before making a release.  You can still use all the new stuff by
> >> compiling HEAD, or grabbing a snapshot distribution.  And it makes it
> >> hard for the Haskell platform if GHC moves too fast. Many people are
> >> still on 7.4.
> >> 
> >> 
> >> 
> >> There seem to be pros and cons each way.  I don’t have a strong
> >> opinion.  If you have a view, let us know.
> >> 
> >> 
> >> 
> >> Simon

___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

RE: GHC 7.8 release?

[p.k.f.holzenspies]

+1

Ph.


> -Original Message-
> From: glasgow-haskell-users-boun...@haskell.org [mailto:glasgow-haskell-
> users-boun...@haskell.org] On Behalf Of Richard Eisenberg
> Sent: donderdag 7 februari 2013 15:01
> To: Geoffrey Mainland
> Cc: parallel-hask...@googlegroups.com; glasgow-haskell-users@haskell.org;
> ghc-d...@haskell.org
> Subject: Re: GHC 7.8 release?
> 
> Geoff's reasoning seems quite sound.
> +1 for February release.
> 
> On Feb 7, 2013, at 3:50 AM, Geoffrey Mainland 
> wrote:
> 
> > In practice the versions of GHC that are widely used are those that are
> > included in the platform. Maybe we should coordinate with their next
> > release? They are targeting a May 6 release, and the release process is
> > starting March 4, so it sounds like the original GHC release plan
> > (February release) would be a good fit for the platform as it would
> > allow library writers to catch up and ensure that STABLE was tested
> > enough for inclusion in the platform. It would be a shame to miss the
> > platform release.
> >
> > Geoff
> >
> > On 02/07/2013 08:25 AM, Simon Peyton-Jones wrote:
> >> Dear GHC users,
> >>
> >> *
> >> *
> >>
> >> *Carter*: Will this RTS update make it into ghc 7.8 update thats coming
> >> up in the next monthish?
> >>
> >> *Andreas*: We are almost there - we are now trying to sort out a
> problem
> >> on mac os x. It would be helpful to know if there is a cutoff date for
> >> getting things into 7.8.
> >>
> >>
> >>
> >> Simon, Ian, and I have just been discussing 7.8, and would be interested
> >> in what you guys think.
> >>
> >>
> >> At ICFP we speculated that we'd make a release of GHC soon after
> >> Christmas to embody tons of stuff that has been included since 7.6,
> >> specifically:
> >>
> >> * major improvements in DPH (vectorisation avoidance, new
> >> vectoriser)
> >>
> >> * type holes
> >>
> >> * rebindable list syntax
> >>
> >> * major changes to the type inference engine
> >>
> >> * type level natural numbers
> >>
> >> * overlapping type families
> >>
> >> * the new code generator
> >>
> >> * support for vector (SSE/AVX) instructions
> >>
> >>
> >>
> >> Whenever it comes it would definitely be great to include Andreas &
> >> friends' work:
> >>
> >> * Scheduler changes to the RTS to improve latency
> >>
> >>
> >>
> >> The original major reason for proposing a post-Xmas release was to get
> >> DPH in a working state out into the wild.  However, making a proper
> >> release imposes costs on everyone else.  Library authors have to scurry
> >> around to make their libraries work, etc.   Some of the new stuff hasn't
> >> been in HEAD for that long, and hence has not been very thoroughly
> >> tested.   (But of course making a release unleashes a huge wave of
> >> testing that doesn't happen otherwise.)
> >>
> >>
> >>
> >> So another alternative is to leave it all as HEAD, and wait another few
> >> months before making a release.  You can still use all the new stuff by
> >> compiling HEAD, or grabbing a snapshot distribution.  And it makes it
> >> hard for the Haskell platform if GHC moves too fast. Many people are
> >> still on 7.4.
> >>
> >>
> >>
> >> There seem to be pros and cons each way.  I don't have a strong
> >> opinion.  If you have a view, let us know.
> >>
> >>
> >>
> >> Simon
> >
> >
> >
> > ___
> > Glasgow-haskell-users mailing list
> > Glasgow-haskell-users@haskell.org
> > http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
> >
> 
> 
> ___
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users@haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: GHC 7.8 release?

[John Lato]

I agree with Ian.  Mid-February is very soon, and there's a lot of stuff
that seems to just be coming in now.  That doesn't leave much time for
testing to get 7.8 out in sync with the platform.

Although my perspective is a bit colored by the last release.  Testing the
7.6.1 RC took several weeks for us because of the number of upstream
packages that needed to be updated (not all trivially).  By the time we
were prepared to begin testing our own systems 7.6.1 was already released,
and we couldn't use it because of a number of bugs (
http://hackage.haskell.org/trac/ghc/ticket/7257 was a blocker, but there
were others also).  Most of the bugs were fixed very quickly (thanks Simon
M. and Simon PJ!), but by then they were already in the wild.  If there had
been a bit more time to test 7.6.1, maybe some of those fixes would have
made it into the release.


John L.


On Thu, Feb 7, 2013 at 10:23 PM, Ian Lynagh  wrote:

>
> I'm not too optimistic we could actually get the final release out
> during February, assuming we want to allow a couple of weeks for people
> to test an RC.
>
> Does the Haskell Platform actually want to commit to using a GHC release
> with "tons of [new] stuff", that has had little testing, days or weeks
> after its release? I thought the idea was that it would favour
> known-good releases over the latest-and-greatest, but perhaps I
> misunderstood or the philosophy has changed.
>
>
> Thanks
> Ian
>
> On Thu, Feb 07, 2013 at 09:00:37AM -0500, Richard Eisenberg wrote:
> > Geoff's reasoning seems quite sound.
> > +1 for February release.
> >
> > On Feb 7, 2013, at 3:50 AM, Geoffrey Mainland 
> wrote:
> >
> > > In practice the versions of GHC that are widely used are those that are
> > > included in the platform. Maybe we should coordinate with their next
> > > release? They are targeting a May 6 release, and the release process is
> > > starting March 4, so it sounds like the original GHC release plan
> > > (February release) would be a good fit for the platform as it would
> > > allow library writers to catch up and ensure that STABLE was tested
> > > enough for inclusion in the platform. It would be a shame to miss the
> > > platform release.
> > >
> > > Geoff
> > >
> > > On 02/07/2013 08:25 AM, Simon Peyton-Jones wrote:
> > >> Dear GHC users,
> > >>
> > >> *
> > >> *
> > >>
> > >> *Carter*: Will this RTS update make it into ghc 7.8 update thats
> coming
> > >> up in the next monthish?
> > >>
> > >> *Andreas*: We are almost there - we are now trying to sort out a
> problem
> > >> on mac os x. It would be helpful to know if there is a cutoff date for
> > >> getting things into 7.8.
> > >>
> > >>
> > >>
> > >> Simon, Ian, and I have just been discussing 7.8, and would be
> interested
> > >> in what you guys think.
> > >>
> > >>
> > >> At ICFP we speculated that we’d make a release of GHC soon after
> > >> Christmas to embody tons of stuff that has been included since 7.6,
> > >> specifically:
> > >>
> > >> · major improvements in DPH (vectorisation avoidance, new
> > >> vectoriser)
> > >>
> > >> · type holes
> > >>
> > >> · rebindable list syntax
> > >>
> > >> · major changes to the type inference engine
> > >>
> > >> · type level natural numbers
> > >>
> > >> · overlapping type families
> > >>
> > >> · the new code generator
> > >>
> > >> · support for vector (SSE/AVX) instructions
> > >>
> > >>
> > >>
> > >> Whenever it comes it would definitely be great to include Andreas &
> > >> friends’ work:
> > >>
> > >> · Scheduler changes to the RTS to improve latency
> > >>
> > >>
> > >>
> > >> The original major reason for proposing a post-Xmas release was to get
> > >> DPH in a working state out into the wild.  However, making a proper
> > >> release imposes costs on everyone else.  Library authors have to
> scurry
> > >> around to make their libraries work, etc.   Some of the new stuff
> hasn’t
> > >> been in HEAD for that long, and hence has not been very thoroughly
> > >> tested.   (But of course making a release unleashes a huge wave of
> > >> testing that doesn’t happen otherwise.)
> > >>
> > >>
> > >>
> > >> So another alternative is to leave it all as HEAD, and wait another
> few
> > >> months before making a release.  You can still use all the new stuff
> by
> > >> compiling HEAD, or grabbing a snapshot distribution.  And it makes it
> > >> hard for the Haskell platform if GHC moves too fast. Many people are
> > >> still on 7.4.
> > >>
> > >>
> > >>
> > >> There seem to be pros and cons each way.  I don’t have a strong
> > >> opinion.  If you have a view, let us know.
> > >>
> > >>
> > >>
> > >> Simon
>
> ___
> Haskell-platform mailing list
> haskell-platf...@projects.haskell.org
> http://projects.haskell.org/cgi-bin/mailman/listinfo/haskell-platform
>
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://w

RE: GHC 7.8 release?

[Simon Peyton-Jones]

It’s fairly simple in my mind. There are two “channels” (if I understand Mark’s 
terminology right):


· Haskell Platform:

o   A stable development environment, lots of libraries known to work

o   Newcomers, and people who value stability, should use the Haskell Platform

o   HP comes with a particular version of GHC, probably not the hottest new 
one, but that doesn’t matter.  It works.



· GHC home page downloads:

o   More features but not so stable

o   Libraries not guaranteed to work

o   Worth releasing, though, as a forcing function to fix bugs, and as a 
checkpoint for people to test, so that by the time the HP adopts a particular 
version it is reasonably solid.

So we already have the two channels Mark asks for, don’t we?  One is called the 
Haskell Platform and one is called the GHC home page.

That leaves a PR issue: we really don’t want newcomers or Joe Users wanting the 
“new shiny”. They want the Haskell Platform, and as Mark says those users 
should not pay the slightest attention until it appears in the Haskell Platform.

So perhaps we principally need a way to point people away from GHC and towards 
HP?  eg We could prominently say at every download point “Stop!  Are you sure 
you want this?  You might be better off with the Haskell Platform!  Here’s 
why...”.

Have I understood aright?  If so, how could we achieve the right social 
dynamics?

Our goal is to let people who value stability get stability, while the 
hot-shots race along in a different channel and pay the price of flat tires etc.

PS: absolutely right to use 7.6.2 for the next HP.  Don’t even think about 7.8.

Simon



From: Mark Lentczner [mailto:mark.lentcz...@gmail.com]
Sent: 07 February 2013 17:43
To: Simon Peyton-Jones
Cc: andreas.voel...@gmail.com; Carter Schonwald; GHC users; Simon Marlow; 
parallel-haskell; kosti...@gmail.com; Edsko de Vries; ghc-d...@haskell.org
Subject: Re: GHC 7.8 release?

I'd say the window for 7.8 in the platform is about closed. If 7.8 were to be 
release in the next two weeks that would be just about the least amount of time 
I'd want to see for libraries in the platform to get all stable with the GHC 
version. And we'd also be counting on the GHC team to be quickly responding to 
bugs so that there could be a point release of 7.8 mid-April. Historically, 
none of that seems likely.

So my current trajectory is to base HP 2013.2.0.0 on GHC 7.6.2.

Since 7.8 will seems like it will be released before May, we will be faced 
again with the bad public relations issue: Everyone will want the new shiny and 
be confused as to why the platform is such a laggard. We'll see four reactions:

  *   New comers who are starting out and figure they should use the latest... 
Many will try to use 7.8, half the libraries on hackage won't work, things will 
be wonky, and they'll have a poor experience.
  *   People doing production / project work will stay on 7.6 and ignore 7.8 
for a few months.
  *   The small group of people exploring the frontiers will know how to get 
things set up and be happy.
  *   Eventually library authors will get around to making sure their stuff 
will work with it.
I wish GHC would radically change it's release process. Things like 7.8 
shouldn't be release as "7.8". That sounds major and stable. The web site will 
have 7.8 at the top. The warning to use the platform will fall flat because it 
makes the platform look out of date. Really, "7.8" should be in a different 
release channel, not on the front page. It should bake in that channel for six 
months - where only the third group of people will use it, until it is getting 
close to merge into main, at which point the fourth group will start to use it, 
so that the day it hits main, all the libraries just work. Ideally, the first 
two groups of people will not pay the slightest attention to it until it is 
further baked.

While we achievements of the GHC team are great, less than half of those 7.8 
features seem interesting from the viewpoint of the aims of the platform. I 
don't think adding syntactic or type-level features are really all that 
important for Haskell at this juncture. And while I do like to see improvements 
in generated code and run-time performance, I think even those are less 
important than making crucial ecosystem improvements to things like package 
management, cross-compilation, and libraries.

- Mark
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: GHC 7.8 release?

[Christian Höner zu Siederdissen]

Hi Simon,

The download page already has a big "Stop" there.
http://www.haskell.org/ghc/download_ghc_7_6_2

Apart from that, I am /really/ looking forward to sse/avx extensions and
the official new-code-gen to further narrow the gap between
high-performance C and high-performance Haskell.

That being said, I would be fine using HEAD, but a release is very
convenient in terms of installing, even if they are in the form of "rc"
(which I typically install to see if something breaks or is faster).

Maybe you want to consider providing a couple of release candidates
instead of 7.8 now?

Gruss,
Christian

* Simon Peyton Jones  [07.02.2013 19:25]:
>It's fairly simple in my mind. There are two "channels" (if I understand
>Mark's terminology right):
> 
> 
> 
>. Haskell Platform:
> 
>o   A stable development environment, lots of libraries known to work
> 
>o   Newcomers, and people who value stability, should use the Haskell
>Platform
> 
>o   HP comes with a particular version of GHC, probably not the hottest
>new one, but that doesn't matter.  It works.
> 
> 
> 
>. GHC home page downloads:
> 
>o   More features but not so stable
> 
>o   Libraries not guaranteed to work
> 
>o   Worth releasing, though, as a forcing function to fix bugs, and as a
>checkpoint for people to test, so that by the time the HP adopts a
>particular version it is reasonably solid.
> 
> 
> 
>So we already have the two channels Mark asks for, don't we?  One is
>called the Haskell Platform and one is called the GHC home page. 
> 
>That leaves a PR issue: we really don't want newcomers or Joe Users
>wanting the "new shiny". They want the Haskell Platform, and as Mark says
>those users should not pay the slightest attention until it appears in the
>Haskell Platform.
> 
> 
> 
>So perhaps we principally need a way to point people away from GHC and
>towards HP?  eg We could prominently say at every download point "Stop! 
>Are you sure you want this?  You might be better off with the Haskell
>Platform!  Here's why...".
> 
> 
> 
>Have I understood aright?  If so, how could we achieve the right social
>dynamics? 
> 
> 
> 
>Our goal is to let people who value stability get stability, while the
>hot-shots race along in a different channel and pay the price of flat
>tires etc.
> 
> 
> 
>PS: absolutely right to use 7.6.2 for the next HP.  Don't even think about
>7.8.
> 
> 
> 
>Simon
> 
> 
> 
> 
> 
> 
> 
>From: Mark Lentczner [mailto:mark.lentcz...@gmail.com]
>Sent: 07 February 2013 17:43
>To: Simon Peyton-Jones
>Cc: andreas.voel...@gmail.com; Carter Schonwald; GHC users; Simon Marlow;
>parallel-haskell; kosti...@gmail.com; Edsko de Vries; ghc-d...@haskell.org
>Subject: Re: GHC 7.8 release?
> 
> 
> 
>I'd say the window for 7.8 in the platform is about closed. If 7.8 were to
>be release in the next two weeks that would be just about the least amount
>of time I'd want to see for libraries in the platform to get all stable
>with the GHC version. And we'd also be counting on the GHC team to be
>quickly responding to bugs so that there could be a point release of 7.8
>mid-April. Historically, none of that seems likely.
> 
> 
> 
>So my current trajectory is to base HP 2013.2.0.0 on GHC 7.6.2.
> 
> 
> 
>Since 7.8 will seems like it will be released before May, we will be faced
>again with the bad public relations issue: Everyone will want the new
>shiny and be confused as to why the platform is such a laggard. We'll see
>four reactions:
> 
>  o New comers who are starting out and figure they should use the
>latest... Many will try to use 7.8, half the libraries on hackage
>won't work, things will be wonky, and they'll have a poor experience.
>  o People doing production / project work will stay on 7.6 and ignore 7.8
>for a few months.
>  o The small group of people exploring the frontiers will know how to get
>things set up and be happy.
>  o Eventually library authors will get around to making sure their stuff
>will work with it.
> 
>I wish GHC would radically change it's release process. Things like 7.8
>shouldn't be release as "7.8". That sounds major and stable. The web site
>will have 7.8 at the top. The warning to use the platform will fall flat
>because it makes the platform look out of date. Really, "7.8" should be in
>a different release channel, not on the front page. It should bake in that
>channel for six months - where only the third group of people will use it,
>until it is getting close to merge into main, at which point the fourth
>group will start to use it, so that the day it hits main, all the
>libraries just work. Ideally, the first two groups of people will not pay
>the slightest attent

Re: GHC 7.8 release?

[Henrik Nilsson]

Hi all,

Ian Lynagh wrote:

> Does the Haskell Platform actually want to commit to using a GHC
> release with "tons of [new] stuff", that has had little testing, days
> or weeks after its release? I thought the idea was that it would
> favour known-good releases over the latest-and-greatest, but perhaps I
> misunderstood or the philosophy has changed.

From a teaching perspective, I'd hope the philosophy is still
"known-good releases".

The Haskell platform is what we deploy on our teaching machines,
and we really need to be able to trust that it will work very
smoothly, or we'd risk losing lots of valuable teaching time and,
even worse, putting lots of students off Haskell. (Getting students
to appreciate Haskell is an upwards struggle at the best of times
anyway.)

Something like new run-time system features sounds like something
that really ought to be tested very thoroughly before being integrated
into the HP.

So, for (general) teaching, at least, stability over new features any
day.

Best,

/Henrik

--
Henrik Nilsson
School of Computer Science
The University of Nottingham
n...@cs.nott.ac.uk
This message and any attachment are intended solely for the addressee and may 
contain confidential information. If you have received this message in error, 
please send it back to me, and immediately delete it.   Please do not use, copy 
or disclose the information contained in this message or in any attachment.  
Any views or opinions expressed by the author of this email do not necessarily 
reflect the views of the University of Nottingham.

This message has been checked for viruses but the contents of an attachment
may still contain software viruses which could damage your computer system:
you are advised to perform your own checks. Email communications with the
University of Nottingham may be monitored as permitted by UK legislation.

___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: GHC 7.8 release?

[Austin Seipp]

This is a slight tangent but, I am always somewhat confused about the
release schedule. When reading this, the basic decision seems to come
down to when do we cut a release, taking into account factors like
reliability/bugs/support/community/other stuff like that.

So, IMO, perhaps one thing that's needed is a more formalized release
schedule, with something like a merge window for 'big changes'? For
example, many projects like LLVM and GCC have fairly fixed release
windows, with an accompanying merge window several months before an
official release. (The Linux kernel does this too, but they have a
much shorter cycle.) If a large feature misses the merge window, it
must go into the next release.

Personally, I am not too worried about necessarily getting every new
feature into a release, even if they're awesome (and they all are!)
And while giving HP users the latest and greatest is great, they want
stability more than anything, in my opinion. So I think they're fine
with that too. What I am worried about is there being a good length of
time where the features integrated have time to bake and see some
polish, without a lot of interference.

There are a lot of issues with this including how to deal with work
that goes on in the mean time, etc. GHC also has far less manpower and
a much different ratio of developer influence and 'spread' than any of
the above projects. And we have to define what qualifies as 'big
change.' But if the issue seems to be one of time, synchronization,
and quality, perhaps we should think about whether or not a change
like a more formalized schedule could help.

I think making releases so people can find bugs is important. But that
will always happen anyway, so I'd rather be a little cautious and wait
this one out, than try to cram it. The new vectoriser only came in
within the past ~48 hours, and SIMD was just pushed in the past week
(and DPH will need SIMD support merged, too!) I think Feburary or even
March is way, way too early for a solid release, and it's certainly
too late for the HP anyway. I see little pain in postponement,
personally.

On Thu, Feb 7, 2013 at 2:25 AM, Simon Peyton-Jones
 wrote:
> Dear GHC users,
>
>
>
> Carter: Will this RTS update make it into ghc 7.8 update thats coming up in
> the next monthish?
>
> Andreas: We are almost there - we are now trying to sort out a problem on
> mac os x. It would be helpful to know if there is a cutoff date for getting
> things into 7.8.
>
>
>
> Simon, Ian, and I have just been discussing 7.8, and would be interested in
> what you guys think.
>
>
> At ICFP we speculated that we’d make a release of GHC soon after Christmas
> to embody tons of stuff that has been included since 7.6, specifically:
>
> · major improvements in DPH (vectorisation avoidance, new
> vectoriser)
>
> · type holes
>
> · rebindable list syntax
>
> · major changes to the type inference engine
>
> · type level natural numbers
>
> · overlapping type families
>
> · the new code generator
>
> · support for vector (SSE/AVX) instructions
>
>
>
> Whenever it comes it would definitely be great to include Andreas & friends’
> work:
>
> · Scheduler changes to the RTS to improve latency
>
>
>
> The original major reason for proposing a post-Xmas release was to get DPH
> in a working state out into the wild.  However, making a proper release
> imposes costs on everyone else.  Library authors have to scurry around to
> make their libraries work, etc.   Some of the new stuff hasn’t been in HEAD
> for that long, and hence has not been very thoroughly tested.   (But of
> course making a release unleashes a huge wave of testing that doesn’t happen
> otherwise.)
>
>
>
> So another alternative is to leave it all as HEAD, and wait another few
> months before making a release.  You can still use all the new stuff by
> compiling HEAD, or grabbing a snapshot distribution.  And it makes it hard
> for the Haskell platform if GHC moves too fast. Many people are still on
> 7.4.
>
>
>
> There seem to be pros and cons each way.  I don’t have a strong opinion.  If
> you have a view, let us know.
>
>
>
> Simon
>
>
>
>
> ___
> ghc-devs mailing list
> ghc-d...@haskell.org
> http://www.haskell.org/mailman/listinfo/ghc-devs
>



-- 
Regards,
Austin

___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: GHC 7.8 release?

[Ben Lippmeier]

On 08/02/2013, at 5:15 AM, Simon Peyton-Jones wrote:

> So perhaps we principally need a way to point people away from GHC and 
> towards HP?  eg We could prominently say at every download point “Stop!  Are 
> you sure you want this?  You might be better off with the Haskell Platform!  
> Here’s why...”.

Right now, the latest packages uploaded to Hackage get built with ghc-7.6 
(only), and all the pages say "Built on ghc-7.6". By doing this we force *all* 
library developers to run GHC 7.6. I think this sends the clearest message 
about what the "real" GHC version is. 

We'd have more chance of turning Joe User off the latest GHC release if Hackage 
was clearly split into stable/testing channels. Linux distros have been doing 
this for years.

Ben.

___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: GHC 7.8 release?

[Johan Tibell]

On Thu, Feb 7, 2013 at 6:48 PM, Ben Lippmeier  wrote:

> Right now, the latest packages uploaded to Hackage get built with ghc-7.6
> (only), and all the pages say "Built on ghc-7.6". By doing this we force
> *all* library developers to run GHC 7.6. I think this sends the clearest
> message about what the "real" GHC version is.
>

I don't know how closely users look at the "built on" line on the package
page. Perhaps they look there, perhaps they don't.

Between Bryan and me we build all of our own packages and all HP packages
on the latest 3 GHC versions:

http://ci.johantibell.com/
https://jenkins.serpentine.com/

That doesn't mean that it would be a bad idea to build all Hackage packages
on using more GHC versions, but that won't happen until the Hackage project
is unstuck (it has been stuck for years!).

-- Johan
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

RE: Cloud Haskell and network latency issues with -threaded

[Edward Z. Yang]

Hey folks,

The latency changes sound relevant to some work on the scheduler I'm doing;
is there a place I can see the changes?

Thanks,
Edward

Excerpts from Simon Peyton-Jones's message of Wed Feb 06 10:10:10 -0800 2013:
> I (with help from Kazu and helpful comments from Bryan and Johan) have nearly 
> completed an overhaul to the IO manager based on my observations and we are 
> in the final stages of getting it into GHC
> 
> This is really helpful. Thank you very much Andreas, Kazu, Bryan, Johan.
> 
> Simon
> 
> From: parallel-hask...@googlegroups.com 
> [mailto:parallel-hask...@googlegroups.com] On Behalf Of Andreas Voellmy
> Sent: 06 February 2013 14:28
> To: watson.timo...@gmail.com
> Cc: kosti...@gmail.com; parallel-haskell; glasgow-haskell-users@haskell.org
> Subject: Re: Cloud Haskell and network latency issues with -threaded
> 
> Hi all,
> 
> I haven't followed the conversations around CloudHaskell closely, but I 
> noticed the discussion around latency using the threaded runtime system, and 
> I thought I'd jump in here.
> 
> I've been developing a server in Haskell that serves hundreds to thousands of 
> clients over very long-lived TCP sockets. I also had latency problems with 
> GHC. For example, with 100 clients I had a 10 ms (millisecond) latency and 
> with 500 clients I had a 29ms latency. I looked into the problem and found 
> that some bottlenecks in the threaded IO manager were the cause. I made some 
> hacks there and got the latency for 100 and 500 clients down to under 0.2 ms. 
> I (with help from Kazu and helpful comments from Bryan and Johan) have nearly 
> completed an overhaul to the IO manager based on my observations and we are 
> in the final stages of getting it into GHC. Hopefully our work will also fix 
> the latency issues in CloudHaskell programs :)
> 
> It would be very helpful if someone has some benchmark CloudHaskell 
> applications and workloads to test with. Does anyone have these handy?
> 
> Cheers,
> Andi
> 
> On Wed, Feb 6, 2013 at 9:09 AM, Tim Watson 
> mailto:watson.timo...@gmail.com>> wrote:
> Hi Kostirya,
> 
> I'm putting the parallel-haskell and ghc-users lists on cc, just in case 
> other (better informed) folks want to chip in here.
> 
> 
> 
> First of all, I'm assuming you're talking about network latency when 
> compiling with -threaded - if not I apologise for misunderstanding!
> 
> There is apparently an outstanding network latency issue when compiling with 
> -threaded, but according to a conversation I had with the other developers on 
> #haskell-distributed, this is not something that's specific to Cloud Haskell. 
> It is something to do with the threaded runtime system, so would need to be 
> solved for GHC (or is it just the Network package!?) in general. Writing up a 
> simple C program and equivalent socket use in Haskell and comparing the 
> latency using -threaded will show this up.
> 
> See the latency section in 
> http://haskell-distributed.github.com/wiki/networktransport.html for some 
> more details. According to that, there *are* some things we might be able to 
> do, but the 20% latency isn't going to change significantly on the face of 
> things.
> 
> We have an open ticket to look into this 
> (https://cloud-haskell.atlassian.net/browse/NTTCP-4) and at some point we'll 
> try and put together the sample programs in a github repository (if that's 
> not already done - I might've missed previous spikes done by Edsko or others) 
> and investigate further.
> 
> One of the other (more experienced!) devs might be able to chip in and 
> proffer a better explanation.
> 
> Cheers,
> Tim
> 
> On 6 Feb 2013, at 13:27, kosti...@gmail.com wrote:
> 
> > Haven't you had a necessity to launch Haskell in no-threaded mode during 
> > the intense network data exchange?
> > I am getting the double performance penalty in threaded mode. But I must 
> > use threaded mode because epoll and kevent are available in the threaded 
> > mode only.
> >
> 
> [snip]
> 
> >
> >
> > среда, 6 февраля 2013 г., 12:33:36 UTC+2 пользователь Tim Watson написал:
> > Hello all,
> >
> > It's been a busy week for Cloud Haskell and I wanted to share a few of
> > our news items with you all.
> >
> > Firstly, we have a new home page at http://haskell-distributed.github.com,
> > into which most of the documentation and wiki pages have been merged. Making
> > sassy looking websites is not really my bag, so I'm very grateful to the
> > various author's whose Creative Commons licensed designs and layouts made
> > it easy to put together. We've already had some pull requests to fix minor
> > problems on the site, so thanks very much to those who've contributed 
> > already!
> >
> > As well as the new site, you will find a few of us hanging out on the
> > #haskell-distributed channel on freenode. Please do come along and join in
> > the conversation.
> >
> > We also recently split up the distributed-process project into separate
> > git repositories, one

Re: GHC 7.8 release?

[Carter Schonwald]

johan, how do you and Bryan have those jenkin's nodes setup?

(I'm planning  to setup something similar  for my own use, and seeing how
thats setup would be awesome)

thanks
-Carter


On Thu, Feb 7, 2013 at 10:55 PM, Johan Tibell wrote:

> On Thu, Feb 7, 2013 at 6:48 PM, Ben Lippmeier  wrote:
>
>> Right now, the latest packages uploaded to Hackage get built with ghc-7.6
>> (only), and all the pages say "Built on ghc-7.6". By doing this we force
>> *all* library developers to run GHC 7.6. I think this sends the clearest
>> message about what the "real" GHC version is.
>>
>
> I don't know how closely users look at the "built on" line on the package
> page. Perhaps they look there, perhaps they don't.
>
> Between Bryan and me we build all of our own packages and all HP packages
> on the latest 3 GHC versions:
>
> http://ci.johantibell.com/
> https://jenkins.serpentine.com/
>
> That doesn't mean that it would be a bad idea to build all Hackage
> packages on using more GHC versions, but that won't happen until the
> Hackage project is unstuck (it has been stuck for years!).
>
> -- Johan
>
>  --
> You received this message because you are subscribed to the Google Groups
> "parallel-haskell" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to parallel-haskell+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: GHC 7.8 release?

[Carter Schonwald]

johan, how do you and Bryan have those jenkin's nodes setup?

(I'm planning  to setup something similar  for my own use, and seeing how
thats setup would be awesome)

thanks
-Carter


On Thu, Feb 7, 2013 at 10:55 PM, Johan Tibell wrote:

> On Thu, Feb 7, 2013 at 6:48 PM, Ben Lippmeier  wrote:
>
>> Right now, the latest packages uploaded to Hackage get built with ghc-7.6
>> (only), and all the pages say "Built on ghc-7.6". By doing this we force
>> *all* library developers to run GHC 7.6. I think this sends the clearest
>> message about what the "real" GHC version is.
>>
>
> I don't know how closely users look at the "built on" line on the package
> page. Perhaps they look there, perhaps they don't.
>
> Between Bryan and me we build all of our own packages and all HP packages
> on the latest 3 GHC versions:
>
> http://ci.johantibell.com/
> https://jenkins.serpentine.com/
>
> That doesn't mean that it would be a bad idea to build all Hackage
> packages on using more GHC versions, but that won't happen until the
> Hackage project is unstuck (it has been stuck for years!).
>
> -- Johan
>
>  --
> You received this message because you are subscribed to the Google Groups
> "parallel-haskell" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to parallel-haskell+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: Cloud Haskell and network latency issues with -threaded

[Andreas Voellmy]

Hi Edward,

I did two things to improve latency for my application: (1) rework the IO
manager and (2) stabilize the work pushing. (1) seems like a big win and we
are almost done with the work on that part. It is less clear whether (2)
will generally help much. It helped me when I developed it against 7.4.1,
but it doesn't seem to have much impact on HEAD on the few measurements I
did. The idea of (2) was to keep running averages of the run queue length
of each capability, then push work when these running averages get too
out-of-balance. The desired effect (which seems to work on my particular
application) is to avoid cases in which threads are pushed back and forth
among cores, which may make cache usage worse. You can see my patch here:
https://github.com/AndreasVoellmy/ghc-arv/commits/push-work-exchange-squashed
.

-Andi


On Fri, Feb 8, 2013 at 12:10 AM, Edward Z. Yang  wrote:

> Hey folks,
>
> The latency changes sound relevant to some work on the scheduler I'm doing;
> is there a place I can see the changes?
>
> Thanks,
> Edward
>
> Excerpts from Simon Peyton-Jones's message of Wed Feb 06 10:10:10 -0800
> 2013:
> > I (with help from Kazu and helpful comments from Bryan and Johan) have
> nearly completed an overhaul to the IO manager based on my observations and
> we are in the final stages of getting it into GHC
> >
> > This is really helpful. Thank you very much Andreas, Kazu, Bryan, Johan.
> >
> > Simon
> >
> > From: parallel-hask...@googlegroups.com [mailto:
> parallel-hask...@googlegroups.com] On Behalf Of Andreas Voellmy
> > Sent: 06 February 2013 14:28
> > To: watson.timo...@gmail.com
> > Cc: kosti...@gmail.com; parallel-haskell;
> glasgow-haskell-users@haskell.org
> > Subject: Re: Cloud Haskell and network latency issues with -threaded
> >
> > Hi all,
> >
> > I haven't followed the conversations around CloudHaskell closely, but I
> noticed the discussion around latency using the threaded runtime system,
> and I thought I'd jump in here.
> >
> > I've been developing a server in Haskell that serves hundreds to
> thousands of clients over very long-lived TCP sockets. I also had latency
> problems with GHC. For example, with 100 clients I had a 10 ms
> (millisecond) latency and with 500 clients I had a 29ms latency. I looked
> into the problem and found that some bottlenecks in the threaded IO manager
> were the cause. I made some hacks there and got the latency for 100 and 500
> clients down to under 0.2 ms. I (with help from Kazu and helpful comments
> from Bryan and Johan) have nearly completed an overhaul to the IO manager
> based on my observations and we are in the final stages of getting it into
> GHC. Hopefully our work will also fix the latency issues in CloudHaskell
> programs :)
> >
> > It would be very helpful if someone has some benchmark CloudHaskell
> applications and workloads to test with. Does anyone have these handy?
> >
> > Cheers,
> > Andi
> >
> > On Wed, Feb 6, 2013 at 9:09 AM, Tim Watson  > wrote:
> > Hi Kostirya,
> >
> > I'm putting the parallel-haskell and ghc-users lists on cc, just in case
> other (better informed) folks want to chip in here.
> >
> > 
> >
> > First of all, I'm assuming you're talking about network latency when
> compiling with -threaded - if not I apologise for misunderstanding!
> >
> > There is apparently an outstanding network latency issue when compiling
> with -threaded, but according to a conversation I had with the other
> developers on #haskell-distributed, this is not something that's specific
> to Cloud Haskell. It is something to do with the threaded runtime system,
> so would need to be solved for GHC (or is it just the Network package!?) in
> general. Writing up a simple C program and equivalent socket use in Haskell
> and comparing the latency using -threaded will show this up.
> >
> > See the latency section in
> http://haskell-distributed.github.com/wiki/networktransport.html for some
> more details. According to that, there *are* some things we might be able
> to do, but the 20% latency isn't going to change significantly on the face
> of things.
> >
> > We have an open ticket to look into this (
> https://cloud-haskell.atlassian.net/browse/NTTCP-4) and at some point
> we'll try and put together the sample programs in a github repository (if
> that's not already done - I might've missed previous spikes done by Edsko
> or others) and investigate further.
> >
> > One of the other (more experienced!) devs might be able to chip in and
> proffer a better explanation.
> >
> > Cheers,
> > Tim
> >
> > On 6 Feb 2013, at 13:27, kosti...@gmail.com
> wrote:
> >
> > > Haven't you had a necessity to launch Haskell in no-threaded mode
> during the intense network data exchange?
> > > I am getting the double performance penalty in threaded mode. But I
> must use threaded mode because epoll and kevent are available in the
> threaded mode only.
> > >
> >
> > [snip]
> >
> > >
> > >
> > > среда,

Re: Cloud Haskell and network latency issues with -threaded

[Edward Z. Yang]

OK. I think it is high priority for us to get some latency benchmarks
into nofib so that GHC devs (including me) can start measuring changes
off them.  I know Edsko has some benchmarks here:
http://www.edsko.net/2013/02/06/performance-problems-with-threaded/
but they depend on network which makes it a little difficult to move into nofib.
I'm working on other scheduler changes that may help you guys out; we
should keep each other updated.

I noticed your patch also incorporates the "make yield actually work" patch;
do you think the improvement in 7.4.1 was due to that specific change?
(Have you instrumented the run queues and checked how your patch changes
the distribution of jobs over your runtime?)

Somewhat unrelatedly, if you have some good latency tests already,
it may be worth a try compiling your copy of GHC -fno-omit-yields, so that
forced context switches get serviced more predictably.

Cheers,
Edward

Excerpts from Andreas Voellmy's message of Thu Feb 07 21:20:25 -0800 2013:
> Hi Edward,
> 
> I did two things to improve latency for my application: (1) rework the IO
> manager and (2) stabilize the work pushing. (1) seems like a big win and we
> are almost done with the work on that part. It is less clear whether (2)
> will generally help much. It helped me when I developed it against 7.4.1,
> but it doesn't seem to have much impact on HEAD on the few measurements I
> did. The idea of (2) was to keep running averages of the run queue length
> of each capability, then push work when these running averages get too
> out-of-balance. The desired effect (which seems to work on my particular
> application) is to avoid cases in which threads are pushed back and forth
> among cores, which may make cache usage worse. You can see my patch here:
> https://github.com/AndreasVoellmy/ghc-arv/commits/push-work-exchange-squashed
> .
> 
> -Andi
> 
> On Fri, Feb 8, 2013 at 12:10 AM, Edward Z. Yang  wrote:
> 
> > Hey folks,
> >
> > The latency changes sound relevant to some work on the scheduler I'm doing;
> > is there a place I can see the changes?
> >
> > Thanks,
> > Edward
> >
> > Excerpts from Simon Peyton-Jones's message of Wed Feb 06 10:10:10 -0800
> > 2013:
> > > I (with help from Kazu and helpful comments from Bryan and Johan) have
> > nearly completed an overhaul to the IO manager based on my observations and
> > we are in the final stages of getting it into GHC
> > >
> > > This is really helpful. Thank you very much Andreas, Kazu, Bryan, Johan.
> > >
> > > Simon
> > >
> > > From: parallel-hask...@googlegroups.com [mailto:
> > parallel-hask...@googlegroups.com] On Behalf Of Andreas Voellmy
> > > Sent: 06 February 2013 14:28
> > > To: watson.timo...@gmail.com
> > > Cc: kosti...@gmail.com; parallel-haskell;
> > glasgow-haskell-users@haskell.org
> > > Subject: Re: Cloud Haskell and network latency issues with -threaded
> > >
> > > Hi all,
> > >
> > > I haven't followed the conversations around CloudHaskell closely, but I
> > noticed the discussion around latency using the threaded runtime system,
> > and I thought I'd jump in here.
> > >
> > > I've been developing a server in Haskell that serves hundreds to
> > thousands of clients over very long-lived TCP sockets. I also had latency
> > problems with GHC. For example, with 100 clients I had a 10 ms
> > (millisecond) latency and with 500 clients I had a 29ms latency. I looked
> > into the problem and found that some bottlenecks in the threaded IO manager
> > were the cause. I made some hacks there and got the latency for 100 and 500
> > clients down to under 0.2 ms. I (with help from Kazu and helpful comments
> > from Bryan and Johan) have nearly completed an overhaul to the IO manager
> > based on my observations and we are in the final stages of getting it into
> > GHC. Hopefully our work will also fix the latency issues in CloudHaskell
> > programs :)
> > >
> > > It would be very helpful if someone has some benchmark CloudHaskell
> > applications and workloads to test with. Does anyone have these handy?
> > >
> > > Cheers,
> > > Andi
> > >
> > > On Wed, Feb 6, 2013 at 9:09 AM, Tim Watson  > > wrote:
> > > Hi Kostirya,
> > >
> > > I'm putting the parallel-haskell and ghc-users lists on cc, just in case
> > other (better informed) folks want to chip in here.
> > >
> > > 
> > >
> > > First of all, I'm assuming you're talking about network latency when
> > compiling with -threaded - if not I apologise for misunderstanding!
> > >
> > > There is apparently an outstanding network latency issue when compiling
> > with -threaded, but according to a conversation I had with the other
> > developers on #haskell-distributed, this is not something that's specific
> > to Cloud Haskell. It is something to do with the threaded runtime system,
> > so would need to be solved for GHC (or is it just the Network package!?) in
> > general. Writing up a simple C program and equivalent socket use in Haskell
> > and comparing the latency using -thr

Re: Cloud Haskell and network latency issues with -threaded

[Andreas Voellmy]

On Fri, Feb 8, 2013 at 12:30 AM, Edward Z. Yang  wrote:

> OK. I think it is high priority for us to get some latency benchmarks
> into nofib so that GHC devs (including me) can start measuring changes
> off them.  I know Edsko has some benchmarks here:
> http://www.edsko.net/2013/02/06/performance-problems-with-threaded/
> but they depend on network which makes it a little difficult to move into
> nofib.
> I'm working on other scheduler changes that may help you guys out; we
> should keep each other updated.
>

That would be great :)


>
> I noticed your patch also incorporates the "make yield actually work"
> patch;
> do you think the improvement in 7.4.1 was due to that specific change?
>

Actually, I believe that patch is irrelevant to the scheduler change and
probably should not be in there, strictly speaking. I actually needed that
patch for the IO manager revisions to work properly.


> (Have you instrumented the run queues and checked how your patch changes
> the distribution of jobs over your runtime?)
>
> I didn't do this very rigorously, but I think I added some print
statements in the scheduler and I looked at some eventlogs in threadscope
to see that threads work pushing slows down after a while. I had planned to
write a script to analyze an event log file to extract these stats, but I
never got around to it.

-Andi



> Somewhat unrelatedly, if you have some good latency tests already,
> it may be worth a try compiling your copy of GHC -fno-omit-yields, so that
> forced context switches get serviced more predictably.
>
> Cheers,
> Edward
>
> Excerpts from Andreas Voellmy's message of Thu Feb 07 21:20:25 -0800 2013:
> > Hi Edward,
> >
> > I did two things to improve latency for my application: (1) rework the IO
> > manager and (2) stabilize the work pushing. (1) seems like a big win and
> we
> > are almost done with the work on that part. It is less clear whether (2)
> > will generally help much. It helped me when I developed it against 7.4.1,
> > but it doesn't seem to have much impact on HEAD on the few measurements I
> > did. The idea of (2) was to keep running averages of the run queue length
> > of each capability, then push work when these running averages get too
> > out-of-balance. The desired effect (which seems to work on my particular
> > application) is to avoid cases in which threads are pushed back and forth
> > among cores, which may make cache usage worse. You can see my patch here:
> >
> https://github.com/AndreasVoellmy/ghc-arv/commits/push-work-exchange-squashed
> > .
> >
> > -Andi
> >
> > On Fri, Feb 8, 2013 at 12:10 AM, Edward Z. Yang  wrote:
> >
> > > Hey folks,
> > >
> > > The latency changes sound relevant to some work on the scheduler I'm
> doing;
> > > is there a place I can see the changes?
> > >
> > > Thanks,
> > > Edward
> > >
> > > Excerpts from Simon Peyton-Jones's message of Wed Feb 06 10:10:10 -0800
> > > 2013:
> > > > I (with help from Kazu and helpful comments from Bryan and Johan)
> have
> > > nearly completed an overhaul to the IO manager based on my
> observations and
> > > we are in the final stages of getting it into GHC
> > > >
> > > > This is really helpful. Thank you very much Andreas, Kazu, Bryan,
> Johan.
> > > >
> > > > Simon
> > > >
> > > > From: parallel-hask...@googlegroups.com [mailto:
> > > parallel-hask...@googlegroups.com] On Behalf Of Andreas Voellmy
> > > > Sent: 06 February 2013 14:28
> > > > To: watson.timo...@gmail.com
> > > > Cc: kosti...@gmail.com; parallel-haskell;
> > > glasgow-haskell-users@haskell.org
> > > > Subject: Re: Cloud Haskell and network latency issues with -threaded
> > > >
> > > > Hi all,
> > > >
> > > > I haven't followed the conversations around CloudHaskell closely,
> but I
> > > noticed the discussion around latency using the threaded runtime
> system,
> > > and I thought I'd jump in here.
> > > >
> > > > I've been developing a server in Haskell that serves hundreds to
> > > thousands of clients over very long-lived TCP sockets. I also had
> latency
> > > problems with GHC. For example, with 100 clients I had a 10 ms
> > > (millisecond) latency and with 500 clients I had a 29ms latency. I
> looked
> > > into the problem and found that some bottlenecks in the threaded IO
> manager
> > > were the cause. I made some hacks there and got the latency for 100
> and 500
> > > clients down to under 0.2 ms. I (with help from Kazu and helpful
> comments
> > > from Bryan and Johan) have nearly completed an overhaul to the IO
> manager
> > > based on my observations and we are in the final stages of getting it
> into
> > > GHC. Hopefully our work will also fix the latency issues in
> CloudHaskell
> > > programs :)
> > > >
> > > > It would be very helpful if someone has some benchmark CloudHaskell
> > > applications and workloads to test with. Does anyone have these handy?
> > > >
> > > > Cheers,
> > > > Andi
> > > >
> > > > On Wed, Feb 6, 2013 at 9:09 AM, Tim Watson  > > > wrote:
> > > > Hi Kost