Re: Release/git plans

[austin seipp]

Hello Simon,

On Thu, Jan 20, 2011 at 9:48 AM, Simon Marlow  wrote:
> ...
>  - make a new stable branch for 7.2, and release 7.2.1 shortly after.
>
> So we'll be doing a 7.2.1 release much earlier than planned.  Our motivation
> for doing this is:
>
>  - the 7.0 branch is darcs, but the HEAD will be git, so the longer
>   this situation persists the more difficult it is to merge changes.
>   Hence we want to fork a new STABLE branch as soon as possible.
>
>  - one of the fixes we need to make soon, to fix equality superclasses,
>   is too large a change to put in the 7.0 branch.  We also have some
>   pending changes to the internal representation of coercions that
>   will be disruptive in HEAD, and will make future merging to 7.0
>   difficult.
>
>  - 7.0.2 should be a nice stable release, and HEAD is actually in
>   pretty good shape right now too.

So, given that 7.2 will be released much earlier than the normal
release cycle, is there any room for anything else to get into HEAD
for the 7.2 release before everything is switched? In particular I
fixed up Max Bolingbroke's old compiler plugin work to be usable with
the latest HEAD, and all the fundamental work is there and done, just
some additional small things are needed (notably having ghc dump
plugin information a la -ddump flags, and testsuite patches are about
it I think.) The patch itself is pretty small and doesn't touch *too*
much code, mostly adding dynamic loading and the plugin API, but it's
arguably adding a 'big' feature for users of GHC to start utilizing,
and perhaps a release in 7.2 would cause problems merging changes
until you cut a new STABLE branch with git, like you said.

It would be nice to have this in GHC 7.2, but I was thinking of
eventually extending the scope of compiler plugins to allow users to
write Cmm optimisations as well. It would likely be best to wait for
the switch to Git and for the new code generator branch to be merged
before that work happens though, but writing plugins that do Core
passes seem to work well right now.

> The GHC git repo that
> we'll be using is here:
>
>  http://darcs.haskell.org/ghc.git

This is an incredibly minor note in my opinion (that was brought up
before IIRC) but, isn't it a little strange for ghc's git repository
to exist on darcs.haskell.org? Not that it's a problem, just slightly
confusing I guess.

-- 
Regards,
Austin

___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: Release/git plans

[Max Bolingbroke]

On 20 January 2011 16:57, austin seipp  wrote:
> It would be nice to have this in GHC 7.2, but I was thinking of
> eventually extending the scope of compiler plugins to allow users to
> write Cmm optimisations as well.

This would be particularly cool because Cmm optimisations in the new
Hoopl framework compose extremely nicely.

(And yes, obviously I would be very happy if Simon commits my plugins
patches ;-))

Max

___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: Release/git plans

[Isaac Dupree]

On 01/20/11 11:57, austin seipp wrote:

The GHC git repo that
we'll be using is here:

  http://darcs.haskell.org/ghc.git


This is an incredibly minor note in my opinion (that was brought up
before IIRC) but, isn't it a little strange for ghc's git repository
to exist on darcs.haskell.org? Not that it's a problem, just slightly
confusing I guess.


Yes, and the commits mailing-list is still called "cvs-...@haskell.org". 
 The niceness of sounding right tends to be outweighed by the 
inconvenience of switching :)


-Isaac

___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: Release/git plans

[austin seipp]

Point taken, I personally think it's rather minor although it was
brought up before like I said, so perhaps others think differently.

Either way, I, for one, welcome our new version control overlord.

On Thu, Jan 20, 2011 at 3:09 PM, Isaac Dupree
 wrote:
> On 01/20/11 11:57, austin seipp wrote:
>>>
>>> The GHC git repo that
>>> we'll be using is here:
>>>
>>>  http://darcs.haskell.org/ghc.git
>>
>> This is an incredibly minor note in my opinion (that was brought up
>> before IIRC) but, isn't it a little strange for ghc's git repository
>> to exist on darcs.haskell.org? Not that it's a problem, just slightly
>> confusing I guess.
>
> Yes, and the commits mailing-list is still called "cvs-...@haskell.org".
>  The niceness of sounding right tends to be outweighed by the inconvenience
> of switching :)
>
> -Isaac
>
> ___
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users@haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>



-- 
Regards,
Austin

___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: Release/git plans

[Christian Höner zu Siederdissen]

This is Haskell. One should abstract away the system used. Call it
vcs-ghc@ and vcs.haskell.org ;-)

Gruss,
Christian

* austin seipp  [20.01.2011 22:19]:
> Point taken, I personally think it's rather minor although it was
> brought up before like I said, so perhaps others think differently.
> 
> Either way, I, for one, welcome our new version control overlord.
> 
> On Thu, Jan 20, 2011 at 3:09 PM, Isaac Dupree
>  wrote:
> > On 01/20/11 11:57, austin seipp wrote:
> >>>
> >>> The GHC git repo that
> >>> we'll be using is here:
> >>>
> >>>  http://darcs.haskell.org/ghc.git
> >>
> >> This is an incredibly minor note in my opinion (that was brought up
> >> before IIRC) but, isn't it a little strange for ghc's git repository
> >> to exist on darcs.haskell.org? Not that it's a problem, just slightly
> >> confusing I guess.
> >
> > Yes, and the commits mailing-list is still called "cvs-...@haskell.org".
> >  The niceness of sounding right tends to be outweighed by the inconvenience
> > of switching :)
> >
> > -Isaac
> >
> > ___
> > Glasgow-haskell-users mailing list
> > Glasgow-haskell-users@haskell.org
> > http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
> >
> 
> 
> 
> -- 
> Regards,
> Austin
> 
> ___
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users@haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users


pgpncMFW5zZsL.pgp
Description: PGP signature
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

RE: Release/git plans

[Simon Peyton-Jones]

Austin

| So, given that 7.2 will be released much earlier than the normal
| release cycle, is there any room for anything else to get into HEAD
| for the 7.2 release before everything is switched? In particular I
| fixed up Max Bolingbroke's old compiler plugin work to be usable with
| the latest HEAD, and all the fundamental work is there and done, just
| some additional small things are needed (notably having ghc dump
| plugin information a la -ddump flags, and testsuite patches are about
| it I think.) The patch itself is pretty small and doesn't touch *too*
| much code, mostly adding dynamic loading and the plugin API, but it's
| arguably adding a 'big' feature for users of GHC to start utilizing,
| and perhaps a release in 7.2 would cause problems merging changes
| until you cut a new STABLE branch with git, like you said.

I'm sorry I've been slow on this.  "Review and apply the plugins patch" is in 
my inbox, but it's been queued up behind too many other things, notably making 
the new typechecker work.

I'm pretty keen on the whole plugin idea, because it makes the compiler more 
extensible and lowers the barrier to entry.  My only reason for delay is that I 
wanted to review the design (as seen by a plug-in author).  Once we provide it, 
we have to support it, and it's harder to change.

The fact that you are actively engaged, have done the work with Max, and are (I 
assume) happy to respond to user queries, fix bugs etc, is a major incentive.  
Thank you!

So yes, because of that I undertake to do this for 7.2 [unless Simon M tells me 
no :-)].  I'll add a few thoughts to the ticket right now.  
http://hackage.haskell.org/trac/ghc/ticket/3843.  Interested parties, add 
yourselves to the cc list of the ticket.

Simon


___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: Release/git plans

[Thomas Schilling]

On 21 January 2011 09:13, Simon Peyton-Jones  wrote:

> I'm pretty keen on the whole plugin idea, because it makes the compiler more 
> extensible and lowers the barrier to entry.  My only reason for delay is that 
> I wanted to review the design (as seen by a plug-in author).  Once we provide 
> it, we have to support it, and it's harder to change.

The Scala compiler has a plugin system.  Here's a document that
describes their approach to dealing with phase ordering of plugins:
.  It seems rather complex, but seems
necessary (or the document is just very long).  There's also a
tutorial on writing Scala plugins, which may help to get a feel for
how their API works. 

/ Thomas

___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: Release/git plans

[austin seipp]

Simon,

On Fri, Jan 21, 2011 at 3:13 AM, Simon Peyton-Jones
 wrote:
> Austin
>
> | So, given that 7.2 will be released much earlier than the normal
> | release cycle, is there any room for anything else to get into HEAD
> | for the 7.2 release before everything is switched? In particular I
> | fixed up Max Bolingbroke's old compiler plugin work to be usable with
> | the latest HEAD, and all the fundamental work is there and done, just
> | some additional small things are needed (notably having ghc dump
> | plugin information a la -ddump flags, and testsuite patches are about
> | it I think.) The patch itself is pretty small and doesn't touch *too*
> | much code, mostly adding dynamic loading and the plugin API, but it's
> | arguably adding a 'big' feature for users of GHC to start utilizing,
> | and perhaps a release in 7.2 would cause problems merging changes
> | until you cut a new STABLE branch with git, like you said.
>
> I'm sorry I've been slow on this.  "Review and apply the plugins patch" is in 
> my inbox, but it's been queued up behind too many other things, notably 
> making the new typechecker work.

That's fine! One thing at a time. The typechecker overhaul is great
IMO, and I've noticed it's fixed many strange GHC bugs relating to
type system feature interactions (including a few I hit.) Much
appreciated work!

> I'm pretty keen on the whole plugin idea, because it makes the compiler more 
> extensible and lowers the barrier to entry.  My only reason for delay is that 
> I wanted to review the design (as seen by a plug-in author).  Once we provide 
> it, we have to support it, and it's harder to change.

This is very reasonable and I was going to mention it once someone
responded. Up to this point, Max has really been the main person to
write any plugins for GHC using the current interface. I would really
welcome anybody interested in hacking on GHC look at the current API
in the patch - my main concern up to this point is that the actual
means of installing passes into the Core pipeline, the installation
function of type `[CoreToDo] -> CoreM [CoreToDo]`, is a little
fragile.

Scala's approach seems to be to have plugins specify 'runAfter' and
'runBefore' constraints, which specify what compiler phases should run
before and after the plugin's pass in question (and certain phases may
run multiple times, so as a consequence, so does your plugin.) If I
remember correctly, this was somewhat similar to the original design
Max proposed which was to specify in which phase the pass is run, and
it essentially amounted to a phase constraint. Perhaps Max can
elaborate on why this design was rejected in favor of the current one,
so we can see how and where it falls down, and what we really want.

Thomas pointed out the Scala compiler plugin design document, so I'll
be sure to read over it this weekend when I get the chance to cook up
ideas.

Also worth noting, like I said, is I would inevitably like to extend
compiler plugins to work on Cmm, once the new codegen (using hoopl)
hits. Maybe even allow you to write a new backend ultimately if you
wanted to, although I'd call that the last and ultimate challenge at
this point (because from my cursory glances, it would require quite a
bit of re-engineering and re-design of the driver and compilation
pipeline, much more so than just the Core plugins did or Cmm plugins
might. That's a bit more time than I have... at the moment ;)

To accumulate ideas and comments I've spent the past few days writing
up a wiki page (mostly written in the wee hours of the morning no
less), documenting the current API, how it might be improved, and
future work like C-- plugins or plugins implementing new backends.
Most of it is half baked. And half-written. Probably grammatical
errors and run-on sentences. Incomplete sections. Likely with big,
stupid ideas. Caveat emptor, etc.

The wiki page in all its (horrid, half-written and mangled) glory can
be located here:

http://hackage.haskell.org/trac/ghc/wiki/NewPlugins

If people can comment and add ideas, it will be much easier to
determine if this should go into 7.2.1. If the current API is too
fragile or needs to be expanded, it would likely be best to back out
and do some work, and try again for 7.4.1. That will also give a much
bigger window for feedback and testing, since 7.2 will be on a much
shorter cycle it seems.

I'll probably end up throwing up a RFC on both glasgow-haskell-users
and cvs-ghc sometime in the next week to get feedback after more
brainstorming, because it's the most important thing as of right now
and for any future work in this area, in my opinion, since it will be
a publicly exposed API for users to write code against.

> The fact that you are actively engaged, have done the work with Max, and are 
> (I assume) happy to respond to user queries, fix bugs etc, is a major 
> incentive.  Thank you!

I'm more than happy to respond to any user queries and respond to bugs
people come up with, yes (Max seems to be w

Re: Release/git plans

[scooter . phd]

Llvm is also a good reference for compiler plugin design; it would appear that 
Scala borrowed from their approach.


-scooter
Sent from my Verizon Wireless BlackBerry

-Original Message-
From: austin seipp 
Sender: glasgow-haskell-users-boun...@haskell.org
Date: Fri, 21 Jan 2011 17:59:32 
To: Simon Peyton-Jones
Cc: Simon Marlow; 
glasgow-haskell-users@haskell.org
Subject: Re: Release/git plans

Simon,

On Fri, Jan 21, 2011 at 3:13 AM, Simon Peyton-Jones
 wrote:
> Austin
>
> | So, given that 7.2 will be released much earlier than the normal
> | release cycle, is there any room for anything else to get into HEAD
> | for the 7.2 release before everything is switched? In particular I
> | fixed up Max Bolingbroke's old compiler plugin work to be usable with
> | the latest HEAD, and all the fundamental work is there and done, just
> | some additional small things are needed (notably having ghc dump
> | plugin information a la -ddump flags, and testsuite patches are about
> | it I think.) The patch itself is pretty small and doesn't touch *too*
> | much code, mostly adding dynamic loading and the plugin API, but it's
> | arguably adding a 'big' feature for users of GHC to start utilizing,
> | and perhaps a release in 7.2 would cause problems merging changes
> | until you cut a new STABLE branch with git, like you said.
>
> I'm sorry I've been slow on this.  "Review and apply the plugins patch" is in 
> my inbox, but it's been queued up behind too many other things, notably 
> making the new typechecker work.

That's fine! One thing at a time. The typechecker overhaul is great
IMO, and I've noticed it's fixed many strange GHC bugs relating to
type system feature interactions (including a few I hit.) Much
appreciated work!

> I'm pretty keen on the whole plugin idea, because it makes the compiler more 
> extensible and lowers the barrier to entry.  My only reason for delay is that 
> I wanted to review the design (as seen by a plug-in author).  Once we provide 
> it, we have to support it, and it's harder to change.

This is very reasonable and I was going to mention it once someone
responded. Up to this point, Max has really been the main person to
write any plugins for GHC using the current interface. I would really
welcome anybody interested in hacking on GHC look at the current API
in the patch - my main concern up to this point is that the actual
means of installing passes into the Core pipeline, the installation
function of type `[CoreToDo] -> CoreM [CoreToDo]`, is a little
fragile.

Scala's approach seems to be to have plugins specify 'runAfter' and
'runBefore' constraints, which specify what compiler phases should run
before and after the plugin's pass in question (and certain phases may
run multiple times, so as a consequence, so does your plugin.) If I
remember correctly, this was somewhat similar to the original design
Max proposed which was to specify in which phase the pass is run, and
it essentially amounted to a phase constraint. Perhaps Max can
elaborate on why this design was rejected in favor of the current one,
so we can see how and where it falls down, and what we really want.

Thomas pointed out the Scala compiler plugin design document, so I'll
be sure to read over it this weekend when I get the chance to cook up
ideas.

Also worth noting, like I said, is I would inevitably like to extend
compiler plugins to work on Cmm, once the new codegen (using hoopl)
hits. Maybe even allow you to write a new backend ultimately if you
wanted to, although I'd call that the last and ultimate challenge at
this point (because from my cursory glances, it would require quite a
bit of re-engineering and re-design of the driver and compilation
pipeline, much more so than just the Core plugins did or Cmm plugins
might. That's a bit more time than I have... at the moment ;)

To accumulate ideas and comments I've spent the past few days writing
up a wiki page (mostly written in the wee hours of the morning no
less), documenting the current API, how it might be improved, and
future work like C-- plugins or plugins implementing new backends.
Most of it is half baked. And half-written. Probably grammatical
errors and run-on sentences. Incomplete sections. Likely with big,
stupid ideas. Caveat emptor, etc.

The wiki page in all its (horrid, half-written and mangled) glory can
be located here:

http://hackage.haskell.org/trac/ghc/wiki/NewPlugins

If people can comment and add ideas, it will be much easier to
determine if this should go into 7.2.1. If the current API is too
fragile or needs to be expanded, it would likely be best to back out
and do some work, and try again for 7.4.1. That will also give a much
bigger window for feedback and testing, since 7.2 will be on a much
shorter cycle it seems.

I'll probably e

Re: Release/git plans

[Max Bolingbroke]

On 21 January 2011 23:59, austin seipp  wrote:
> Perhaps Max can
> elaborate on why this design was rejected in favor of the current one,
> so we can see how and where it falls down, and what we really want.

The only reason really is that it added a lot of mechanism. From the
top of my head:
 * Parsing etc for PHASE pragmas that declared phase objects
 * A new namespace for phases
 * Stuff to gather declared phases from all imported modules during compilation
 * A built-in phase for each core pass
 * A solver that ordered core passes and plugin passes according to the phases

So it was a lot of trouble for relatively little gain. In an effort to
keep the delta against GHC small I threw it out in favour of the much,
much simpler design we have today.

> Thomas pointed out the Scala compiler plugin design document, so I'll
> be sure to read over it this weekend when I get the chance to cook up
> ideas.

The Scala plugins project was just starting when I was working on GHC
plugins so there was no design doc I could refer to at that time.
Shame :-(

Thanks for taking the lead on resurrecting plugins, Austin!

Cheers,
Max

___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: Release/git plans

[Neil Mitchell]

Hi Austin,

The compiler plugins work is a great, and I'd be a likely user. The
original version wasn't supported on Windows, because GHC on Windows
lacked various forms of dynamic linking. Does the current patch you've
prepared work on Windows?

Thanks, Neil

On Sat, Jan 22, 2011 at 10:29 AM, Max Bolingbroke
 wrote:
> On 21 January 2011 23:59, austin seipp  wrote:
>> Perhaps Max can
>> elaborate on why this design was rejected in favor of the current one,
>> so we can see how and where it falls down, and what we really want.
>
> The only reason really is that it added a lot of mechanism. From the
> top of my head:
>  * Parsing etc for PHASE pragmas that declared phase objects
>  * A new namespace for phases
>  * Stuff to gather declared phases from all imported modules during 
> compilation
>  * A built-in phase for each core pass
>  * A solver that ordered core passes and plugin passes according to the phases
>
> So it was a lot of trouble for relatively little gain. In an effort to
> keep the delta against GHC small I threw it out in favour of the much,
> much simpler design we have today.
>
>> Thomas pointed out the Scala compiler plugin design document, so I'll
>> be sure to read over it this weekend when I get the chance to cook up
>> ideas.
>
> The Scala plugins project was just starting when I was working on GHC
> plugins so there was no design doc I could refer to at that time.
> Shame :-(
>
> Thanks for taking the lead on resurrecting plugins, Austin!
>
> Cheers,
> Max
>
> ___
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users@haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>

___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users