RE: Plugins: Accessing unexported bindings

2015-12-08 Thread Simon Peyton Jones 
I think it’s probably the desugarer, and the plugin can’t do anything before 
desugaring!

By all means open a ticket.  I can advise if anyone wants to take it up.

Simon

From: Levent Erkok [mailto:erk...@gmail.com]
Sent: 08 December 2015 03:32
To: Simon Peyton Jones <simo...@microsoft.com>
Cc: Eric Seidel <e...@seidel.io>; omeraga...@gmail.com; ezy...@mit.edu; 
ghc-devs@haskell.org
Subject: Re: Plugins: Accessing unexported bindings

Thanks for the explanation, Simon. I think Eric is spot on. Looking at some of 
the "ghc -v3" output, it appears GHC always runs the following passes before 
any plugins are run:

   *** Simplify:
   *** CorePrep:
   *** ByteCodeGen:
   *** Desugar:

Just judging by the name, it could either be the first "Simplify" or the 
"Desugar" that gets rid of the dead-code at this point, I'm not sure which.

It would help if this stage didn't remove dead-bindings. If doing so 
automatically is not the best option, requiring a user given pragma like 
'KeepAlive' wouldn't be too onerous either.

-Levent.

On Mon, Dec 7, 2015 at 8:22 AM, Simon Peyton Jones 
<simo...@microsoft.com<mailto:simo...@microsoft.com>> wrote:
It would not be hard to stop the desugarer dropping dead bindings, if that was 
helpful.

S

|  -Original Message-
|  From: Eric Seidel [mailto:e...@seidel.io<mailto:e...@seidel.io>]
|  Sent: 07 December 2015 15:44
|  To: Simon Peyton Jones <simo...@microsoft.com<mailto:simo...@microsoft.com>>
|  Cc: Levent Erkok <erk...@gmail.com<mailto:erk...@gmail.com>>; 
omeraga...@gmail.com<mailto:omeraga...@gmail.com>;
|  ezy...@mit.edu<mailto:ezy...@mit.edu>; 
ghc-devs@haskell.org<mailto:ghc-devs@haskell.org>
|  Subject: Re: Plugins: Accessing unexported bindings
|
|  The problem, as I recall, is that GHC does an initial bit of dead-code
|  elimination in the desugarer, before the plugins have a chance to run.
|  (I believe this is part of simpleOptPgm, but may be mistaken)
|
|  I'm not sure why this is done in the desugarer, it seems to be out of
|  place there.
|
|  On Mon, Dec 7, 2015, at 05:14, Simon Peyton Jones wrote:
|  > Plugins get to edit the entire core-to-core pipeline!  There is no
|  magic.
|  >  At least I don’t think so
|  >
|  > file:///Z:/tmp/users_guide/compiler-plugins.html
|  >
|  > S
|  >
|  > From: Levent Erkok [mailto:erk...@gmail.com<mailto:erk...@gmail.com>]
|  > Sent: 07 December 2015 13:11
|  > To: Simon Peyton Jones 
<simo...@microsoft.com<mailto:simo...@microsoft.com>>
|  > Cc: Eric Seidel <e...@seidel.io<mailto:e...@seidel.io>>; 
omeraga...@gmail.com<mailto:omeraga...@gmail.com>;
|  > ezy...@mit.edu<mailto:ezy...@mit.edu>; 
ghc-devs@haskell.org<mailto:ghc-devs@haskell.org>
|  > Subject: Re: Plugins: Accessing unexported bindings
|  >
|  > That's a good point; keeping all annotated bindings alive seems to
|  be
|  > an overkill..
|  >
|  > Regarding implementing "pass at the start." I'm not sure if plugin
|  > authors have any freedom as to decide when their plugin actually
|  runs.
|  > It seems GHC magically determines the order and runs them. Can you
|  > point me to some code/docs that tells me how to go "first" in that
|  > sense? (Or at least before the pass that drops dead code.)
|  >
|  > On Dec 7, 2015, at 4:45 AM, Simon Peyton Jones
|  > 
<simo...@microsoft.com<mailto:simo...@microsoft.com><mailto:simo...@microsoft.com<mailto:simo...@microsoft.com>>>
 wrote:
|  > Indeed. How about this: if there's an ANN on a binder (any ANN),
|  then
|  > GHC should keep it alive.
|  >
|  > Really?  It might be something like “don’t give warnings for this
|  > binding” or “don’t inline me” or something.   To say *any*
|  annotation
|  > seems a bit brutal doesn’t it?Mind you I don’t have a better
|  idea.
|  >
|  > One thought: your plugin could add a pass right at the start, which
|  > marks everything you want as keep-alive.
|  >
|  > S
|  >
|  > From: Levent Erkok [mailto:erk...@gmail.com<mailto:erk...@gmail.com>]
|  > Sent: 07 December 2015 12:42
|  > To: Simon Peyton Jones
|  > 
<simo...@microsoft.com<mailto:simo...@microsoft.com><mailto:simo...@microsoft.com<mailto:simo...@microsoft.com>>>
|  > Cc: Eric Seidel 
<e...@seidel.io<mailto:e...@seidel.io><mailto:e...@seidel.io<mailto:e...@seidel.io>>>;
|  > 
omeraga...@gmail.com<mailto:omeraga...@gmail.com><mailto:omeraga...@gmail.com<mailto:omeraga...@gmail.com>>;
|  > 
ezy...@mit.edu<mailto:ezy...@mit.edu><mailto:ezy...@mit.edu<mailto:ezy...@mit.edu>>;
|  > 
ghc-devs@haskell.org<mailto:ghc-devs@haskell.org><mailto:ghc-devs@haskell.org<mailto:ghc-devs@haskell.org>>
|

Re: Plugins: Accessing unexported bindings

2015-12-08 Thread Levent Erkok 
http://ghc.haskell.org/trac/ghc/ticket/11179#ticket

Thanks!

-Levent.

On Tue, Dec 8, 2015 at 4:33 AM, Simon Peyton Jones <simo...@microsoft.com>
wrote:

> I think it’s probably the desugarer, and the plugin can’t do anything
> before desugaring!
>
>
>
> By all means open a ticket.  I can advise if anyone wants to take it up.
>
>
>
> Simon
>
>
>
> *From:* Levent Erkok [mailto:erk...@gmail.com]
> *Sent:* 08 December 2015 03:32
>
> *To:* Simon Peyton Jones <simo...@microsoft.com>
> *Cc:* Eric Seidel <e...@seidel.io>; omeraga...@gmail.com; ezy...@mit.edu;
> ghc-devs@haskell.org
> *Subject:* Re: Plugins: Accessing unexported bindings
>
>
>
> Thanks for the explanation, Simon. I think Eric is spot on. Looking at
> some of the "ghc -v3" output, it appears GHC always runs the following
> passes before any plugins are run:
>
>
>
>*** Simplify:
>
>*** CorePrep:
>
>*** ByteCodeGen:
>
>*** Desugar:
>
>
>
> Just judging by the name, it could either be the first "Simplify" or the
> "Desugar" that gets rid of the dead-code at this point, I'm not sure which.
>
>
>
> It would help if this stage didn't remove dead-bindings. If doing so
> automatically is not the best option, requiring a user given pragma like
> 'KeepAlive' wouldn't be too onerous either.
>
>
>
> -Levent.
>
>
>
> On Mon, Dec 7, 2015 at 8:22 AM, Simon Peyton Jones <simo...@microsoft.com>
> wrote:
>
> It would not be hard to stop the desugarer dropping dead bindings, if that
> was helpful.
>
> S
>
> |  -Original Message-
> |  From: Eric Seidel [mailto:e...@seidel.io]
> |  Sent: 07 December 2015 15:44
> |  To: Simon Peyton Jones <simo...@microsoft.com>
>
> |  Cc: Levent Erkok <erk...@gmail.com>; omeraga...@gmail.com;
> |  ezy...@mit.edu; ghc-devs@haskell.org
> |  Subject: Re: Plugins: Accessing unexported bindings
> |
> |  The problem, as I recall, is that GHC does an initial bit of dead-code
> |  elimination in the desugarer, before the plugins have a chance to run.
> |  (I believe this is part of simpleOptPgm, but may be mistaken)
> |
> |  I'm not sure why this is done in the desugarer, it seems to be out of
> |  place there.
> |
> |  On Mon, Dec 7, 2015, at 05:14, Simon Peyton Jones wrote:
> |  > Plugins get to edit the entire core-to-core pipeline!  There is no
> |  magic.
> |  >  At least I don’t think so
> |  >
> |  > file:///Z:/tmp/users_guide/compiler-plugins.html
> |  >
> |  > S
> |  >
> |  > From: Levent Erkok [mailto:erk...@gmail.com]
> |  > Sent: 07 December 2015 13:11
> |  > To: Simon Peyton Jones <simo...@microsoft.com>
> |  > Cc: Eric Seidel <e...@seidel.io>; omeraga...@gmail.com;
> |  > ezy...@mit.edu; ghc-devs@haskell.org
> |  > Subject: Re: Plugins: Accessing unexported bindings
> |  >
> |  > That's a good point; keeping all annotated bindings alive seems to
> |  be
> |  > an overkill..
> |  >
> |  > Regarding implementing "pass at the start." I'm not sure if plugin
> |  > authors have any freedom as to decide when their plugin actually
> |  runs.
> |  > It seems GHC magically determines the order and runs them. Can you
> |  > point me to some code/docs that tells me how to go "first" in that
> |  > sense? (Or at least before the pass that drops dead code.)
> |  >
> |  > On Dec 7, 2015, at 4:45 AM, Simon Peyton Jones
> |  > <simo...@microsoft.com<mailto:simo...@microsoft.com>> wrote:
> |  > Indeed. How about this: if there's an ANN on a binder (any ANN),
> |  then
> |  > GHC should keep it alive.
> |  >
> |  > Really?  It might be something like “don’t give warnings for this
> |  > binding” or “don’t inline me” or something.   To say *any*
> |  annotation
> |  > seems a bit brutal doesn’t it?Mind you I don’t have a better
> |  idea.
> |  >
> |  > One thought: your plugin could add a pass right at the start, which
> |  > marks everything you want as keep-alive.
> |  >
> |  > S
> |  >
> |  > From: Levent Erkok [mailto:erk...@gmail.com]
> |  > Sent: 07 December 2015 12:42
> |  > To: Simon Peyton Jones
> |  > <simo...@microsoft.com<mailto:simo...@microsoft.com>>
> |  > Cc: Eric Seidel <e...@seidel.io<mailto:e...@seidel.io>>;
> |  > omeraga...@gmail.com<mailto:omeraga...@gmail.com>;
> |  > ezy...@mit.edu<mailto:ezy...@mit.edu>;
> |  > ghc-devs@haskell.org<mailto:ghc-devs@haskell.org>
> |  > Subject: Re: Plugins: Accessing unexported

Re: Plugins: Accessing unexported bindings

2015-12-07 Thread Eric Seidel 
The problem, as I recall, is that GHC does an initial bit of dead-code
elimination in the desugarer, before the plugins have a chance to run.
(I believe this is part of simpleOptPgm, but may be mistaken)

I'm not sure why this is done in the desugarer, it seems to be out of
place there. 

On Mon, Dec 7, 2015, at 05:14, Simon Peyton Jones wrote:
> Plugins get to edit the entire core-to-core pipeline!  There is no magic.
>  At least I don’t think so
> 
> file:///Z:/tmp/users_guide/compiler-plugins.html
> 
> S
> 
> From: Levent Erkok [mailto:erk...@gmail.com]
> Sent: 07 December 2015 13:11
> To: Simon Peyton Jones <simo...@microsoft.com>
> Cc: Eric Seidel <e...@seidel.io>; omeraga...@gmail.com; ezy...@mit.edu;
> ghc-devs@haskell.org
> Subject: Re: Plugins: Accessing unexported bindings
> 
> That's a good point; keeping all annotated bindings alive seems to be an
> overkill..
> 
> Regarding implementing "pass at the start." I'm not sure if plugin
> authors have any freedom as to decide when their plugin actually runs. It
> seems GHC magically determines the order and runs them. Can you point me
> to some code/docs that tells me how to go "first" in that sense? (Or at
> least before the pass that drops dead code.)
> 
> On Dec 7, 2015, at 4:45 AM, Simon Peyton Jones
> <simo...@microsoft.com<mailto:simo...@microsoft.com>> wrote:
> Indeed. How about this: if there's an ANN on a binder (any ANN), then GHC
> should keep it alive.
> 
> Really?  It might be something like “don’t give warnings for this
> binding” or “don’t inline me” or something.   To say *any* annotation
> seems a bit brutal doesn’t it?Mind you I don’t have a better idea.
> 
> One thought: your plugin could add a pass right at the start, which marks
> everything you want as keep-alive.
> 
> S
> 
> From: Levent Erkok [mailto:erk...@gmail.com]
> Sent: 07 December 2015 12:42
> To: Simon Peyton Jones
> <simo...@microsoft.com<mailto:simo...@microsoft.com>>
> Cc: Eric Seidel <e...@seidel.io<mailto:e...@seidel.io>>;
> omeraga...@gmail.com<mailto:omeraga...@gmail.com>;
> ezy...@mit.edu<mailto:ezy...@mit.edu>;
> ghc-devs@haskell.org<mailto:ghc-devs@haskell.org>
> Subject: Re: Plugins: Accessing unexported bindings
> 
> Indeed. How about this: if there's an ANN on a binder (any ANN), then GHC
> should keep it alive.
> 
> Is that something one of the core-developers can implement? Happy to open
> a ticket if that helps.
> 
> On Dec 7, 2015, at 4:14 AM, Simon Peyton Jones
> <simo...@microsoft.com<mailto:simo...@microsoft.com>> wrote:
> If it's "dead" in this sense, it's already removed from ModGuts, no?
> 
> Yes, if it’s dead it’s gone.   That’s not too surprising, is it?
> 
> So you need a way to keep it alive. Maybe we need a pragma for that.   Or
> how would you like to signal it in the source code?
> 
> Simon
> 
> From: Levent Erkok [mailto:erk...@gmail.com]
> Sent: 07 December 2015 12:05
> To: Simon Peyton Jones
> <simo...@microsoft.com<mailto:simo...@microsoft.com>>
> Cc: Eric Seidel <e...@seidel.io<mailto:e...@seidel.io>>;
> omeraga...@gmail.com<mailto:omeraga...@gmail.com>;
> ezy...@mit.edu<mailto:ezy...@mit.edu>;
> ghc-devs@haskell.org<mailto:ghc-devs@haskell.org>
> Subject: Re: Plugins: Accessing unexported bindings
> 
> Thanks Simon.. But I remain utterly confused. As a "plugin" author, how
> do I get my hands on the Id associated with a top-level binder? If it's
> "dead" in this sense, it's already removed from ModGuts, no?
> 
> That is, by the time GHC runs my plugin, the Id has already disappeared
> for me to mark it "Local Exported." Is that not correct?
> 
> On Dec 7, 2015, at 2:28 AM, Simon Peyton Jones
> <simo...@microsoft.com<mailto:simo...@microsoft.com>> wrote:
> In the mean time, I'm still looking for a solution that doesn't involve
> exporting such identifiers from modules. As Eric pointed out, that seems
> to be the only current work-around for the time being.
> 
> “Exported” in this context only means “keep alive”. It does not mean
> exported in the Haskell source code sense. I’ve just added this comment
> to Var.hs.
> 
> So I think it does just what you want.
> 
> Simon
> 
> data ExportFlag   -- See Note [ExportFlag on binders]
>   = NotExported   -- ^ Not exported: may be discarded as dead code.
>   | Exported  -- ^ Exported: kept alive
> 
> {- Note [ExportFlag on binders]
> ~~~
> An ExportFlag of "Exported" on a top-level binder says "keep this
> binding alive; do not drop it as dead c

RE: Plugins: Accessing unexported bindings

2015-12-07 Thread Ben Gamari 
Simon Peyton Jones  writes:

> Plugins get to edit the entire core-to-core pipeline!  There is no magic.  At 
> least I don’t think so
>
> file:///Z:/tmp/users_guide/compiler-plugins.html
>
Or for those of us who aren't using Simon's desktop...

http://downloads.haskell.org/~ghc/master/users-guide/extending_ghc.html#compiler-plugins


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

RE: Plugins: Accessing unexported bindings

2015-12-07 Thread Simon Peyton Jones 
It would not be hard to stop the desugarer dropping dead bindings, if that was 
helpful.

S

|  -Original Message-
|  From: Eric Seidel [mailto:e...@seidel.io]
|  Sent: 07 December 2015 15:44
|  To: Simon Peyton Jones <simo...@microsoft.com>
|  Cc: Levent Erkok <erk...@gmail.com>; omeraga...@gmail.com;
|  ezy...@mit.edu; ghc-devs@haskell.org
|  Subject: Re: Plugins: Accessing unexported bindings
|  
|  The problem, as I recall, is that GHC does an initial bit of dead-code
|  elimination in the desugarer, before the plugins have a chance to run.
|  (I believe this is part of simpleOptPgm, but may be mistaken)
|  
|  I'm not sure why this is done in the desugarer, it seems to be out of
|  place there.
|  
|  On Mon, Dec 7, 2015, at 05:14, Simon Peyton Jones wrote:
|  > Plugins get to edit the entire core-to-core pipeline!  There is no
|  magic.
|  >  At least I don’t think so
|  >
|  > file:///Z:/tmp/users_guide/compiler-plugins.html
|  >
|  > S
|  >
|  > From: Levent Erkok [mailto:erk...@gmail.com]
|  > Sent: 07 December 2015 13:11
|  > To: Simon Peyton Jones <simo...@microsoft.com>
|  > Cc: Eric Seidel <e...@seidel.io>; omeraga...@gmail.com;
|  > ezy...@mit.edu; ghc-devs@haskell.org
|  > Subject: Re: Plugins: Accessing unexported bindings
|  >
|  > That's a good point; keeping all annotated bindings alive seems to
|  be
|  > an overkill..
|  >
|  > Regarding implementing "pass at the start." I'm not sure if plugin
|  > authors have any freedom as to decide when their plugin actually
|  runs.
|  > It seems GHC magically determines the order and runs them. Can you
|  > point me to some code/docs that tells me how to go "first" in that
|  > sense? (Or at least before the pass that drops dead code.)
|  >
|  > On Dec 7, 2015, at 4:45 AM, Simon Peyton Jones
|  > <simo...@microsoft.com<mailto:simo...@microsoft.com>> wrote:
|  > Indeed. How about this: if there's an ANN on a binder (any ANN),
|  then
|  > GHC should keep it alive.
|  >
|  > Really?  It might be something like “don’t give warnings for this
|  > binding” or “don’t inline me” or something.   To say *any*
|  annotation
|  > seems a bit brutal doesn’t it?Mind you I don’t have a better
|  idea.
|  >
|  > One thought: your plugin could add a pass right at the start, which
|  > marks everything you want as keep-alive.
|  >
|  > S
|  >
|  > From: Levent Erkok [mailto:erk...@gmail.com]
|  > Sent: 07 December 2015 12:42
|  > To: Simon Peyton Jones
|  > <simo...@microsoft.com<mailto:simo...@microsoft.com>>
|  > Cc: Eric Seidel <e...@seidel.io<mailto:e...@seidel.io>>;
|  > omeraga...@gmail.com<mailto:omeraga...@gmail.com>;
|  > ezy...@mit.edu<mailto:ezy...@mit.edu>;
|  > ghc-devs@haskell.org<mailto:ghc-devs@haskell.org>
|  > Subject: Re: Plugins: Accessing unexported bindings
|  >
|  > Indeed. How about this: if there's an ANN on a binder (any ANN),
|  then
|  > GHC should keep it alive.
|  >
|  > Is that something one of the core-developers can implement? Happy to
|  > open a ticket if that helps.
|  >
|  > On Dec 7, 2015, at 4:14 AM, Simon Peyton Jones
|  > <simo...@microsoft.com<mailto:simo...@microsoft.com>> wrote:
|  > If it's "dead" in this sense, it's already removed from ModGuts, no?
|  >
|  > Yes, if it’s dead it’s gone.   That’s not too surprising, is it?
|  >
|  > So you need a way to keep it alive. Maybe we need a pragma for that.
|  Or
|  > how would you like to signal it in the source code?
|  >
|  > Simon
|  >
|  > From: Levent Erkok [mailto:erk...@gmail.com]
|  > Sent: 07 December 2015 12:05
|  > To: Simon Peyton Jones
|  > <simo...@microsoft.com<mailto:simo...@microsoft.com>>
|  > Cc: Eric Seidel <e...@seidel.io<mailto:e...@seidel.io>>;
|  > omeraga...@gmail.com<mailto:omeraga...@gmail.com>;
|  > ezy...@mit.edu<mailto:ezy...@mit.edu>;
|  > ghc-devs@haskell.org<mailto:ghc-devs@haskell.org>
|  > Subject: Re: Plugins: Accessing unexported bindings
|  >
|  > Thanks Simon.. But I remain utterly confused. As a "plugin" author,
|  > how do I get my hands on the Id associated with a top-level binder?
|  If
|  > it's "dead" in this sense, it's already removed from ModGuts, no?
|  >
|  > That is, by the time GHC runs my plugin, the Id has already
|  > disappeared for me to mark it "Local Exported." Is that not correct?
|  >
|  > On Dec 7, 2015, at 2:28 AM, Simon Peyton Jones
|  > <simo...@microsoft.com<mailto:simo...@microsoft.com>> wrote:
|  > In the mean time, I'm still looking for a solution that doesn't
|  > involve exporting such identifiers from modules. As Eric pointed
|  out,
|

RE: Plugins: Accessing unexported bindings

2015-12-07 Thread Simon Peyton Jones 
In the mean time, I'm still looking for a solution that doesn't involve 
exporting such identifiers from modules. As Eric pointed out, that seems to be 
the only current work-around for the time being.

“Exported” in this context only means “keep alive”. It does not mean exported 
in the Haskell source code sense. I’ve just added this comment to Var.hs.

So I think it does just what you want.

Simon

data ExportFlag   -- See Note [ExportFlag on binders]
  = NotExported   -- ^ Not exported: may be discarded as dead code.
  | Exported  -- ^ Exported: kept alive

{- Note [ExportFlag on binders]
~~~
An ExportFlag of "Exported" on a top-level binder says "keep this
binding alive; do not drop it as dead code".  This transititively
keeps alive all the other top-level bindings that this binding refers
to.  This property is persisted all the way down the pipeline, so that
the binding will be compiled all the way to object code, and its
symbols will appear in the linker symbol table.

However, note that this use of "exported" is quite different to the
export list on a Haskell module.  Setting the ExportFlag on an Id does
/not/ mean that if you import the module (in Haskell source code you
will see this Id.  Of course, things that appear in the export list
of the source Haskell module do indeed have their ExportFlag set.
But many other things, such as dictionary functions, are kept alive
by having their ExportFlag set, even though they are not exported
in the source-code sense.

We should probably use a different term for ExportFlag, like
KeepAlive.

From: ghc-devs [mailto:ghc-devs-boun...@haskell.org] On Behalf Of Levent Erkok
Sent: 06 December 2015 20:32
To: Eric Seidel <e...@seidel.io>; omeraga...@gmail.com; ezy...@mit.edu
Cc: ghc-devs@haskell.org
Subject: Re: Plugins: Accessing unexported bindings

Omer, Eric, Ed: Thanks for the comments.

Omer: I think Eric's observation is at play here. We're talking about 
"dead-code," i.e., a binding that is neither exported, nor used by any binding 
inside the module. Those seem to be getting dropped by the time user-plugins 
are run. Unfortunately, this is precisely what one would do with "properties" 
embedded in code. They serve as documentation perhaps, but are otherwise not 
needed by any other binding nor it makes sense to export them.

Edward: Can you provide some more info into your solution? Sounds like a 
chicken-egg issue to me: As a plugin author, I need the bindings to access the 
Ids, and looks like I need the Ids to access the binders?

A simple solution would be to simply keep all top-level bindings around while 
the plugin are running, but that obviously can lead to unnecessary work if the 
code is truly dead. A compromise could be that the annotations can serve as 
entry points as well: I.e., if there's an annotation on a top-level binder, 
then it should *not* be considered dead-code at least until after all the 
plugins are run. That would definitely simplify life. Would that be an 
acceptable alternative?

In the mean time, I'm still looking for a solution that doesn't involve 
exporting such identifiers from modules. As Eric pointed out, that seems to be 
the only current work-around for the time being.

Thanks,

-Levent.

On Sun, Dec 6, 2015 at 11:08 AM, Eric Seidel 
<e...@seidel.io<mailto:e...@seidel.io>> wrote:
GHC should only drop un-exported bindings from the ModGuts if they're
also unused, ie *dead code*.

The only way I know to get around this is to use the bindings somewhere,
or just export them.

On Sat, Dec 5, 2015, at 23:01, Levent Erkok wrote:
> Hello,
>
> The mg_binds field of the ModGuts seem to only contain the bindings that
> are exported from the module being compiled.
>
> I guess GHC must be running user-plugins after it drops the bindings that
> are not exported, which makes perfect sense for most use cases. However,
> I'm working on a plugin where the end-programmer embeds "properties" in
> the
> form of functions inside his/her code, which are not necessarily exported
> from the module under consideration.
>
> Is there a way to access all top-level bindings in a module from a
> plugin,
> even if those bindings are not exported?
>
> Thanks,
>
> -Levent.
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org<mailto:ghc-devs@haskell.org>
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fmail.haskell.org%2fcgi-bin%2fmailman%2flistinfo%2fghc-devs=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7cac4cbfe22e314080909908d2fe7c4ed8%7c72f988bf86f141af91ab2d7cd011db47%7c1=1z6DcZxjIAKj0PcsLeALphRLWJ3i%2fxvyaPtq0qo6elY%3d>
___
ghc-devs mailing list
ghc-devs@haskell.org<mailto:ghc-devs@haskell.o

RE: Plugins: Accessing unexported bindings

2015-12-07 Thread Simon Peyton Jones 
If it's "dead" in this sense, it's already removed from ModGuts, no?

Yes, if it’s dead it’s gone.   That’s not too surprising, is it?

So you need a way to keep it alive. Maybe we need a pragma for that.   Or how 
would you like to signal it in the source code?

Simon

From: Levent Erkok [mailto:erk...@gmail.com]
Sent: 07 December 2015 12:05
To: Simon Peyton Jones <simo...@microsoft.com>
Cc: Eric Seidel <e...@seidel.io>; omeraga...@gmail.com; ezy...@mit.edu; 
ghc-devs@haskell.org
Subject: Re: Plugins: Accessing unexported bindings

Thanks Simon.. But I remain utterly confused. As a "plugin" author, how do I 
get my hands on the Id associated with a top-level binder? If it's "dead" in 
this sense, it's already removed from ModGuts, no?

That is, by the time GHC runs my plugin, the Id has already disappeared for me 
to mark it "Local Exported." Is that not correct?

On Dec 7, 2015, at 2:28 AM, Simon Peyton Jones 
<simo...@microsoft.com<mailto:simo...@microsoft.com>> wrote:
In the mean time, I'm still looking for a solution that doesn't involve 
exporting such identifiers from modules. As Eric pointed out, that seems to be 
the only current work-around for the time being.

“Exported” in this context only means “keep alive”. It does not mean exported 
in the Haskell source code sense. I’ve just added this comment to Var.hs.

So I think it does just what you want.

Simon

data ExportFlag   -- See Note [ExportFlag on binders]
  = NotExported   -- ^ Not exported: may be discarded as dead code.
  | Exported  -- ^ Exported: kept alive

{- Note [ExportFlag on binders]
~~~
An ExportFlag of "Exported" on a top-level binder says "keep this
binding alive; do not drop it as dead code".  This transititively
keeps alive all the other top-level bindings that this binding refers
to.  This property is persisted all the way down the pipeline, so that
the binding will be compiled all the way to object code, and its
symbols will appear in the linker symbol table.

However, note that this use of "exported" is quite different to the
export list on a Haskell module.  Setting the ExportFlag on an Id does
/not/ mean that if you import the module (in Haskell source code you
will see this Id.  Of course, things that appear in the export list
of the source Haskell module do indeed have their ExportFlag set.
But many other things, such as dictionary functions, are kept alive
by having their ExportFlag set, even though they are not exported
in the source-code sense.

We should probably use a different term for ExportFlag, like
KeepAlive.

From: ghc-devs [mailto:ghc-devs-boun...@haskell.org] On Behalf Of Levent Erkok
Sent: 06 December 2015 20:32
To: Eric Seidel <e...@seidel.io<mailto:e...@seidel.io>>; 
omeraga...@gmail.com<mailto:omeraga...@gmail.com>; 
ezy...@mit.edu<mailto:ezy...@mit.edu>
Cc: ghc-devs@haskell.org<mailto:ghc-devs@haskell.org>
Subject: Re: Plugins: Accessing unexported bindings

Omer, Eric, Ed: Thanks for the comments.

Omer: I think Eric's observation is at play here. We're talking about 
"dead-code," i.e., a binding that is neither exported, nor used by any binding 
inside the module. Those seem to be getting dropped by the time user-plugins 
are run. Unfortunately, this is precisely what one would do with "properties" 
embedded in code. They serve as documentation perhaps, but are otherwise not 
needed by any other binding nor it makes sense to export them.

Edward: Can you provide some more info into your solution? Sounds like a 
chicken-egg issue to me: As a plugin author, I need the bindings to access the 
Ids, and looks like I need the Ids to access the binders?

A simple solution would be to simply keep all top-level bindings around while 
the plugin are running, but that obviously can lead to unnecessary work if the 
code is truly dead. A compromise could be that the annotations can serve as 
entry points as well: I.e., if there's an annotation on a top-level binder, 
then it should *not* be considered dead-code at least until after all the 
plugins are run. That would definitely simplify life. Would that be an 
acceptable alternative?

In the mean time, I'm still looking for a solution that doesn't involve 
exporting such identifiers from modules. As Eric pointed out, that seems to be 
the only current work-around for the time being.

Thanks,

-Levent.

On Sun, Dec 6, 2015 at 11:08 AM, Eric Seidel 
<e...@seidel.io<mailto:e...@seidel.io>> wrote:
GHC should only drop un-exported bindings from the ModGuts if they're
also unused, ie *dead code*.

The only way I know to get around this is to use the bindings somewhere,
or just export them.

On Sat, Dec 5, 2015, at 23:01, Levent Erkok wrote:
> Hello,
>
> The mg_binds field of the ModGuts seem to only contain the bindings that
> are exported from the module being compiled

Re: Plugins: Accessing unexported bindings

2015-12-07 Thread Levent Erkok 
Thanks Simon.. But I remain utterly confused. As a "plugin" author, how do I 
get my hands on the Id associated with a top-level binder? If it's "dead" in 
this sense, it's already removed from ModGuts, no?

That is, by the time GHC runs my plugin, the Id has already disappeared for me 
to mark it "Local Exported." Is that not correct?

> On Dec 7, 2015, at 2:28 AM, Simon Peyton Jones <simo...@microsoft.com> wrote:
> 
> In the mean time, I'm still looking for a solution that doesn't involve 
> exporting such identifiers from modules. As Eric pointed out, that seems to 
> be the only current work-around for the time being.
>  
> “Exported” in this context only means “keep alive”. It does not mean exported 
> in the Haskell source code sense. I’ve just added this comment to Var.hs.
>  
> So I think it does just what you want.
>  
> Simon
>  
> data ExportFlag   -- See Note [ExportFlag on binders]
>   = NotExported   -- ^ Not exported: may be discarded as dead code.
>   | Exported  -- ^ Exported: kept alive
>  
> {- Note [ExportFlag on binders]
> ~~~
> An ExportFlag of "Exported" on a top-level binder says "keep this
> binding alive; do not drop it as dead code".  This transititively
> keeps alive all the other top-level bindings that this binding refers
> to.  This property is persisted all the way down the pipeline, so that
> the binding will be compiled all the way to object code, and its
> symbols will appear in the linker symbol table.
>  
> However, note that this use of "exported" is quite different to the
> export list on a Haskell module.  Setting the ExportFlag on an Id does
> /not/ mean that if you import the module (in Haskell source code you
> will see this Id.  Of course, things that appear in the export list
> of the source Haskell module do indeed have their ExportFlag set.
> But many other things, such as dictionary functions, are kept alive
> by having their ExportFlag set, even though they are not exported
> in the source-code sense.
>  
> We should probably use a different term for ExportFlag, like
> KeepAlive.
>  
> From: ghc-devs [mailto:ghc-devs-boun...@haskell.org] On Behalf Of Levent Erkok
> Sent: 06 December 2015 20:32
> To: Eric Seidel <e...@seidel.io>; omeraga...@gmail.com; ezy...@mit.edu
> Cc: ghc-devs@haskell.org
> Subject: Re: Plugins: Accessing unexported bindings
>  
> Omer, Eric, Ed: Thanks for the comments.
>  
> Omer: I think Eric's observation is at play here. We're talking about 
> "dead-code," i.e., a binding that is neither exported, nor used by any 
> binding inside the module. Those seem to be getting dropped by the time 
> user-plugins are run. Unfortunately, this is precisely what one would do with 
> "properties" embedded in code. They serve as documentation perhaps, but are 
> otherwise not needed by any other binding nor it makes sense to export them.
>  
> Edward: Can you provide some more info into your solution? Sounds like a 
> chicken-egg issue to me: As a plugin author, I need the bindings to access 
> the Ids, and looks like I need the Ids to access the binders?
>  
> A simple solution would be to simply keep all top-level bindings around while 
> the plugin are running, but that obviously can lead to unnecessary work if 
> the code is truly dead. A compromise could be that the annotations can serve 
> as entry points as well: I.e., if there's an annotation on a top-level 
> binder, then it should *not* be considered dead-code at least until after all 
> the plugins are run. That would definitely simplify life. Would that be an 
> acceptable alternative?
>  
> In the mean time, I'm still looking for a solution that doesn't involve 
> exporting such identifiers from modules. As Eric pointed out, that seems to 
> be the only current work-around for the time being.
>  
> Thanks,
>  
> -Levent.
>  
> On Sun, Dec 6, 2015 at 11:08 AM, Eric Seidel <e...@seidel.io> wrote:
> GHC should only drop un-exported bindings from the ModGuts if they're
> also unused, ie *dead code*.
> 
> The only way I know to get around this is to use the bindings somewhere,
> or just export them.
> 
> On Sat, Dec 5, 2015, at 23:01, Levent Erkok wrote:
> > Hello,
> >
> > The mg_binds field of the ModGuts seem to only contain the bindings that
> > are exported from the module being compiled.
> >
> > I guess GHC must be running user-plugins after it drops the bindings that
> > are not exported, which makes perfect sense for most use cases. However,
> > I'm working on a plugin where the end-programmer embeds "properties" in
> > the
> > form of functio

Re: Plugins: Accessing unexported bindings

2015-12-07 Thread Levent Erkok 
Indeed. How about this: if there's an ANN on a binder (any ANN), then GHC 
should keep it alive.

Is that something one of the core-developers can implement? Happy to open a 
ticket if that helps. 

> On Dec 7, 2015, at 4:14 AM, Simon Peyton Jones <simo...@microsoft.com> wrote:
> 
> If it's "dead" in this sense, it's already removed from ModGuts, no?
>  
> Yes, if it’s dead it’s gone.   That’s not too surprising, is it?
>  
> So you need a way to keep it alive. Maybe we need a pragma for that.   Or how 
> would you like to signal it in the source code?
>  
> Simon
>  
> From: Levent Erkok [mailto:erk...@gmail.com] 
> Sent: 07 December 2015 12:05
> To: Simon Peyton Jones <simo...@microsoft.com>
> Cc: Eric Seidel <e...@seidel.io>; omeraga...@gmail.com; ezy...@mit.edu; 
> ghc-devs@haskell.org
> Subject: Re: Plugins: Accessing unexported bindings
>  
> Thanks Simon.. But I remain utterly confused. As a "plugin" author, how do I 
> get my hands on the Id associated with a top-level binder? If it's "dead" in 
> this sense, it's already removed from ModGuts, no?
>  
> That is, by the time GHC runs my plugin, the Id has already disappeared for 
> me to mark it "Local Exported." Is that not correct?
> 
> On Dec 7, 2015, at 2:28 AM, Simon Peyton Jones <simo...@microsoft.com> wrote:
> 
> In the mean time, I'm still looking for a solution that doesn't involve 
> exporting such identifiers from modules. As Eric pointed out, that seems to 
> be the only current work-around for the time being.
>  
> “Exported” in this context only means “keep alive”. It does not mean exported 
> in the Haskell source code sense. I’ve just added this comment to Var.hs.
>  
> So I think it does just what you want.
>  
> Simon
>  
> data ExportFlag   -- See Note [ExportFlag on binders]
>   = NotExported   -- ^ Not exported: may be discarded as dead code.
>   | Exported  -- ^ Exported: kept alive
>  
> {- Note [ExportFlag on binders]
> ~~~
> An ExportFlag of "Exported" on a top-level binder says "keep this
> binding alive; do not drop it as dead code".  This transititively
> keeps alive all the other top-level bindings that this binding refers
> to.  This property is persisted all the way down the pipeline, so that
> the binding will be compiled all the way to object code, and its
> symbols will appear in the linker symbol table.
>  
> However, note that this use of "exported" is quite different to the
> export list on a Haskell module.  Setting the ExportFlag on an Id does
> /not/ mean that if you import the module (in Haskell source code you
> will see this Id.  Of course, things that appear in the export list
> of the source Haskell module do indeed have their ExportFlag set.
> But many other things, such as dictionary functions, are kept alive
> by having their ExportFlag set, even though they are not exported
> in the source-code sense.
>  
> We should probably use a different term for ExportFlag, like
> KeepAlive.
>  
> From: ghc-devs [mailto:ghc-devs-boun...@haskell.org] On Behalf Of Levent Erkok
> Sent: 06 December 2015 20:32
> To: Eric Seidel <e...@seidel.io>; omeraga...@gmail.com; ezy...@mit.edu
> Cc: ghc-devs@haskell.org
> Subject: Re: Plugins: Accessing unexported bindings
>  
> Omer, Eric, Ed: Thanks for the comments.
>  
> Omer: I think Eric's observation is at play here. We're talking about 
> "dead-code," i.e., a binding that is neither exported, nor used by any 
> binding inside the module. Those seem to be getting dropped by the time 
> user-plugins are run. Unfortunately, this is precisely what one would do with 
> "properties" embedded in code. They serve as documentation perhaps, but are 
> otherwise not needed by any other binding nor it makes sense to export them.
>  
> Edward: Can you provide some more info into your solution? Sounds like a 
> chicken-egg issue to me: As a plugin author, I need the bindings to access 
> the Ids, and looks like I need the Ids to access the binders?
>  
> A simple solution would be to simply keep all top-level bindings around while 
> the plugin are running, but that obviously can lead to unnecessary work if 
> the code is truly dead. A compromise could be that the annotations can serve 
> as entry points as well: I.e., if there's an annotation on a top-level 
> binder, then it should *not* be considered dead-code at least until after all 
> the plugins are run. That would definitely simplify life. Would that be an 
> acceptable alternative?
>  
> In the mean time, I'm still looking for a solution that doesn't involve 
> exporting such identifiers from modules.

RE: Plugins: Accessing unexported bindings

2015-12-07 Thread Simon Peyton Jones 
Indeed. How about this: if there's an ANN on a binder (any ANN), then GHC 
should keep it alive.

Really?  It might be something like “don’t give warnings for this binding” or 
“don’t inline me” or something.   To say *any* annotation seems a bit brutal 
doesn’t it?Mind you I don’t have a better idea.

One thought: your plugin could add a pass right at the start, which marks 
everything you want as keep-alive.

S

From: Levent Erkok [mailto:erk...@gmail.com]
Sent: 07 December 2015 12:42
To: Simon Peyton Jones <simo...@microsoft.com>
Cc: Eric Seidel <e...@seidel.io>; omeraga...@gmail.com; ezy...@mit.edu; 
ghc-devs@haskell.org
Subject: Re: Plugins: Accessing unexported bindings

Indeed. How about this: if there's an ANN on a binder (any ANN), then GHC 
should keep it alive.

Is that something one of the core-developers can implement? Happy to open a 
ticket if that helps.

On Dec 7, 2015, at 4:14 AM, Simon Peyton Jones 
<simo...@microsoft.com<mailto:simo...@microsoft.com>> wrote:
If it's "dead" in this sense, it's already removed from ModGuts, no?

Yes, if it’s dead it’s gone.   That’s not too surprising, is it?

So you need a way to keep it alive. Maybe we need a pragma for that.   Or how 
would you like to signal it in the source code?

Simon

From: Levent Erkok [mailto:erk...@gmail.com]
Sent: 07 December 2015 12:05
To: Simon Peyton Jones <simo...@microsoft.com<mailto:simo...@microsoft.com>>
Cc: Eric Seidel <e...@seidel.io<mailto:e...@seidel.io>>; 
omeraga...@gmail.com<mailto:omeraga...@gmail.com>; 
ezy...@mit.edu<mailto:ezy...@mit.edu>; 
ghc-devs@haskell.org<mailto:ghc-devs@haskell.org>
Subject: Re: Plugins: Accessing unexported bindings

Thanks Simon.. But I remain utterly confused. As a "plugin" author, how do I 
get my hands on the Id associated with a top-level binder? If it's "dead" in 
this sense, it's already removed from ModGuts, no?

That is, by the time GHC runs my plugin, the Id has already disappeared for me 
to mark it "Local Exported." Is that not correct?

On Dec 7, 2015, at 2:28 AM, Simon Peyton Jones 
<simo...@microsoft.com<mailto:simo...@microsoft.com>> wrote:
In the mean time, I'm still looking for a solution that doesn't involve 
exporting such identifiers from modules. As Eric pointed out, that seems to be 
the only current work-around for the time being.

“Exported” in this context only means “keep alive”. It does not mean exported 
in the Haskell source code sense. I’ve just added this comment to Var.hs.

So I think it does just what you want.

Simon

data ExportFlag   -- See Note [ExportFlag on binders]
  = NotExported   -- ^ Not exported: may be discarded as dead code.
  | Exported  -- ^ Exported: kept alive

{- Note [ExportFlag on binders]
~~~
An ExportFlag of "Exported" on a top-level binder says "keep this
binding alive; do not drop it as dead code".  This transititively
keeps alive all the other top-level bindings that this binding refers
to.  This property is persisted all the way down the pipeline, so that
the binding will be compiled all the way to object code, and its
symbols will appear in the linker symbol table.

However, note that this use of "exported" is quite different to the
export list on a Haskell module.  Setting the ExportFlag on an Id does
/not/ mean that if you import the module (in Haskell source code you
will see this Id.  Of course, things that appear in the export list
of the source Haskell module do indeed have their ExportFlag set.
But many other things, such as dictionary functions, are kept alive
by having their ExportFlag set, even though they are not exported
in the source-code sense.

We should probably use a different term for ExportFlag, like
KeepAlive.

From: ghc-devs [mailto:ghc-devs-boun...@haskell.org] On Behalf Of Levent Erkok
Sent: 06 December 2015 20:32
To: Eric Seidel <e...@seidel.io<mailto:e...@seidel.io>>; 
omeraga...@gmail.com<mailto:omeraga...@gmail.com>; 
ezy...@mit.edu<mailto:ezy...@mit.edu>
Cc: ghc-devs@haskell.org<mailto:ghc-devs@haskell.org>
Subject: Re: Plugins: Accessing unexported bindings

Omer, Eric, Ed: Thanks for the comments.

Omer: I think Eric's observation is at play here. We're talking about 
"dead-code," i.e., a binding that is neither exported, nor used by any binding 
inside the module. Those seem to be getting dropped by the time user-plugins 
are run. Unfortunately, this is precisely what one would do with "properties" 
embedded in code. They serve as documentation perhaps, but are otherwise not 
needed by any other binding nor it makes sense to export them.

Edward: Can you provide some more info into your solution? Sounds like a 
chicken-egg issue to me: As a plugin author, I need the bindings to access the 
Ids, and looks like I need the Ids to access the binders?

Re: Plugins: Accessing unexported bindings

2015-12-07 Thread Levent Erkok 
That's a good point; keeping all annotated bindings alive seems to be an 
overkill..

Regarding implementing "pass at the start." I'm not sure if plugin authors have 
any freedom as to decide when their plugin actually runs. It seems GHC 
magically determines the order and runs them. Can you point me to some 
code/docs that tells me how to go "first" in that sense? (Or at least before 
the pass that drops dead code.)

> On Dec 7, 2015, at 4:45 AM, Simon Peyton Jones <simo...@microsoft.com> wrote:
> 
> Indeed. How about this: if there's an ANN on a binder (any ANN), then GHC 
> should keep it alive.
>  
> Really?  It might be something like “don’t give warnings for this binding” or 
> “don’t inline me” or something.   To say *any* annotation seems a bit brutal 
> doesn’t it?Mind you I don’t have a better idea.
>  
> One thought: your plugin could add a pass right at the start, which marks 
> everything you want as keep-alive.
>  
> S
>  
> From: Levent Erkok [mailto:erk...@gmail.com] 
> Sent: 07 December 2015 12:42
> To: Simon Peyton Jones <simo...@microsoft.com>
> Cc: Eric Seidel <e...@seidel.io>; omeraga...@gmail.com; ezy...@mit.edu; 
> ghc-devs@haskell.org
> Subject: Re: Plugins: Accessing unexported bindings
>  
> Indeed. How about this: if there's an ANN on a binder (any ANN), then GHC 
> should keep it alive.
>  
> Is that something one of the core-developers can implement? Happy to open a 
> ticket if that helps. 
>  
> On Dec 7, 2015, at 4:14 AM, Simon Peyton Jones <simo...@microsoft.com> wrote:
> 
> If it's "dead" in this sense, it's already removed from ModGuts, no?
>  
> Yes, if it’s dead it’s gone.   That’s not too surprising, is it?
>  
> So you need a way to keep it alive. Maybe we need a pragma for that.   Or how 
> would you like to signal it in the source code?
>  
> Simon
>  
> From: Levent Erkok [mailto:erk...@gmail.com] 
> Sent: 07 December 2015 12:05
> To: Simon Peyton Jones <simo...@microsoft.com>
> Cc: Eric Seidel <e...@seidel.io>; omeraga...@gmail.com; ezy...@mit.edu; 
> ghc-devs@haskell.org
> Subject: Re: Plugins: Accessing unexported bindings
>  
> Thanks Simon.. But I remain utterly confused. As a "plugin" author, how do I 
> get my hands on the Id associated with a top-level binder? If it's "dead" in 
> this sense, it's already removed from ModGuts, no?
>  
> That is, by the time GHC runs my plugin, the Id has already disappeared for 
> me to mark it "Local Exported." Is that not correct?
> 
> On Dec 7, 2015, at 2:28 AM, Simon Peyton Jones <simo...@microsoft.com> wrote:
> 
> In the mean time, I'm still looking for a solution that doesn't involve 
> exporting such identifiers from modules. As Eric pointed out, that seems to 
> be the only current work-around for the time being.
>  
> “Exported” in this context only means “keep alive”. It does not mean exported 
> in the Haskell source code sense. I’ve just added this comment to Var.hs.
>  
> So I think it does just what you want.
>  
> Simon
>  
> data ExportFlag   -- See Note [ExportFlag on binders]
>   = NotExported   -- ^ Not exported: may be discarded as dead code.
>   | Exported  -- ^ Exported: kept alive
>  
> {- Note [ExportFlag on binders]
> ~~~
> An ExportFlag of "Exported" on a top-level binder says "keep this
> binding alive; do not drop it as dead code".  This transititively
> keeps alive all the other top-level bindings that this binding refers
> to.  This property is persisted all the way down the pipeline, so that
> the binding will be compiled all the way to object code, and its
> symbols will appear in the linker symbol table.
>  
> However, note that this use of "exported" is quite different to the
> export list on a Haskell module.  Setting the ExportFlag on an Id does
> /not/ mean that if you import the module (in Haskell source code you
> will see this Id.  Of course, things that appear in the export list
> of the source Haskell module do indeed have their ExportFlag set.
> But many other things, such as dictionary functions, are kept alive
> by having their ExportFlag set, even though they are not exported
> in the source-code sense.
>  
> We should probably use a different term for ExportFlag, like
> KeepAlive.
>  
> From: ghc-devs [mailto:ghc-devs-boun...@haskell.org] On Behalf Of Levent Erkok
> Sent: 06 December 2015 20:32
> To: Eric Seidel <e...@seidel.io>; omeraga...@gmail.com; ezy...@mit.edu
> Cc: ghc-devs@haskell.org
> Subject: Re: Plugins: Accessing unexported bindings
>  
> Omer, Eric, Ed: Thanks for the comments.
>  
> Omer: I think Eri

RE: Plugins: Accessing unexported bindings

2015-12-07 Thread Simon Peyton Jones 
Plugins get to edit the entire core-to-core pipeline!  There is no magic.  At 
least I don’t think so

file:///Z:/tmp/users_guide/compiler-plugins.html

S

From: Levent Erkok [mailto:erk...@gmail.com]
Sent: 07 December 2015 13:11
To: Simon Peyton Jones <simo...@microsoft.com>
Cc: Eric Seidel <e...@seidel.io>; omeraga...@gmail.com; ezy...@mit.edu; 
ghc-devs@haskell.org
Subject: Re: Plugins: Accessing unexported bindings

That's a good point; keeping all annotated bindings alive seems to be an 
overkill..

Regarding implementing "pass at the start." I'm not sure if plugin authors have 
any freedom as to decide when their plugin actually runs. It seems GHC 
magically determines the order and runs them. Can you point me to some 
code/docs that tells me how to go "first" in that sense? (Or at least before 
the pass that drops dead code.)

On Dec 7, 2015, at 4:45 AM, Simon Peyton Jones 
<simo...@microsoft.com<mailto:simo...@microsoft.com>> wrote:
Indeed. How about this: if there's an ANN on a binder (any ANN), then GHC 
should keep it alive.

Really?  It might be something like “don’t give warnings for this binding” or 
“don’t inline me” or something.   To say *any* annotation seems a bit brutal 
doesn’t it?Mind you I don’t have a better idea.

One thought: your plugin could add a pass right at the start, which marks 
everything you want as keep-alive.

S

From: Levent Erkok [mailto:erk...@gmail.com]
Sent: 07 December 2015 12:42
To: Simon Peyton Jones <simo...@microsoft.com<mailto:simo...@microsoft.com>>
Cc: Eric Seidel <e...@seidel.io<mailto:e...@seidel.io>>; 
omeraga...@gmail.com<mailto:omeraga...@gmail.com>; 
ezy...@mit.edu<mailto:ezy...@mit.edu>; 
ghc-devs@haskell.org<mailto:ghc-devs@haskell.org>
Subject: Re: Plugins: Accessing unexported bindings

Indeed. How about this: if there's an ANN on a binder (any ANN), then GHC 
should keep it alive.

Is that something one of the core-developers can implement? Happy to open a 
ticket if that helps.

On Dec 7, 2015, at 4:14 AM, Simon Peyton Jones 
<simo...@microsoft.com<mailto:simo...@microsoft.com>> wrote:
If it's "dead" in this sense, it's already removed from ModGuts, no?

Yes, if it’s dead it’s gone.   That’s not too surprising, is it?

So you need a way to keep it alive. Maybe we need a pragma for that.   Or how 
would you like to signal it in the source code?

Simon

From: Levent Erkok [mailto:erk...@gmail.com]
Sent: 07 December 2015 12:05
To: Simon Peyton Jones <simo...@microsoft.com<mailto:simo...@microsoft.com>>
Cc: Eric Seidel <e...@seidel.io<mailto:e...@seidel.io>>; 
omeraga...@gmail.com<mailto:omeraga...@gmail.com>; 
ezy...@mit.edu<mailto:ezy...@mit.edu>; 
ghc-devs@haskell.org<mailto:ghc-devs@haskell.org>
Subject: Re: Plugins: Accessing unexported bindings

Thanks Simon.. But I remain utterly confused. As a "plugin" author, how do I 
get my hands on the Id associated with a top-level binder? If it's "dead" in 
this sense, it's already removed from ModGuts, no?

That is, by the time GHC runs my plugin, the Id has already disappeared for me 
to mark it "Local Exported." Is that not correct?

On Dec 7, 2015, at 2:28 AM, Simon Peyton Jones 
<simo...@microsoft.com<mailto:simo...@microsoft.com>> wrote:
In the mean time, I'm still looking for a solution that doesn't involve 
exporting such identifiers from modules. As Eric pointed out, that seems to be 
the only current work-around for the time being.

“Exported” in this context only means “keep alive”. It does not mean exported 
in the Haskell source code sense. I’ve just added this comment to Var.hs.

So I think it does just what you want.

Simon

data ExportFlag   -- See Note [ExportFlag on binders]
  = NotExported   -- ^ Not exported: may be discarded as dead code.
  | Exported  -- ^ Exported: kept alive

{- Note [ExportFlag on binders]
~~~
An ExportFlag of "Exported" on a top-level binder says "keep this
binding alive; do not drop it as dead code".  This transititively
keeps alive all the other top-level bindings that this binding refers
to.  This property is persisted all the way down the pipeline, so that
the binding will be compiled all the way to object code, and its
symbols will appear in the linker symbol table.

However, note that this use of "exported" is quite different to the
export list on a Haskell module.  Setting the ExportFlag on an Id does
/not/ mean that if you import the module (in Haskell source code you
will see this Id.  Of course, things that appear in the export list
of the source Haskell module do indeed have their ExportFlag set.
But many other things, such as dictionary functions, are kept alive
by having their ExportFlag set, even though they are not exported
in the source-code sense.

We should probably use a different te

Re: Plugins: Accessing unexported bindings

2015-12-07 Thread Levent Erkok 
Thanks for the explanation, Simon. I think Eric is spot on. Looking at some
of the "ghc -v3" output, it appears GHC always runs the following passes
before any plugins are run:

   *** Simplify:
   *** CorePrep:
   *** ByteCodeGen:
   *** Desugar:

Just judging by the name, it could either be the first "Simplify" or the
"Desugar" that gets rid of the dead-code at this point, I'm not sure which.

It would help if this stage didn't remove dead-bindings. If doing so
automatically is not the best option, requiring a user given pragma like
'KeepAlive' wouldn't be too onerous either.

-Levent.

On Mon, Dec 7, 2015 at 8:22 AM, Simon Peyton Jones <simo...@microsoft.com>
wrote:

> It would not be hard to stop the desugarer dropping dead bindings, if that
> was helpful.
>
> S
>
> |  -Original Message-
> |  From: Eric Seidel [mailto:e...@seidel.io]
> |  Sent: 07 December 2015 15:44
> |  To: Simon Peyton Jones <simo...@microsoft.com>
> |  Cc: Levent Erkok <erk...@gmail.com>; omeraga...@gmail.com;
> |  ezy...@mit.edu; ghc-devs@haskell.org
> |  Subject: Re: Plugins: Accessing unexported bindings
> |
> |  The problem, as I recall, is that GHC does an initial bit of dead-code
> |  elimination in the desugarer, before the plugins have a chance to run.
> |  (I believe this is part of simpleOptPgm, but may be mistaken)
> |
> |  I'm not sure why this is done in the desugarer, it seems to be out of
> |  place there.
> |
> |  On Mon, Dec 7, 2015, at 05:14, Simon Peyton Jones wrote:
> |  > Plugins get to edit the entire core-to-core pipeline!  There is no
> |  magic.
> |  >  At least I don’t think so
> |  >
> |  > file:///Z:/tmp/users_guide/compiler-plugins.html
> |  >
> |  > S
> |  >
> |  > From: Levent Erkok [mailto:erk...@gmail.com]
> |  > Sent: 07 December 2015 13:11
> |  > To: Simon Peyton Jones <simo...@microsoft.com>
> |  > Cc: Eric Seidel <e...@seidel.io>; omeraga...@gmail.com;
> |  > ezy...@mit.edu; ghc-devs@haskell.org
> |  > Subject: Re: Plugins: Accessing unexported bindings
> |  >
> |  > That's a good point; keeping all annotated bindings alive seems to
> |  be
> |  > an overkill..
> |  >
> |  > Regarding implementing "pass at the start." I'm not sure if plugin
> |  > authors have any freedom as to decide when their plugin actually
> |  runs.
> |  > It seems GHC magically determines the order and runs them. Can you
> |  > point me to some code/docs that tells me how to go "first" in that
> |  > sense? (Or at least before the pass that drops dead code.)
> |  >
> |  > On Dec 7, 2015, at 4:45 AM, Simon Peyton Jones
> |  > <simo...@microsoft.com<mailto:simo...@microsoft.com>> wrote:
> |  > Indeed. How about this: if there's an ANN on a binder (any ANN),
> |  then
> |  > GHC should keep it alive.
> |  >
> |  > Really?  It might be something like “don’t give warnings for this
> |  > binding” or “don’t inline me” or something.   To say *any*
> |  annotation
> |  > seems a bit brutal doesn’t it?Mind you I don’t have a better
> |  idea.
> |  >
> |  > One thought: your plugin could add a pass right at the start, which
> |  > marks everything you want as keep-alive.
> |  >
> |  > S
> |  >
> |  > From: Levent Erkok [mailto:erk...@gmail.com]
> |  > Sent: 07 December 2015 12:42
> |  > To: Simon Peyton Jones
> |  > <simo...@microsoft.com<mailto:simo...@microsoft.com>>
> |  > Cc: Eric Seidel <e...@seidel.io<mailto:e...@seidel.io>>;
> |  > omeraga...@gmail.com<mailto:omeraga...@gmail.com>;
> |  > ezy...@mit.edu<mailto:ezy...@mit.edu>;
> |  > ghc-devs@haskell.org<mailto:ghc-devs@haskell.org>
> |  > Subject: Re: Plugins: Accessing unexported bindings
> |  >
> |  > Indeed. How about this: if there's an ANN on a binder (any ANN),
> |  then
> |  > GHC should keep it alive.
> |  >
> |  > Is that something one of the core-developers can implement? Happy to
> |  > open a ticket if that helps.
> |  >
> |  > On Dec 7, 2015, at 4:14 AM, Simon Peyton Jones
> |  > <simo...@microsoft.com<mailto:simo...@microsoft.com>> wrote:
> |  > If it's "dead" in this sense, it's already removed from ModGuts, no?
> |  >
> |  > Yes, if it’s dead it’s gone.   That’s not too surprising, is it?
> |  >
> |  > So you need a way to keep it alive. Maybe we need a pragma for that.
> |  Or
> |  > how would you like to signal it in the source code?
> |  >
> |  > Simon
> |  >
> |  > From: Levent Erkok [mailto:erk...@gmail.com]
> |  &g

Re: Plugins: Accessing unexported bindings

2015-12-06 Thread Edward Z. Yang 
If you have control over the Ids for the properties, if you can
arrange for them to be marked "local exported" that should solve
your problem.

Edward

Excerpts from Levent Erkok's message of 2015-12-05 23:01:29 -0800:
> Hello,
> 
> The mg_binds field of the ModGuts seem to only contain the bindings that
> are exported from the module being compiled.
> 
> I guess GHC must be running user-plugins after it drops the bindings that
> are not exported, which makes perfect sense for most use cases. However,
> I'm working on a plugin where the end-programmer embeds "properties" in the
> form of functions inside his/her code, which are not necessarily exported
> from the module under consideration.
> 
> Is there a way to access all top-level bindings in a module from a plugin,
> even if those bindings are not exported?
> 
> Thanks,
> 
> -Levent.
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Plugins: Accessing unexported bindings

2015-12-06 Thread Ömer Sinan Ağacan 
2015-12-06 2:01 GMT-05:00 Levent Erkok :
> The mg_binds field of the ModGuts seem to only contain the bindings that are
> exported from the module being compiled.

This is not true, it contains all the definitions in the module and I'm relying
on this all the time. I just tested again and it definitely has all the
definitions in the module, exported or not.

If it doesn't export everything in the module some optimizers couldn't really
work. At least in my case the plugin API would be basically useless.

May I ask how are you testing this?
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Plugins: Accessing unexported bindings

2015-12-06 Thread Levent Erkok 
Omer, Eric, Ed: Thanks for the comments.

Omer: I think Eric's observation is at play here. We're talking about
"dead-code," i.e., a binding that is neither exported, nor used by any
binding inside the module. Those seem to be getting dropped by the time
user-plugins are run. Unfortunately, this is precisely what one would do
with "properties" embedded in code. They serve as documentation perhaps,
but are otherwise not needed by any other binding nor it makes sense to
export them.

Edward: Can you provide some more info into your solution? Sounds like a
chicken-egg issue to me: As a plugin author, I need the bindings to access
the Ids, and looks like I need the Ids to access the binders?

A simple solution would be to simply keep all top-level bindings around
while the plugin are running, but that obviously can lead to unnecessary
work if the code is truly dead. A compromise could be that the annotations
can serve as entry points as well: I.e., if there's an annotation on a
top-level binder, then it should *not* be considered dead-code at least
until after all the plugins are run. That would definitely simplify life.
Would that be an acceptable alternative?

In the mean time, I'm still looking for a solution that doesn't involve
exporting such identifiers from modules. As Eric pointed out, that seems to
be the only current work-around for the time being.

Thanks,

-Levent.

On Sun, Dec 6, 2015 at 11:08 AM, Eric Seidel  wrote:

> GHC should only drop un-exported bindings from the ModGuts if they're
> also unused, ie *dead code*.
>
> The only way I know to get around this is to use the bindings somewhere,
> or just export them.
>
> On Sat, Dec 5, 2015, at 23:01, Levent Erkok wrote:
> > Hello,
> >
> > The mg_binds field of the ModGuts seem to only contain the bindings that
> > are exported from the module being compiled.
> >
> > I guess GHC must be running user-plugins after it drops the bindings that
> > are not exported, which makes perfect sense for most use cases. However,
> > I'm working on a plugin where the end-programmer embeds "properties" in
> > the
> > form of functions inside his/her code, which are not necessarily exported
> > from the module under consideration.
> >
> > Is there a way to access all top-level bindings in a module from a
> > plugin,
> > even if those bindings are not exported?
> >
> > Thanks,
> >
> > -Levent.
> > ___
> > ghc-devs mailing list
> > ghc-devs@haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Plugins: Accessing unexported bindings

2015-12-06 Thread Eric Seidel 
GHC should only drop un-exported bindings from the ModGuts if they're
also unused, ie *dead code*.

The only way I know to get around this is to use the bindings somewhere,
or just export them.

On Sat, Dec 5, 2015, at 23:01, Levent Erkok wrote:
> Hello,
> 
> The mg_binds field of the ModGuts seem to only contain the bindings that
> are exported from the module being compiled.
> 
> I guess GHC must be running user-plugins after it drops the bindings that
> are not exported, which makes perfect sense for most use cases. However,
> I'm working on a plugin where the end-programmer embeds "properties" in
> the
> form of functions inside his/her code, which are not necessarily exported
> from the module under consideration.
> 
> Is there a way to access all top-level bindings in a module from a
> plugin,
> even if those bindings are not exported?
> 
> Thanks,
> 
> -Levent.
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs