Re: TTG: Handling Source Locations

[Shayan Najd]

* "About the latter, until #15247 is fixed" ---> "About the latter,
until #15884 is fixed"

On Wed, 13 Feb 2019 at 14:00, Shayan Najd  wrote:
>
> >is there any plan to get rid of all those panics?
>
> There are two sorts of panics related to TTG: the ones due to #15247
> (i.e. unused extension constructors), and the ones due to #15884 (i.e.
> issues with view patterns).
>
> About the former, I believe we all agree. Moreover, using Solution A
> discussed above, there will be way less unused extension constructors
> anyway: HsSyn types will use their extension constructors for the
> location wrapper constructor.
> About the latter, until #15247 is fixed, we can do rewrites as Ryan
> suggests. Hopefully, there will also be less of such panics around
> after making the code idiomatic to match Solution A discussed above.
>
> /Shayan
>
> On Wed, 13 Feb 2019 at 12:07, Ryan Scott  wrote:
> >
> > Yes, I agree. This will require sprinkling the codebase with EmptyCase due 
> > to [1], but that's still a sight better than calling `panic`. After GHC 
> > 8.10 is released (and the minimum version of GHC that HEAD supports is 
> > 8.8), we can even remove these empty cases by making the empty data type 
> > fields strict (see [2]).
> >
> > Ryan S.
> > -
> > [1] https://ghc.haskell.org/trac/ghc/ticket/15247#comment:4
> > [2] https://ghc.haskell.org/trac/ghc/ticket/15305
> > ___
> > ghc-devs mailing list
> > ghc-devs@haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: TTG: Handling Source Locations

[Shayan Najd]

>is there any plan to get rid of all those panics?

There are two sorts of panics related to TTG: the ones due to #15247
(i.e. unused extension constructors), and the ones due to #15884 (i.e.
issues with view patterns).

About the former, I believe we all agree. Moreover, using Solution A
discussed above, there will be way less unused extension constructors
anyway: HsSyn types will use their extension constructors for the
location wrapper constructor.
About the latter, until #15247 is fixed, we can do rewrites as Ryan
suggests. Hopefully, there will also be less of such panics around
after making the code idiomatic to match Solution A discussed above.

/Shayan

On Wed, 13 Feb 2019 at 12:07, Ryan Scott  wrote:
>
> Yes, I agree. This will require sprinkling the codebase with EmptyCase due to 
> [1], but that's still a sight better than calling `panic`. After GHC 8.10 is 
> released (and the minimum version of GHC that HEAD supports is 8.8), we can 
> even remove these empty cases by making the empty data type fields strict 
> (see [2]).
>
> Ryan S.
> -
> [1] https://ghc.haskell.org/trac/ghc/ticket/15247#comment:4
> [2] https://ghc.haskell.org/trac/ghc/ticket/15305
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: TTG: Handling Source Locations

[Shayan Najd]

> Someone could easily call rnPatAndThen when they should call rnLPatAndThen. 
> This would cause a panic.

With Solution A, there shouldn't be two functions `rnLPatAndThen` and
`rnPatAndThen` anyways. There should be only `rnPatAndThen` with an
extra case for the wrapper node.

> There's also the problem that the pattern-match checker can't usefully look 
> through view patterns.

Yes, I have reported it while back. I don't know of the progress in fixing this.

On Tue, 12 Feb 2019 at 16:24, Richard Eisenberg  wrote:
>
> That's true, but how would it play out in practice? For example, take a look 
> at RnPat. There is a rnLPatAndThen which uses wrapSrcSpanCps to extract the 
> location and then call rnPatAndThen. rnPatAndThen, in turn, just panics if it 
> sees the extension point, because that's an unexpected constructor. Someone 
> could easily call rnPatAndThen when they should call rnLPatAndThen. This 
> would cause a panic.
>
> There's also the problem that the pattern-match checker can't usefully look 
> through view patterns. If there is a nested pattern-match (that is, we see 
> dL->L _ (SomeOtherConstructor), then there is no way to guarantee a complete 
> pattern-match short of a catch-all. So it doesn't seem to me that the 
> pattern-match checker is really helping us achieve what we want here.
>
> Richard
>
> > On Feb 12, 2019, at 9:30 AM, Shayan Najd  wrote:
> >
> >> My problem, though, is that this is just a convention -- no one checks it. 
> >> It would be easy to forget.
> >
> > I am not sure if I understand: shouldn't the totality checker warn if
> > there is no pattern for the wrapper constructor (hence enforce the
> > convention)?
> >
> >
> > On Tue, 12 Feb 2019 at 15:19, Richard Eisenberg  
> > wrote:
> >>
> >>
> >>
> >>> On Feb 12, 2019, at 5:19 AM, Shayan Najd  wrote:
> >>>
> >>> About the new code, the convention is straightforward: anytime you
> >>> destruct an AST node, assume a wrapper node inside (add an extra
> >>> case), or use the smart constructors/pattern synonyms.
> >>
> >> Aha! This, I did not know. So, you're saying that all the consumers of the 
> >> GHC AST need to remember to use dL every time they pattern-match. With the 
> >> new design, using dL when it's unnecessary doesn't hurt, but forgetting it 
> >> is problematic. So: just use it every time. My problem, though, is that 
> >> this is just a convention -- no one checks it. It would be easy to forget.
> >>
> >>> On Feb 12, 2019, at 6:00 AM, Simon Peyton Jones via ghc-devs 
> >>>  wrote:
> >>>
> >>> One way to think of it is this: we can now put SrcSpans where they make 
> >>> sense, rather than everywhere.
> >>
> >> This has some logic to it, but I'm not quite sold. Another way of saying 
> >> this is that the new design favors flexibility for the producer, at the 
> >> cost of requiring consumers to be aware of and consistently apply the 
> >> convention Shayan describes above. The problem is, though, that if the 
> >> producer is stingy in adding source locations, the consumer won't know 
> >> which locations are expected to be informative. Is the consumer expected 
> >> to collect locations from a variety of places and try to combine them 
> >> somehow? I doubt it. So this means that the flexibility for the producer 
> >> isn't really there -- the type system will accept arbitrary choices of 
> >> where to put locations, but consumers won't get the locations where they 
> >> expect them.
> >>
> >>>  We can still say (Located t) in places where we want to guarantee a 
> >>> SrcSpan.
> >>
> >> This seems to go against the TTG philosophy. We can do this in, say, the 
> >> return type of a function, but we can't in the AST proper, because that's 
> >> shared among a number of clients, some of whom don't want the source 
> >> locations.
> >>
> >>>
> >>> Yes, this lets us add more than one; that's redundant but not harmful.
> >>
> >> I disagree here. If we add locations to a node twice, then we'll have to 
> >> use dL twice to find the underlying constructor. This is another case 
> >> there the type system offers the producer flexibility but hamstrings the 
> >> consumer.
> >>
> >>
> >>> On Feb 12, 2019, at 7:32 AM, Vladislav Zavialov  
> >>> wrote:
> >>>
> >>> I claim an SrcSpan makes sense everywhere, so this is not a useful
> >>&g

Re: TTG: Handling Source Locations

[Shayan Najd]

> [Richard:] I disagree here. If we add locations to a node twice, then we'll 
> have to use dL twice to find the underlying constructor. This is another case 
> there the type system offers the producer flexibility but hamstrings the 
> consumer.

Depends on the semantics of `dL`: currently  (for `Pat`) it returns
the top-level `SrcSpan` and then the underlying node with all the
inner wrappers stripped away. So one use of `dL` is enough in this
semantic.
(see https://github.com/ghc/ghc/blob/master/compiler/hsSyn/HsPat.hs#L341)

> [Vlad:] As to the better solution, I think we should just go with Solution B 
> from the Wiki page.
> [Richard:] I support this counter-proposal. Perhaps if it required writing 
> loads of extra type instances, I wouldn't be as much in favour.

It may help to identify at least three sorts of functions commonly
used *currently* in GHC when interacting with AST nodes (please add,
if I am missing some):
(a) those that ignore source locations;
(b) those that generically handle source locations regardless of the
constructor of the underlying node; and
(c) those that handle source locations case-by-case (often by nested
pattern matching).

The key issue with Solution B, as listed in the wiki, is that it ruins
the separation of two concerns in functions working on AST nodes:
handling source locations, and the actual logic of the function.
With the ping-pong style, handling of source locations is sometimes
refactored in a separate function, and with Solution A refactored in a
separate case/function clause.
With Solution B, however, every time we construct a node we should
have a source location ready to put into it.
That is, with Solution B, (a) and (b) are not cleanly implemented.

(I can explain more if not clear.)

/Shayan

On Tue, 12 Feb 2019 at 15:30, Shayan Najd  wrote:
>
> > My problem, though, is that this is just a convention -- no one checks it. 
> > It would be easy to forget.
>
> I am not sure if I understand: shouldn't the totality checker warn if
> there is no pattern for the wrapper constructor (hence enforce the
> convention)?
>
>
> On Tue, 12 Feb 2019 at 15:19, Richard Eisenberg  wrote:
> >
> >
> >
> > > On Feb 12, 2019, at 5:19 AM, Shayan Najd  wrote:
> > >
> > > About the new code, the convention is straightforward: anytime you
> > > destruct an AST node, assume a wrapper node inside (add an extra
> > > case), or use the smart constructors/pattern synonyms.
> >
> > Aha! This, I did not know. So, you're saying that all the consumers of the 
> > GHC AST need to remember to use dL every time they pattern-match. With the 
> > new design, using dL when it's unnecessary doesn't hurt, but forgetting it 
> > is problematic. So: just use it every time. My problem, though, is that 
> > this is just a convention -- no one checks it. It would be easy to forget.
> >
> > > On Feb 12, 2019, at 6:00 AM, Simon Peyton Jones via ghc-devs 
> > >  wrote:
> > >
> > > One way to think of it is this: we can now put SrcSpans where they make 
> > > sense, rather than everywhere.
> >
> > This has some logic to it, but I'm not quite sold. Another way of saying 
> > this is that the new design favors flexibility for the producer, at the 
> > cost of requiring consumers to be aware of and consistently apply the 
> > convention Shayan describes above. The problem is, though, that if the 
> > producer is stingy in adding source locations, the consumer won't know 
> > which locations are expected to be informative. Is the consumer expected to 
> > collect locations from a variety of places and try to combine them somehow? 
> > I doubt it. So this means that the flexibility for the producer isn't 
> > really there -- the type system will accept arbitrary choices of where to 
> > put locations, but consumers won't get the locations where they expect them.
> >
> > >   We can still say (Located t) in places where we want to guarantee a 
> > > SrcSpan.
> >
> > This seems to go against the TTG philosophy. We can do this in, say, the 
> > return type of a function, but we can't in the AST proper, because that's 
> > shared among a number of clients, some of whom don't want the source 
> > locations.
> >
> > >
> > > Yes, this lets us add more than one; that's redundant but not harmful.
> >
> > I disagree here. If we add locations to a node twice, then we'll have to 
> > use dL twice to find the underlying constructor. This is another case there 
> > the type system offers the producer flexibility but hamstrings the consumer.
> >
> >
> > > On Feb 12, 2019, at 7:32 AM, Vladislav Zavialov  
> > > wrote:

Re: TTG: Handling Source Locations

[Shayan Najd]

> My problem, though, is that this is just a convention -- no one checks it. It 
> would be easy to forget.

I am not sure if I understand: shouldn't the totality checker warn if
there is no pattern for the wrapper constructor (hence enforce the
convention)?


On Tue, 12 Feb 2019 at 15:19, Richard Eisenberg  wrote:
>
>
>
> > On Feb 12, 2019, at 5:19 AM, Shayan Najd  wrote:
> >
> > About the new code, the convention is straightforward: anytime you
> > destruct an AST node, assume a wrapper node inside (add an extra
> > case), or use the smart constructors/pattern synonyms.
>
> Aha! This, I did not know. So, you're saying that all the consumers of the 
> GHC AST need to remember to use dL every time they pattern-match. With the 
> new design, using dL when it's unnecessary doesn't hurt, but forgetting it is 
> problematic. So: just use it every time. My problem, though, is that this is 
> just a convention -- no one checks it. It would be easy to forget.
>
> > On Feb 12, 2019, at 6:00 AM, Simon Peyton Jones via ghc-devs 
> >  wrote:
> >
> > One way to think of it is this: we can now put SrcSpans where they make 
> > sense, rather than everywhere.
>
> This has some logic to it, but I'm not quite sold. Another way of saying this 
> is that the new design favors flexibility for the producer, at the cost of 
> requiring consumers to be aware of and consistently apply the convention 
> Shayan describes above. The problem is, though, that if the producer is 
> stingy in adding source locations, the consumer won't know which locations 
> are expected to be informative. Is the consumer expected to collect locations 
> from a variety of places and try to combine them somehow? I doubt it. So this 
> means that the flexibility for the producer isn't really there -- the type 
> system will accept arbitrary choices of where to put locations, but consumers 
> won't get the locations where they expect them.
>
> >   We can still say (Located t) in places where we want to guarantee a 
> > SrcSpan.
>
> This seems to go against the TTG philosophy. We can do this in, say, the 
> return type of a function, but we can't in the AST proper, because that's 
> shared among a number of clients, some of whom don't want the source 
> locations.
>
> >
> > Yes, this lets us add more than one; that's redundant but not harmful.
>
> I disagree here. If we add locations to a node twice, then we'll have to use 
> dL twice to find the underlying constructor. This is another case there the 
> type system offers the producer flexibility but hamstrings the consumer.
>
>
> > On Feb 12, 2019, at 7:32 AM, Vladislav Zavialov  
> > wrote:
> >
> > I claim an SrcSpan makes sense everywhere, so this is not a useful
> > distinction. Think about it as code provenance, an AST node always
> > comes from somewhere
>
> I agree with this observation. Perhaps SrcSpan is a bad name, and 
> SrcProvenance is better. We could even imagine using the new HasCallStack 
> feature to track where generated code comes from (perhaps only in DEBUG 
> compilers). Do we need to do this today? I'm not sure there's a crying need. 
> But philosophically, we are able to attach a provenance to every slice of 
> AST, so there's really no reason for uninformative locations.
>
> > My concrete proposal: let's just put SrcSpan in the extension fields
> > of each node
>
> I support this counter-proposal. Perhaps if it required writing loads of 
> extra type instances, I wouldn't be as much in favor. But we already have to 
> write those instances -- they just change from NoExt to SrcSpan. This seems 
> to solve all the problems nicely, at relatively low cost. And, I'm sure it's 
> more efficient at runtime than either the previous ping-pong style or the 
> current scenario, as we can pattern-match on constructors directly, requiring 
> one less pointer-chase or function call.
>
> One downside of this proposal is that it means that more care will have to be 
> taken when setting the extension field of AST nodes after a pass, making sure 
> to preserve the location. (This isn't really all that different from 
> location-shuffling today.) A quick mock-up shows that record-updates can make 
> this easier:
>
> > data Phase = Parsed | Renamed
> >
> > data Exp p = Node (XNode p) Int
> >
> > type family XNode (p :: Phase)
> > type instance XNode p = NodeExt p
> >
> > data NodeExt p where
> >   NodeExt :: { flag :: Bool, fvs :: RenamedOnly p String } -> NodeExt p
> >
> > type family RenamedOnly p t where
> >   RenamedOnly Parsed _ = ()
> >   RenamedOnly Renamed t = t
> >
> > example 

Re: TTG: Handling Source Locations

[Shayan Najd]

Hi Richard,

> [Richard:]
> It seems to me, though, that this move makes us *less typed*.
> [and]
> However, worse, we might forget to *add* a location when downstream functions 
> expect one.

We had a more sophisticated version of TTG that could support the
ping-pong style (and hence typed tagging of locations), but it came at
the price of more complicated encoding [0].
We have decided to abandon the more typed variant since tracking
whether a node is located or not is inherently a dynamic/run-time
process, not a static/compile-time process:
there are some nodes that are generated in the process by the compiler
by an arbitrary logic (hard to encode by types), hence have no
location (in the source code).

The types `LHsExpr`, `LPat`, and the like will be deleted! It will be
all `HsExpr`, `Pat` and the like.
Baking-in, e.g. `LHsExpr` into `HsExpr`, was a mistake in the first
place: we were cheating using `Maybe` type anyway, when for example an
`LHsExpr` was forcibly required but we had only `HsExpr` and used
`noLoc`.

> Sorry for reopening something that has already been debated, but (unless I'm 
> missing something) the current state of affairs seems like a potential 
> wellspring of subtle bugs.

We were really careful about the refactoring. The new code aside, I
don't see how we can introduce any bugs by the refactoring of the old
code explained in the wiki.
About the new code, the convention is straightforward: anytime you
destruct an AST node, assume a wrapper node inside (add an extra
case), or use the smart constructors/pattern synonyms.

I'd be happy to rediscuss the design space here. It would be great to
have everyone fully on board as it is not a trivial change.

/Shayan

[0] https://github.com/shayan-najd/HsAST/blob/master/Paper.pdf

On Sat, 9 Feb 2019 at 17:19, Richard Eisenberg  wrote:
>
> Hi devs,
>
> I just came across [TTG: Handling Source Locations], as I was poking around 
> in RdrHsSyn and found wondrous things like (dL->L wiz waz) all over the place.
>
> General outline: 
> https://ghc.haskell.org/trac/ghc/wiki/ImplementingTreesThatGrow/HandlingSourceLocations
> Phab diff: https://phabricator.haskell.org/D5036
> Trac ticket: https://ghc.haskell.org/trac/ghc/ticket/15495
> Commit: 
> https://gitlab.haskell.org/ghc/ghc/commit/509d5be69c7507ba5d0a5f39ffd1613a59e73eea
>
> I see why this change is wanted and how the new version works.
>
> It seems to me, though, that this move makes us *less typed*. That is, it 
> would be very easy (and disastrous) to forget to match on a location node. 
> For example, I can now do this:
>
> > foo :: LPat p -> ...
> > foo (VarPat ...) = ...
>
> Note that I have declared that foo takes a located pat, but then I forgot to 
> extract the location with dL. This would type-check, but it would fail. 
> Previously, the type checker would ensure that I didn't forget to match on 
> the L constructor. This error would get caught after some poking about, 
> because foo just wouldn't work.
>
> However, worse, we might forget to *add* a location when downstream functions 
> expect one. This would be harder to detect, for two reasons:
> 1. The problem is caught at deconstruction, and figuring out where an object 
> was constructed can be quite hard.
> 2. The problem might silently cause trouble, because dL won't actually fail 
> on a node missing a location -- it just gives noSrcSpan. So the problem would 
> manifest as a subtle degradation in the quality of an error message, perhaps 
> not caught until several patches (or years!) later.
>
> So I'm uncomfortable with this direction of travel.
>
> Has this aspect of this design been brought up before? I have to say I don't 
> have a great solution to suggest. Perhaps the best I can think of is to make 
> Located a type family. It would branch on the type index to HsSyn types, 
> introducing a Located node for GhcPass but not for other types. This Isn't 
> really all that extensible (I think) and it gives special status to GHC's 
> usage of the AST. But it seems to solve the immediate problems without the 
> downside above.
>
> Sorry for reopening something that has already been debated, but (unless I'm 
> missing something) the current state of affairs seems like a potential 
> wellspring of subtle bugs.
>
> Thanks,
> Richard
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Suppressing False Incomplete Pattern Matching Warnings for Polymorphic Pattern Synonyms

[Shayan Najd]

I also realised the COMPLETE pragmas are even less helpful (at least
for my case mentioned above) as they cannot be used when orphaned.
For example, I tried to follow the partial solution of introducing
multiple COMPLETE pragmas, one per a concrete instance of
`HasSrcSpan`. I defined the pattern synonym `LL` in
`basicTypes/SrcLoc.hs` and wanted to naturally define a COMPLETE
pragma for when `LL` is used for `Pat` in `hsSyn/HsPat.hs`. I got an
error complaining about orphaned COMPLETE pragmas. The only
unacceptable way (that I can see) that makes it work is to have a
module defining `LL`, importing all the instances of `HasSrcSpan`, and
all the COMPLETE pragmas (one per instance).

I have made a ticket about the original concern here:
  https://ghc.haskell.org/trac/ghc/ticket/15885

Until we come up with a good and general solution to this ticket, do
you know of a hack to suppress the false incomplete pattern matching
warnings in GHC? Should we add clauses like `foo _ = panic
"Impossible! How did you manage to bypass a complete matching?"`?

/Shayan

On Mon, 29 Oct 2018 at 14:52, Sylvain Henry  wrote:
>
> I've just found this related ticket: 
> https://ghc.haskell.org/trac/ghc/ticket/14422
>
>
> On 10/26/18 7:04 PM, Richard Eisenberg wrote:
>
> Aha. So you're viewing complete sets as a type-directed property, where we 
> can take a type and look up what complete sets of patterns of that type might 
> be.
>
> Then, when checking a pattern-match for completeness, we use the inferred 
> type of the pattern, access its complete sets, and if these match up. 
> (Perhaps an implementation may optimize this process.)
>
> What I like about this approach is that it works well with GADTs, where, 
> e.g., VNil is a complete set for type Vec a Zero but not for Vec a n.
>
> I take back my claim of "No types!" then, as this does sound like it has the 
> right properties.
>
> For now, I don't want to get bogged down by syntax -- let's figure out how 
> the idea should work first, and then we can worry about syntax.
>
> Here's a stab at a formalization of this idea, written in metatheory, not 
> Haskell:
>
> Let C : Type -> Set of set of patterns. C will be the lookup function for 
> complete sets. Suppose we have a pattern match, at type tau, matching against 
> patterns Ps. Let S = C(tau). S is then a set of sets of patterns. The 
> question is this: Is there a set s in S such that Ps is a superset of s? If 
> yes, then the match is complete.
>
> What do we think of this design? Of course, the challenge is in building C, 
> but we'll tackle that next.
>
> Richard
>
> On Oct 26, 2018, at 5:20 AM, Sylvain Henry  wrote:
>
> Sorry I wasn't clear. I'm not an expert on the topic but it seems to me that 
> there are two orthogonal concerns:
>
> 1) How does the checker retrieve COMPLETE sets.
>
> Currently it seems to "attach" them to data type constructors (e.g. Maybe). 
> If instead it retrieved them by matching types (e.g. "Maybe a", "a") we could 
> write:
>
> {-# COMPLETE LL #-}
>
> From an implementation point of view, it seems to me that finding complete 
> sets would become similar to finding (flexible, overlapping) class instances. 
> Pseudo-code:
>
> class Complete a where
>   conlikes :: [ConLike]
> instance Complete (Maybe a) where
>   conlikes = [Nothing @a, Just @a]
> instance Complete (Maybe a) where
>   conlikes = [N @a, J @a]
> instance Complete a where
>   conlikes = [LL @a]
> ...
>
>
> 2) COMPLETE set depending on the matched type.
>
> It is a thread hijack from me but while we are thinking about refactoring 
> COMPLETE pragmas to support polymorphism, maybe we could support this too. 
> The idea is to build a different set of conlikes depending on the matched 
> type. Pseudo-code:
>
> instance Complete (Variant cs) where
>   conlikes = [V @c | c <- cs] -- cs is a type list
>
> (I don't really care about the pragma syntax)
>
> Sorry for the thread hijack!
> Regards,
> Sylvain
>
>
> On 10/26/18 5:59 AM, Richard Eisenberg wrote:
>
> I'm afraid I don't understand what your new syntax means. And, while I know 
> it doesn't work today, what's wrong (in theory) with
>
> {-# COMPLETE LL #-}
>
> No types! (That's a rare thing for me to extol...)
>
> I feel I must be missing something here.
>
> Thanks,
> Richard
>
> On Oct 25, 2018, at 12:24 PM, Sylvain Henry  wrote:
>
> > In the case where all the patterns are polymorphic, a user must
> > provide a type signature but we accept the definition regardless of
> > the type signature they provide.
>
> Currently we can specify the type *constructor* in a COMPLETE pragma:
>
> pattern J :: a -> Maybe a
> pattern J a = Just a
>
> pattern N :: Maybe a
> pattern N = Nothing
>
> {-# COMPLETE N, J :: Maybe #-}
>
>
> Instead if we could specify the type with its free vars, we could refer to 
> them in conlike signatures:
>
> {-# COMPLETE N, [ J :: a -> Maybe a ] :: Maybe a #-}
>
> The COMPLETE pragma for LL could be:
>
> {-# COMPLETE [ LL :: HasSrcSpan a => SrcSpan -> SrcSpanLess a 

Re: Suppressing False Incomplete Pattern Matching Warnings for Polymorphic Pattern Synonyms

[Shayan Najd]

> [Ryan Scott:]
> In other words, this would work provided that you'd be willing to list
> every single instance of `HasSrcSpan` in its own COMPLETE set. It's
> tedious, but possible.

Yes, for the cases where `LL` is used monomorphically, we can use the
per-instance COMPLETE pragma, but it does not scale to the generic
cases where `LL` is used polymorphically.
Consider the following where `Exp` and `Pat` are both instances of `HasSrcSpan`:
{{{
{-# COMPLETE LL :: Exp #-}
{-# COMPLETE LL :: Pat #-}

unLocExp :: Exp -> Exp
unLocExp (LL _ m) = m

unLocPat :: Pat -> Pat
unLocPat (LL _ m) = m

unLocGen :: HasLoc a => a -> SrcSpanLess a
unLocGen (LL _ m) = m
}}}

In the functions `unLocExp` and `unLocPat`, the related COMPLETE
pragmas rightly avoid the false incomplete pattern matching warnings.

However, to avoid the false incomplete pattern matching warning in
`unLocGen`, either I should add a catch-all case
  `unLocGen _ = error "Impossible!"`
or expose the internal view patterns and hence break the abstraction
  `unLocGen (decomposeSrcSpan->(_ , m)) = m`

We want to avoid both solutions and unfortunately, this problem arises
frequently enough.
For example, in GHC, there are plenty of such generic cases (e.g.,
`sL1` or the like in the parser).

I believe the source of the issue is what you just mentioned:

> [Ryan Scott:]
> Complete sets of patterns must be identified relative to a type.

Technically `HasSrcSpan a0  |- a0` is a type, for a Skolem variable
`a0`; I understand if you mean relative to a concrete type, but I
don't understand why (I have no experience with GHC's totality checker
code).
Why can't it be syntactic? We should allow programmers to express
things like "when you see `LL` treat the related pattern matching
group as total".

/Shayan
On Thu, Oct 25, 2018 at 4:40 PM Ryan Scott  wrote:
>
> You *can* put `LL` into a COMPLETE set, but under the stipulation that
> you specify which type constructor it's monomorphized to. To quote the
> wiki page on COMPLETE sets:
>
> In the case where all the patterns are polymorphic, a user must
> provide a type signature but we accept the definition regardless of
> the type signature they provide. The type constructor for the whole
> set of patterns is the type constructor as specified by the user. If
> the user does not provide a type signature then the definition is
> rejected as ambiguous.
>
> This design is a consequence of the design of the pattern match
> checker. Complete sets of patterns must be identified relative to a
> type. This is a sanity check as users would never be able to match on
> all constructors if the set of patterns is inconsistent in this
> manner.
>
> In other words, this would work provided that you'd be willing to list
> every single instance of `HasSrcSpan` in its own COMPLETE set. It's
> tedious, but possible.
>
> Ryan S.
> -
> [1] https://ghc.haskell.org/trac/ghc/wiki/PatternSynonyms/CompleteSigs#Typing
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Suppressing False Incomplete Pattern Matching Warnings for Polymorphic Pattern Synonyms

[Shayan Najd]

Dear GHC hackers,

On our work on the new front-end AST for GHC [0] based on TTG [1], we
would like to use a pattern synonym like the following [2]:

{{{
pattern LL :: HasSrcSpan a => SrcSpan -> SrcSpanLess a -> a
pattern LL s m <- (decomposeSrcSpan -> (m , s))
  where
LL s m =  composeSrcSpan (m , s)
}}}

We know that any match on `LL` patterns, makes the pattern matching
total, as it uses a view pattern with a total output pattern (i.e., in
`decomposeSrcSpan -> (m , s)`, the pattern `(m , s)` is total).

As far as I understand, currently COMPLETE pragmas cannot be used with
such a polymorphic pattern.

What do you suggest us to do to avoid the false incomplete pattern
matching warnings?

Thanks,
  Shayan

[0] https://ghc.haskell.org/trac/ghc/wiki/ImplementingTreesThatGrow
[1] 
https://ghc.haskell.org/trac/ghc/wiki/ImplementingTreesThatGrow/TreesThatGrowGuidance
[2] 
https://ghc.haskell.org/trac/ghc/wiki/ImplementingTreesThatGrow/HandlingSourceLocations
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Availability of Coercible Instances at TH Runtime

[Shayan Najd]

> I think this may be waiting for the trees-that-grow work to be completed, and 
> as far as I know, no one is actively working on this.

We've just got a new Google SoC project accepted to push this forward :)

/Shayan

On Mon, May 7, 2018 at 3:14 PM, Richard Eisenberg  wrote:
> Yes: https://ghc.haskell.org/trac/ghc/wiki/TemplateHaskell/Introspective
>
> I think this may be waiting for the trees-that-grow work to be completed, and 
> as far as I know, no one is actively working on this. But I still think it 
> would be a Good Thing.
>
> Richard
>
>> On May 7, 2018, at 12:37 AM, David Kraeutmann  wrote:
>>
>> Tangentially related, but wasn't there a plan at some point to integrate TH 
>> more tightly with GHC?
>>
>> On 5/7/2018 4:39 AM, Richard Eisenberg wrote:
>>>
>>> No, it doesn't, but the Q monad is actually a wrapper around TcM, the 
>>> type-checker monad. I don't think it would be all that hard to do this. 
>>> Could be a suitable project for someone new to GHC hacking...
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: True multi stage Haskell

[Shayan Najd]

Hi Tim,

About using GHC as a library module, have you see the ongoing work on
"native metaprogramming" in GHC at
  https://ghc.haskell.org/trac/ghc/wiki/NativeMetaprogramming

and a child project at
   https://ghc.haskell.org/trac/ghc/wiki/ImplementingTreesThatGrow

There, online code generation is not of the highest priority, rather we are
trying to open up the compiler from top to down:
making available GHC's AST and then the parser and the typechecker
have priority over the other functionalities down the pipeline (like the
code generator).

> If any one wanted to help with this, that would be great.

I am interested to hear more about the approach you have in mind, and would
be happy to help.

Yours,
  Shayan

On Fri, Nov 17, 2017 at 6:05 PM, Tim Sheard  wrote:

> After many years of hoping someone else would do this, I would like to
> make GHC into a true multi-stage programming language. Here is how I
> thought I might approach this.
>
> 1) Use the GHC as a library module.
> 2) Use the LLVM backend.
>
> I have no experience with either of these tools.
> Lets start simple, How would I write functions like
>
> compile :: String -> IO PtrToLLVMCode  -- where the string is a small
> Haskell program.
> llvmCompile :: PtrToLLVMCode -> IO PtrToMachineCode
> jumpTo:: PtrToMachineCode -> IO ans   -- where ans is the "type" of the
> string.
>
>
> Any thoughts on how to get started? What papers to read, examples to look
> at?
>
> I'd love to move to some more disciplined input type, a sort of (mono)
> typed program
> representation (with similar complexity) to Template Haskell Exp type.
>
> where (Exp t) is a data structure representing a Haskell program of type t.
>
> All offers of advice accepted. I'm trying to get started soon, and good
> advice
> about what to avoid is especially welcome.  If any one wanted to help with
> this,
> that would be great.
>
> Tim Sheard
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Trees that Grow and constraints

[Shayan Najd]

Isn't the solution always
  if generic programming makes things complicated, avoid it!

Here generic programming is where you define instances parametric on the
phase index.
Why not defining three instances of the type class, one per each phase?
Yes, we get code duplication (which in this case is still not much as we
use automatic deriving), but the compilation gets faster (which is the
motivation), like [2].

[2] http://lpaste.net/360019

/Shayan



On Mon, Nov 13, 2017 at 2:05 PM, Alan & Kim Zimmerman 
wrote:

> Just to clarify, this example is a simplification, in practice we would be
> applying different type for each type instance
>
> e.g.
>
> type instance XEOverLit (GhcPass 'Parsed ) = PlaceHolder
> type instance XEOverLit (GhcPass 'Renamed) = PlaceHolder
> type instance XEOverLit (GhcPass 'Typechecked) = Type
>
> (modelling existing PostTc)
>
> Alan
>
> On 13 November 2017 at 11:23, Alan & Kim Zimmerman 
> wrote:
>
>> At the moment the Trees that Grow implementation in GHC master makes use
>> of massive constraint types to provide Data instances for the hsSyn AST.
>>
>> I am trying to remove the need for this, and have hit a problem.
>>
>> The example I have reduced it to is here [1]
>>
>> The essence of the problem is
>>
>> ---
>> data Experiment p
>>   = Experiment {
>>   e_ext :: (XEOverLit p),
>>   e_val :: Int }
>>   | XExperiment (XXOverLit p)
>> deriving instance (Data (GhcPass p)) => Data (Experiment (GhcPass p))
>>
>> type family XEOverLit  x
>> type family XXOverLit x
>>
>> -- The following line works
>> -- type instance XEOverLit (GhcPass _) = PlaceHolder
>>
>> -- The following 3 lines do noe
>> type instance XEOverLit (GhcPass 'Parsed ) = PlaceHolder
>> type instance XEOverLit (GhcPass 'Renamed) = PlaceHolder
>> type instance XEOverLit (GhcPass 'Typechecked) = PlaceHolder
>>
>> type instance XXOverLit (GhcPass _) = PlaceHolder
>> ---
>>
>> Where specifying the type instance with a wild card keeps GHC happy (the
>> XXOverLit case), but specifying for each of the three constructors for pass
>> does not (the XEOverLit case)
>>
>> The exact error message is
>>
>> --
>> Experiment.hs:34:1: error:
>> • Could not deduce (Data (XEOverLit (GhcPass p)))
>> arising from a use of ‘k’
>>   from the context: Data (GhcPass p)
>> bound by the instance declaration at Experiment.hs:34:1-69
>> • In the first argument of ‘k’, namely ‘(z Experiment `k` a1)’
>>   In the expression: ((z Experiment `k` a1) `k` a2)
>>   In an equation for ‘gfoldl’:
>>   gfoldl k z Experiment a1 a2 = ((z Experiment `k` a1) `k` a2)
>>   When typechecking the code for ‘gfoldl’
>> in a derived instance for ‘Data (Experiment (GhcPass p))’:
>> To see the code I am typechecking, use -ddump-deriv
>>|
>> 34 | deriving instance (Data (GhcPass p)) => Data (Experiment (GhcPass p))
>>| 
>> ^
>> --
>>
>> Alan
>>
>> [1] http://lpaste.net/360017
>>
>>
>>
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Discussion: Static Safety via Distinct Interfaces for HsSyn ASTs

[Shayan Najd]

>
> (1)-(3) appears to be three different approaches, but I don’t think that’s
> what you intend.  I think there are only two: add the indirection layer or
> not?


(1)-(3) are just steps when we do choose to add the indirection layer: add
the layer, and do the changes when desired.
If we choose to not to add the indirection layer, nothing needs to be
changed and the internals of the encoding (`PostTc`, place holders, etc)
remain visible in the code.

Can you give an example function or two, and what it would look like under
> the different approaches.


For example, the function clause

> rnExpr (HsMultiIf _ty alts)
>  = do { (alts', fvs) <- mapFvRn (rnGRHS IfAlt rnLExpr) alts
>   ; return (HsMultiIf placeHolderType alts', fvs) }

becomes

> rnExpr (PsMultiIf alts)
>  = do { (alts', fvs) <- mapFvRn (rnGRHS IfAlt rnLExpr) alts
>   ; return (RnMultiIf alts', fvs) }

I hope it clarifies what I mean a bit.

There is always a choice between how distinct we want the phases to be.
The more distinct they are, the higher static guarantees. The code also
gets more clear in a way, e.g. `RnMultiIf` is talking about a renamed
expression, `PsMultiIf` about a parsed expression, while `HsMultiIf` is
talking about an expression of any phase.
At the same, distinctness means more work for the programmer.
Also, such distinction sometimes implies a pedagogic burdon, as readers
should now learn about more than one AST. However, this burden is very low
here thanks to the prefixing convention: `PsMultiIf` and `RnMultiIf` are
easily understood to represent the same thing in different phases.
Finally, such distinctions often lead to code duplication. But in our case,
Trees that Grow machinery saves us from such duplication, e.g., we have the
same base ASTs and we can write generic programmers over the bases ASTs
anytime we want (point/step (3) above).

Thanks,
  Shayan

On Thu, Aug 24, 2017 at 3:35 PM, Simon Peyton Jones <simo...@microsoft.com>
wrote:

> I’m keen NOT to introduce these layers of indirection.  I think they make
> the code harder to understand.
>
>
> Can you give an example function or two, and what it would look like under
> the different approaches.
>
>
>
> (1)-(3) appears to be three different approaches, but I don’t think that’s
> what you intend.  I think there are only two: add the indirection layer or
> not?
>
>
>
> S
>
>
>
> *From:* Shayan Najd [mailto:sh.n...@gmail.com]
> *Sent:* 23 August 2017 13:26
> *To:* ghc-devs@haskell.org
> *Cc:* Simon Peyton Jones <simo...@microsoft.com>; Alan & Kim Zimmerman <
> alan.z...@gmail.com>
> *Subject:* Discussion: Static Safety via Distinct Interfaces for HsSyn
> ASTs
>
>
>
> In this thread, I am going to raise a topic for discussion. Please share
> your opinions and related experiences.
>
>
>
> Evaluation of type families within HsSyn ASTs, such as `PostTc`, with a
> fixed phase index, such as `GhcPs`, gives us distinct ASTs at the
> *compile-time*.
>
> However, when programming with these ASTs, we use patterns, such as `HsMultiIf
> :: PostTc p Type -> [LGRHS p (LHsExpr p)] -> HsExpr p` that are shared
> among phases.
>
> We can
>
> (1) introduce a layer of abstraction providing a set of type and pattern
> synonyms specific to each phase, such as `PsMultiIf :: [LPsGRHS  LPsExpr]
> -> PsExpr`; and
>
> (2) updating code working on ASTs of specific phase to use the interface
> specific to the phase, such as by changing prefixes from `Hs` to `Ps` and
> by removing unused variables and placeholders; and
>
> (3) leaving untouched code working uniformly on ASTs of different phases
> (i.e., the generic functions in Trees that Grow terminology), such as the
> existing functions whose types are polymorphic on phase index.
>
>
>
> Some comments:
>
>
>
> - It can be done gradually and smoothly: we add three separate files in
> HsSyn (per each phase) containing the phase-specific interfaces, and
> gradually import them and do the changes per module.
>
> - Using the interfaces is optional: code using the current method (e.g.,
> using `HsMultiIf`) should work just fine.
>
> - It introduces a layer of indirection and three more files to maintain.
>
> - It makes code working on HsSyn ASTs, such as the renamer, appear cleaner
> as placeholders and similar machinery are abstracted away by the interfaces
> (e.g., no need to import bits and pieces of `HsExtension`)
>
> - In theory, there should be zero impact on GHC's runtime performance.
>
>
>
> I am myself undecided about its benefit-cost ratio, but willing to at
> least implement the phase-specific interfaces.
>
> For me, abstracting away all the `PostRn` stuff, `Out` prefixed
> constructors, and dummy placeholders from the front-end code is the most
> valuable.
>
>
>
> Yours,
>
>   Shayan
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Discussion: Static Safety via Distinct Interfaces for HsSyn ASTs

[Shayan Najd]

In this thread, I am going to raise a topic for discussion. Please share
your opinions and related experiences.

Evaluation of type families within HsSyn ASTs, such as `PostTc`, with a
fixed phase index, such as `GhcPs`, gives us distinct ASTs at the
*compile-time*.
However, when programming with these ASTs, we use patterns, such as `HsMultiIf
:: PostTc p Type -> [LGRHS p (LHsExpr p)] -> HsExpr p` that are shared
among phases.
We can
(1) introduce a layer of abstraction providing a set of type and pattern
synonyms specific to each phase, such as `PsMultiIf :: [LPsGRHS  LPsExpr]
-> PsExpr`; and
(2) updating code working on ASTs of specific phase to use the interface
specific to the phase, such as by changing prefixes from `Hs` to `Ps` and
by removing unused variables and placeholders; and
(3) leaving untouched code working uniformly on ASTs of different phases
(i.e., the generic functions in Trees that Grow terminology), such as the
existing functions whose types are polymorphic on phase index.

Some comments:

- It can be done gradually and smoothly: we add three separate files in
HsSyn (per each phase) containing the phase-specific interfaces, and
gradually import them and do the changes per module.
- Using the interfaces is optional: code using the current method (e.g.,
using `HsMultiIf`) should work just fine.
- It introduces a layer of indirection and three more files to maintain.
- It makes code working on HsSyn ASTs, such as the renamer, appear cleaner
as placeholders and similar machinery are abstracted away by the interfaces
(e.g., no need to import bits and pieces of `HsExtension`)
- In theory, there should be zero impact on GHC's runtime performance.

I am myself undecided about its benefit-cost ratio, but willing to at least
implement the phase-specific interfaces.
For me, abstracting away all the `PostRn` stuff, `Out` prefixed
constructors, and dummy placeholders from the front-end code is the most
valuable.

Yours,
  Shayan
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Failing Test Cases in HEAD

[Shayan Najd]

In a freshly cloned GHC code base (c6462ab0...), in my system,`make test
TEST="T13780c T13822"` fails.
That is while `./validate` passes.

I am rather new to GHC code base, and not so familiar with its build and
testing system.
So I was wondering whether this is an intended behaviour (e.g., some test
cases are intentionally left out of `./validate`).

Specifically, I follow the following steps:
(1) `git clone --recursive git://git.haskell.org/ghc.git`
(2) `cd ghc`
(3) `cp mk/build.mk.sample mk/build.mk`
(4) set `BuildFlavour = devel2` and then
(5) ./boot
(6) ./configure
(7) make -j4
(8) make test TEST="T13780c T13822"

I build in a variant of the docker image for GHC development on Linux (64
bit), if it matters.

As it seems, Alan also gets the similar result on his system.

I am investigating this, since my development of Growable ASTs is stalled
due to some "GHC panic bug" when validating.
(In my repo[1], I can successfully build with steps (2)-(7) above but
`./validate` (and `make test TEST="T13780c T13822"`) fails)

Thanks,
  Shayan

[1] https://github.com/shayan-najd/GrowableGHC
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Restructuring hsSyn

[Shayan Najd]

Currently AST declarations, their relate utilities, and `Outputable`
instances are defined in the same files.
Does anyone object to moving `Outputable` instances to separate files?
The purpose is to gradually identify reusable functionalities, group them
together, polish them (e.g., remove some unnecessary dependencies), and
expose them to the end-users.
At this stage, I don't expect any changes outside hsSyn.

/Shayan
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Operating on HsSyn

[Shayan Najd]

Simon,

Back to your original questions:

1. I think we should probably use a superclass instead of a type synonym
>
> class (Data p, ForallX Data p, ) => DataId p where {}
>
> Why?  Only one argument to pass, and to pass on to successive calls.  I
> see no downside.


Sounds good to me. I have not experimented with it yet, but I see no
downside as well.

Shall we treat Outputable like Data?  (I.e. make an abbreviation for a long
> list of Outputable instances and use it everywhere)


Besides, the above point about semantics and reusing it as classical
pretty-printer, I suggest a bit more fine-grained abbreviations, if we are
going along this path:
 we define the abbreviations per datatype (as opposed to all), we define
`ForallXNAME :: constraint -> *` (as in Section 3.7 of the paper) where
`NAME` is name of the datatype.

However, we do NOT need Trees that Grow in its full generality: as far as I
understand, we do not have GADTs or existentials in GHC ASTs.
This means we can use the simpler design of Trees that Grow that all
extension families had the same arity and kind, then we could factor them
all into one type family dispatching on some unique identifier.
(we used type-level strings, or promoted datatypes for annotations.)

For example, the following in absence of existentials

> type family XVar x a
> type family XApp x a
> type family XAbs x a
> data Exp x a
>  = Var (XVar x a) a
>  | Abs (XAbs x a) a (Exp x a)
>  | App (XApp x a) (Exp x a) (Exp x a)

is as good as (one of our very first extensible encodings)

> type family XDispatch name x a
> data Lbl = VarL | AbsL | AppL
> data Exp x a
>  = Var (XDispatch VarL x a) a
>  | Abs (XDispatch AbsL x a) a (Exp x a)
>  | App (XDispatch AppL x a) (Exp x a) (Exp x a)

We could do the same with promoted datatypes as annotations (instead of
type-level strings).

The advantage of the simpler encoding is that now, in theory, we can define

> ForallXExp (c :: Constraint) x a = forall l. c (XDispatch l x a)

Which gives us for example `ForallXExp Outputable x a `.

Is this encoding faster, in comparison? Does it help?

/Shayan









On Fri, Jul 28, 2017 at 10:11 PM, Shayan Najd <sh.n...@gmail.com> wrote:

> MarLinn,
>Thanks for correcting me, and spelling this out.
>I did mean what Alan mentioned: "re-parsing a pretty printed parse tree
> gives you back a parse tree identical to the original (ignoring SrcSpans)".
>As I recall, we had to go a bit further to give 'Something' some more
> structure to take into account things like "(ignoring SrcSpans)" (e.g., to
> define exact-printers, etc).
>Provided I have failed twice to properly recall the invariant, I
> refrain from trying to recall the rest tonight :)
>
> Not diverging from my point above, as far as I understand, an ideal
> `Outputable` machinery is going to be a bit different from the traditional
> pretty printers.
> I believe with a proper design we can even reuse `Outputable` machinery
> and provide it as a pretty printer for Haskell terms.
> It resembles the scenario in Section 3.7 compared to Section 3.6 of Trees
> that Grow [1].
>
> Having said all these, we ARE diverging from the original thread, and
> Simon's questions.
>
> How about taking printer-design related discussion to the following wiki
> page (and/or a new ghc-dev thread if needed):
>   https://ghc.haskell.org/trac/ghc/wiki/HaskellSyntaxPrinters
>
> Cheers,
>   Shayan
>
> [1] http://www.jucs.org/jucs_23_1/trees_that_grow/jucs_23_
> 01_0042_0062_najd.pdf
>
> On Fri, Jul 28, 2017 at 8:43 PM, Alan & Kim Zimmerman <alan.z...@gmail.com
> > wrote:
>
>> I agree. 4 is the current GHC invariant.
>>
>> i.e., re-parsing a pretty printed parse tree gives you back a parse tree
>> identical to the original (ignoring SrcSpans)
>>
>> Alan
>>
>> On 28 July 2017 at 20:34, MarLinn <monkle...@gmail.com> wrote:
>>
>>> by
>>>
>>>  (parser . prettyPrint . parser) = id
>>>
>>> I meant
>>>
>>> (prettyPrint . parser . prettyPrint) = id
>>>
>>> for a valid input.
>>>
>>> Simplifying, (parser ∷ String → something), and (prettyPrint ∷
>>> something → String).
>>>
>>> Therefore, (parser . prettyPrint . parser ∷ String → something) and 
>>> (prettyPrint
>>> . parser . prettyPrint ∷ something → String).
>>>
>>> Therefore, both criteria could only apply for (something ~ String). But
>>> as pretty printing adds quotation marks, not even that is true.
>>>
>>> There are four formulations that might be applicable:
>>>
>>>1.
>>>
>>>parser . prettyPrint ≍ 

Re: Operating on HsSyn

[Shayan Najd]

MarLinn,
   Thanks for correcting me, and spelling this out.
   I did mean what Alan mentioned: "re-parsing a pretty printed parse tree
gives you back a parse tree identical to the original (ignoring SrcSpans)".
   As I recall, we had to go a bit further to give 'Something' some more
structure to take into account things like "(ignoring SrcSpans)" (e.g., to
define exact-printers, etc).
   Provided I have failed twice to properly recall the invariant, I refrain
from trying to recall the rest tonight :)

Not diverging from my point above, as far as I understand, an ideal
`Outputable` machinery is going to be a bit different from the traditional
pretty printers.
I believe with a proper design we can even reuse `Outputable` machinery and
provide it as a pretty printer for Haskell terms.
It resembles the scenario in Section 3.7 compared to Section 3.6 of Trees
that Grow [1].

Having said all these, we ARE diverging from the original thread, and
Simon's questions.

How about taking printer-design related discussion to the following wiki
page (and/or a new ghc-dev thread if needed):
  https://ghc.haskell.org/trac/ghc/wiki/HaskellSyntaxPrinters

Cheers,
  Shayan

[1]
http://www.jucs.org/jucs_23_1/trees_that_grow/jucs_23_01_0042_0062_najd.pdf

On Fri, Jul 28, 2017 at 8:43 PM, Alan & Kim Zimmerman 
wrote:

> I agree. 4 is the current GHC invariant.
>
> i.e., re-parsing a pretty printed parse tree gives you back a parse tree
> identical to the original (ignoring SrcSpans)
>
> Alan
>
> On 28 July 2017 at 20:34, MarLinn  wrote:
>
>> by
>>
>>  (parser . prettyPrint . parser) = id
>>
>> I meant
>>
>> (prettyPrint . parser . prettyPrint) = id
>>
>> for a valid input.
>>
>> Simplifying, (parser ∷ String → something), and (prettyPrint ∷ something
>> → String).
>>
>> Therefore, (parser . prettyPrint . parser ∷ String → something) and 
>> (prettyPrint
>> . parser . prettyPrint ∷ something → String).
>>
>> Therefore, both criteria could only apply for (something ~ String). But
>> as pretty printing adds quotation marks, not even that is true.
>>
>> There are four formulations that might be applicable:
>>
>>1.
>>
>>parser . prettyPrint ≍ id
>>2.
>>
>>prettyPrint . parser ≍ id -- ∷ String → String, useless here
>>3.
>>
>>prettyPrint . parser . prettyPrint ≍ prettyPrint
>>4.
>>
>>parser . prettyPrint . parser ≍ parser
>>5. Well, you could go beyond to (prettyPrint . parser . prettyPrint .
>>parser ≍ prettyPrint . parser) etc…
>>
>> I don't think 1 (or 2) follow from one of the last two. But 1 does imply
>> them. So it is a stronger criterion than both, and therefore probably not
>> the one to choose. Assuming the parser is internally consistent, 3 just
>> says something about the internal consistency of the pretty printer, while
>> 4 says something about the relationship of the pretty printer to the
>> parser. Thus 4 looks like the best candidate for a criterion. Possibly with
>> 3 as a secondary target.
>>
>> Cheers,
>> MarLinn
>>
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>>
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Operating on HsSyn

[Shayan Najd]

by

 (parser . prettyPrint . parser) = id


I meant

 (prettyPrint . parser . prettyPrint) = id

for a valid input.


On Fri, Jul 28, 2017 at 4:32 PM, Shayan Najd <sh.n...@gmail.com> wrote:

> On the contrary, inside GHC I /do/ want to print them. Otherwise how can I
>> see what the renamer has done?
>
>
> Right. So if I understand correctly, with this semantics, `Outputable` is
> somewhere between pretty printing as often used in program manipulation
> libraries (like Haskell-Src-Exts (HSE)) which is closer to syntax, and
> `Show` which is closer to Haskell representation.
> (There are also "exact printers" (as in HSE) that are even closer to
> syntax in some sense.)
> Often, pretty printers generate only grammatically valid terms, not the
> ones polluted with extra annotations (hence grammatically invalid), e.g.,
> what is the grammatically valid form of `OpApp` printed via `Outputable`
> that includes the fixity annotation.
>
> If I recall correctly, we have briefly studied these in the past summer,
> we came up with some roundtrip correctness criteria, like the following
> (bar error handling; assume valid input):
>
>  (parser . prettyPrint . parser) = id
>
> [paging in Jacques]
>
> The reason I am trying to flesh out the semantics is the /big/ gains on
> code reuse later on in the process: one does not need to define a separate
> pretty printing library for Haskell syntax, and can reuse the well-tested
> and well-maintained one in GHC.
>
> Reformulating part of your concern, based on my understanding (if I may),
> the questions is: what is the proper design of an "outputer"
> (debug-printer?) where /annotated/ terms can be pretty-printed including
> any printable (pretty?showable?) annotations.
> In particular, we may want to take advantage of extensibility of data
> types for this.
> Am I far off?
>
> Note: with proper design, an extensible debug-printer can still subsume
> corresponding pretty-printers.
>
>
> On Fri, Jul 28, 2017 at 2:36 PM, Simon Peyton Jones <simo...@microsoft.com
> > wrote:
>
>> I have been under the impression that we don't even want to print those.
>>
>> On the contrary, inside GHC I /do/ want to print them. Otherwise how can
>> I see what the renamer has done?
>>
>>
>>
>> Simon
>>
>>
>>
>> *From:* Shayan Najd [mailto:sh.n...@gmail.com]
>> *Sent:* 28 July 2017 12:20
>> *To:* Simon Peyton Jones <simo...@microsoft.com>
>> *Cc:* ghc-devs@haskell.org; Alan & Kim Zimmerman <alan.z...@gmail.com>
>> *Subject:* Re: Operating on HsSyn
>>
>>
>>
>> Before all this, we may need to discuss a bit about the intended
>> semantics of
>>
>> `Outputable`: does it need to print `PostRn`, or `PostTc` fields; or `Out`
>>
>> suffixed constructors?  If not, then we only need to write a set of
>> instances
>>
>> for the base growable AST, once and for all.  Such instances will be
>> polymorphic
>>
>> on the extension descriptor `p`, and do not need to mention the
>> constraints like
>>
>> `(PostRn p (IdP p)`, since these are just extensions and not part of the
>> base
>>
>> growable AST.  Or, am I missing something about the intended semantics of
>>
>> `Outputable`?
>>
>>
>>
>> You write
>>
>>
>>
>> So today we never print these annotations, to avoid bloating the instance
>> contexts, which can be painful.
>>
>>
>>
>> I have been under the impression that we don't even want to print those.
>>
>>
>>
>> Of course, there are scenarios (like `Show` instances) where we do want
>> to write
>>
>> compositional / generic functions that take into account the extensions.
>>
>> Here is my abstract overview of the scenario, that may help the
>> discussion.
>>
>> Consider data types `A`, `B`, and `C` (say, one AST datatype per compiler
>> phase) that
>>
>>  are defined as extensions to a base datatype `T`:
>>
>>
>>
>> > A = T XA
>>
>> > B = T XB
>>
>> > C = T XC
>>
>>
>>
>> where `X*`s are extension descriptors.
>>
>> Now, say we want to a define functions `f_A`, `f_B`, and `f_C` over `A`,
>> `B`, and `C`.
>>
>> We have two main alternatives:
>>
>> (a) either we write these  (manually or using the deriving mechanism)
>> separately
>>
>> (b) or we write a generic / parametric function `g` over `T`, and reuse
>> that to define `f_*`s
>>
>>
>>
>> Of course, (b) is p

Re: Operating on HsSyn

[Shayan Najd]

>
> On the contrary, inside GHC I /do/ want to print them. Otherwise how can I
> see what the renamer has done?


Right. So if I understand correctly, with this semantics, `Outputable` is
somewhere between pretty printing as often used in program manipulation
libraries (like Haskell-Src-Exts (HSE)) which is closer to syntax, and
`Show` which is closer to Haskell representation.
(There are also "exact printers" (as in HSE) that are even closer to syntax
in some sense.)
Often, pretty printers generate only grammatically valid terms, not the
ones polluted with extra annotations (hence grammatically invalid), e.g.,
what is the grammatically valid form of `OpApp` printed via `Outputable`
that includes the fixity annotation.

If I recall correctly, we have briefly studied these in the past summer, we
came up with some roundtrip correctness criteria, like the following (bar
error handling; assume valid input):

 (parser . prettyPrint . parser) = id

[paging in Jacques]

The reason I am trying to flesh out the semantics is the /big/ gains on
code reuse later on in the process: one does not need to define a separate
pretty printing library for Haskell syntax, and can reuse the well-tested
and well-maintained one in GHC.

Reformulating part of your concern, based on my understanding (if I may),
the questions is: what is the proper design of an "outputer"
(debug-printer?) where /annotated/ terms can be pretty-printed including
any printable (pretty?showable?) annotations.
In particular, we may want to take advantage of extensibility of data types
for this.
Am I far off?

Note: with proper design, an extensible debug-printer can still subsume
corresponding pretty-printers.


On Fri, Jul 28, 2017 at 2:36 PM, Simon Peyton Jones <simo...@microsoft.com>
wrote:

> I have been under the impression that we don't even want to print those.
>
> On the contrary, inside GHC I /do/ want to print them. Otherwise how can I
> see what the renamer has done?
>
>
>
> Simon
>
>
>
> *From:* Shayan Najd [mailto:sh.n...@gmail.com]
> *Sent:* 28 July 2017 12:20
> *To:* Simon Peyton Jones <simo...@microsoft.com>
> *Cc:* ghc-devs@haskell.org; Alan & Kim Zimmerman <alan.z...@gmail.com>
> *Subject:* Re: Operating on HsSyn
>
>
>
> Before all this, we may need to discuss a bit about the intended semantics
> of
>
> `Outputable`: does it need to print `PostRn`, or `PostTc` fields; or `Out`
>
> suffixed constructors?  If not, then we only need to write a set of
> instances
>
> for the base growable AST, once and for all.  Such instances will be
> polymorphic
>
> on the extension descriptor `p`, and do not need to mention the
> constraints like
>
> `(PostRn p (IdP p)`, since these are just extensions and not part of the
> base
>
> growable AST.  Or, am I missing something about the intended semantics of
>
> `Outputable`?
>
>
>
> You write
>
>
>
> So today we never print these annotations, to avoid bloating the instance
> contexts, which can be painful.
>
>
>
> I have been under the impression that we don't even want to print those.
>
>
>
> Of course, there are scenarios (like `Show` instances) where we do want to
> write
>
> compositional / generic functions that take into account the extensions.
>
> Here is my abstract overview of the scenario, that may help the discussion.
>
> Consider data types `A`, `B`, and `C` (say, one AST datatype per compiler
> phase) that
>
>  are defined as extensions to a base datatype `T`:
>
>
>
> > A = T XA
>
> > B = T XB
>
> > C = T XC
>
>
>
> where `X*`s are extension descriptors.
>
> Now, say we want to a define functions `f_A`, `f_B`, and `f_C` over `A`, `
> B`, and `C`.
>
> We have two main alternatives:
>
> (a) either we write these  (manually or using the deriving mechanism)
> separately
>
> (b) or we write a generic / parametric function `g` over `T`, and reuse
> that to define `f_*`s
>
>
>
> Of course, (b) is preferable in theory , but not always possible or
> preferable in practice.
>
> In which case, we can always resort to (a).
>
> The more varying are the definitions of `f_A`, `f_B`, and `f_C` the more
> parametric should
>
> `g` get, as this is the case for any generic function.
>
>
>
> With a correct design, I believe, these are all independent of Trees that
> Grow story itself:
>
> we are now not only trying to reuse data types, and functions agnostic
> towards extensions
>
> (pretty printers in my view of their semantics), but also reuse functions
> with parametric /
>
> varying behaviour with respect to extensions.
>
>
>
> /Shayan
>
>
>
>
>
>
>
> On Fri, Jul 28, 2017 

Re: Operating on HsSyn

[Shayan Najd]

Before all this, we may need to discuss a bit about the intended semantics
of
`Outputable`: does it need to print `PostRn`, or `PostTc` fields; or `Out`
suffixed constructors?  If not, then we only need to write a set of
instances
for the base growable AST, once and for all.  Such instances will be
polymorphic
on the extension descriptor `p`, and do not need to mention the constraints
like
`(PostRn p (IdP p)`, since these are just extensions and not part of the
base
growable AST.  Or, am I missing something about the intended semantics of
`Outputable`?

You write

So today we never print these annotations, to avoid bloating the instance
> contexts, which can be painful.
>

I have been under the impression that we don't even want to print those.

Of course, there are scenarios (like `Show` instances) where we do want to
write
compositional / generic functions that take into account the extensions.
Here is my abstract overview of the scenario, that may help the discussion.
Consider data types `A`, `B`, and `C` (say, one AST datatype per compiler
phase) that
 are defined as extensions to a base datatype `T`:

> A = T XA
> B = T XB
> C = T XC

where `X*`s are extension descriptors.
Now, say we want to a define functions `f_A`, `f_B`, and `f_C` over `A`, `B`,
and `C`.
We have two main alternatives:
(a) either we write these  (manually or using the deriving mechanism)
separately
(b) or we write a generic / parametric function `g` over `T`, and reuse
that to define `f_*`s

Of course, (b) is preferable in theory , but not always possible or
preferable in practice.
In which case, we can always resort to (a).
The more varying are the definitions of `f_A`, `f_B`, and `f_C` the more
parametric should
`g` get, as this is the case for any generic function.

With a correct design, I believe, these are all independent of Trees that
Grow story itself:
we are now not only trying to reuse data types, and functions agnostic
towards extensions
(pretty printers in my view of their semantics), but also reuse functions
with parametric /
varying behaviour with respect to extensions.

/Shayan



On Fri, Jul 28, 2017 at 10:18 AM, Simon Peyton Jones 
wrote:

> Devs,
>
>
>
> Shayan is working away on “Trees that grow”… do keep it on your radar:
>
>
>
> *To:* ghc-devs
> *Sent:* 25 May 2017 23:49
>
> Do take a look at this:
>
>
>
> · We propose to re-engineer HsSyn itself.  This will touch a *lot*
> of code.
>
> · But it’s very neat, and will bring big long-term advantages
>
> · And we can do it a bit at a time
>
>
>
> The wiki page https://ghc.haskell.org/trac/ghc/wiki/
> ImplementingTreesThatGrow has the details.  It’s entirely an internal
> change, not a change to GHC’s specification, so it’s independent of the GHC
> proposals process.  But I’d value the opinion of other GHC devs
>
>
>
> Meanwhile I have a question. When pretty-printing HsSyn we often have a
> situation like this:
>
>
>
>   data Pass = Parsed | Renamed | Typechecked
>
>
>
>   data HsExpr (p :: Pass) = HsVar (IdP p) | 
>
>
>
>   type famliy IdP p where
>
>  IdP Parsed  = RdrName
>
>  IdP Renamed = Name
>
>  IdP Typechecked = Id
>
>
>
>   instance (Outputable (IdP p)) => Outputable (HsExpr p) where
>
>  ppr (HsVar v) = ppr v
>
>
>
> The (ppr v) requires (Outputable (IdP p)), hence the context.
>
>
>
> Moreover, and more seriously, there are things we just can't pretty-print
>
> right now.  For example, HsExpr has this data constructor:
>
>
>
>   data HsExpr p = ...
>
> | OpApp   (LHsExpr p)
>
>   (LHsExpr p)
>
>   (PostRn p Fixity)
>
>   (LHsExpr p)
>
>
>
> To pretty-print the third argument, we'd need to add
>
>
>
>   instance (Outputable (IdP p),
>
> Outputable (PostRn p Fixity))   -- New
>
> => Outputable (HsExpr p) where
>
>  ppr (HsVar v) = ppr v
>
>
>
> and that gets onerous.  *So today we never print these annotations*, to
> avoid bloating the instance contexts, which can be painful.  It bit me
> yesterday.
>
>
>
> We have bitten that bullet for the Data class: look at HsExtension.DataId,
> which abbreviates the long list of dictionaries:
>
>
>
>   type DataId p =
>
> ( Data p
>
> , ForallX Data p
>
> , Data (NameOrRdrName (IdP p))
>
> , Data (IdP p)
>
> , Data (PostRn p (IdP p))
>
> , Data (PostRn p (Located Name))
>
> , Data (PostRn p Bool)
>
> , Data (PostRn p Fixity)
>
>  ,..and nine more... )
>
>
>
> Let me note in passing that [wiki:QuantifiedContexts
> ] would make
> this somewhat shorter
>
>
>
>   type DataId p =
>
> ( Data p
>
> , ForallX Data p
>
> , Data (NameOrRdrName (IdP p))
>
> , Data (IdP p)
>
> , forall t. Data t => Data (PostRn p t))
>
>
>
> But we still need one item in this list for each type function,
>
> and I am worried about how this scales to the
>
> 

Re: Trees that Grow in the hsSyn AST

[Shayan Najd]

Hi David,

Which is better to start with: HsSyn or Core? Intuition suggests this sort
> of thing could be very helpful for making zapping more reliable and
> ensuring its efficiency, but there may be better reasons to start with
> HsSyn.


- Why not making Core growable as well?
We just have not considered making Core AST growable so far, though it is
entirely doable if needed.
Specially, if you have some motivating examples that a growable Core AST
"could be very helpful" like for "making zapping more reliable and ensuring
its efficiency", let us know; we can consider making Core growable as well
(possibly as an independent project).

- Why not making Core growable first?
Here the idea is to make HsSyn AST growable for the long-term goals stated
on the wiki page, like getting rid of the multiple representations of
Haskell syntax (i.e., HsSyn, Template Haskell, and Haskell-Src-Exts).
I imagine making Core AST growable does not get us closer to our current
long-term goals (except maybe as an experiment to study the impact of such
AST changes on GHC, like we did in D3609)

2. If we're making intrusive changes to representations, would now be a
> sensible era to consider switching to a different variable representation
> (unbound, bound, abt, etc)?


- While we are at it, can we consider improving some bits of the AST?
For the first few steps, I really hope to keep the changes to
representations as little as possible, at least to ease the reviewing
process and to avoid introducing bugs (provided the large scale of the
changes).
For the next steps, we can indeed consider such improvements. (We do so for
source locations, at least)

Do you have some specific changes in mind? Specially the ones that may
overlap with our work?

Thanks.

/Shayan



On Fri, May 26, 2017 at 2:11 AM, David Feuer <da...@well-typed.com> wrote:

> I haven't looked in detail yet, but there seem to be good ideas. I have
> two questions:
>
> 1. Which is better to start with: HsSyn or Core? Intuition suggests this
> sort of thing could be very helpful for making zapping more reliable and
> ensuring its efficiency, but there may be better reasons to start with
> HsSyn.
>
> 2. If we're making intrusive changes to representations, would now be a
> sensible era to consider switching to a different variable representation
> (unbound, bound, abt, etc)?
>
>
> David Feuer
> Well-Typed, LLP
>
>  Original message 
> From: Simon Peyton Jones via ghc-devs <ghc-devs@haskell.org>
> Date: 5/25/17 6:48 PM (GMT-05:00)
> To: Alan & Kim Zimmerman <alan.z...@gmail.com>, ghc-devs@haskell.org
> Subject: RE: Trees that Grow in the hsSyn AST
>
> Folks
>
> Do take a look at this:
>
>
> ·We propose to re-engineer HsSyn itself.  This will touch a lot of
> code.
>
> ·But it’s very neat, and will bring big long-term advantages
>
> ·And we can do it a bit at a time
>
> The wiki page https://ghc.haskell.org/trac/ghc/wiki/
> ImplementingTreesThatGrow has the details.
>
> It’s entirely an internal change, not a change to GHC’s specification, so
> it’s independent of the GHC proposals process.  But I’d value the opinion
> of other GHC devs.
>
> Alan has done a prototype first step, which worked out rather well.
> Rather than having
>HsExpr Id
> (which we all know means “HsExpr after the typechecker” but tha’s a bit
> inexplicit, we have
>HsExpr GhcTc
> meaning “HsExpr after GHC’s Tc pass”.   In some ways this is quite
> superficial, but it unlocks the Trees That Grow machiney.
>
> Please ask questions etc.  Alan and Shayan can record the answers in the
> wiki.  I’m inclined to go ahead with this, so yell soon if you disagree.
>
> Simon
>
> From: ghc-devs [mailto:ghc-devs-boun...@haskell.org] On Behalf Of Alan &
> Kim Zimmerman
> Sent: 24 May 2017 22:52
> To: ghc-devs@haskell.org
> Subject: Trees that Grow in the hsSyn AST
>
> Hi all
>
> You may be aware that Shayan Najd presented the paper  "Trees that
> Grow"[1] at HIW last year.
> Based on the following mandate
> > As in my previous email to Shayan (attached).  Wiki page, describe
> goals, design,
> > approach.  Point to prototype implementation.  Seek comments.   You can
> say that
> >I am supportive!
> >
> > Simon
>
> We have set up a Wiki page at [2] describing a prototype implementation of
> the first stage of this for the hsSyn AST, which is to change the
> polymorphic variable from one of RdrName / Name / Id to an index type. This
> is presented as a fabricator diff at [3].
> Please take a look and provide feedback.
> Regards
>   Alan
>
>
> [1] http://www.jucs.org/jucs_23_1/trees_that_grow/jucs_23_01_
> 

Re: Request for feedback: deriving strategies syntax

[Shayan Najd]

Hi all,

Shayan, have you written anything describing how things are going?


Ben,
  thank you for reaching out.

I am not sure about the history and the context of the discussions in this
thread so far, but here is a brief description of what we intend to do and
how far we have come so far.

The idea is simple: allow grand larceny, with a clear conscience!
We are trying to allow metaprograms to directly access and reuse the
"native" machinery inside the compiler. There is no need to have a separate
representation of syntax, and the associated sets of tools, as in Template
Haskell, or even as in third-party libraries like Haskell-Src-Exts (HSE)
and others in Haskell-Suite. GHC has a "native" representation of terms,
with many tools already built on top of that, including the compiler passes
like the renamer, or the typechecker. We would certainly like to unify
these representations and tools as much as possible, and allow metaprograms
to directly access these internal machineries.

The eventual goal is indeed more than sole reuse of the AST in GHC (HsSyn);
metaprograms should also be able to reuse the "infrastructure" like the
different environments and monads used for name resolution or typechecking.
If we are to treat `deriving`, and many other similar constructs in the
language simply as metaprograms, access to types is *sometimes* crucial:
they describe a type-directed elaboration process.
I am a strong advocate of such simplifications for both the front-end
(i.e., how users perceive constructs in the language), and the back-end
(i.e., how the compiler implements them). However, there is a huge, often
underestimated, gap between having these constructs as built-in, and
 having them as yet another metaprogram. The gap is both in theory, e.g.,
the equational and algebraic properties of the constructs, and in practice,
e.g., handling the error messages and maintain the general
macro/metaprogramming system allowing for such definitions. Many
researchers, including me, have worked on these problems; there is still
work to be done. We can discuss some of these related works, if you are
interested.

For now, as the first step, we are focusing on how to reuse GHC AST (HsSyn)
for metaprogramming. The immediate problem with reusing GHC AST is that it
comes with a large set of extra fields and constructors carrying the
information only necessary for the passes inside GHC.
Users (metaprogrammers?) do not want to, and do not need to, deal with
these extra fields and constructors. Moreover, the AST should be,
to some degree, easy to use, and we have no exact metric for the ease of
use.

Our solution is twofold, addressing the two mentioned problems:
  (a) after a rather deep analysis, we have come up with a simple, yet
powerful, encoding of extensible ASTs that allows us to extend a base AST
with new fields and constructors;
  (b) we are updating GHC AST to match HSE AST, hence unifying the two
(based on its popularity, we can say HSE is easy enough to use).

In theory, due to (a), code for GHC passes do not need to change.
In practice, due to (b), we may need to update the code in GHC passes in
some cases.

It is needless to say, that this is all work in progress.
Following Simon's suggestion, I am planning to give a talk on this in
Haskell Implementors' Workshop in Japan. But since then, for further
information, you may be brave enough to read our sketchy notes / pieces of
code:
- An example using the extensible encoding (a bit outdated variant though):

https://github.com/shayan-najd/NativeMetaprogramming/blob/master/HsSyn/Example.hs
- Discussion on why this extensible encoding is suitable:

https://github.com/shayan-najd/NativeMetaprogramming/wiki/Extensions-&-Annotations
- To see progress on unifying HSE AST and GHC AST compare
  the updated GHC AST at

https://github.com/shayan-najd/NativeMetaprogramming/blob/master/HsSyn/SyntaxExtensibleAutoSplitted.hs
  and the updated HSE AST at

https://github.com/shayan-najd/NativeMetaprogramming/blob/master/HSE/SyntaxExtensibleAuto.hs
   They should be line-by-line comparable unless the comments says so
otherwise.

There is much to be done without a doubt, specially to synchronise with and
learn form communities working on similar topics

Yours,
  Shayan

On Fri, Aug 5, 2016 at 11:06 AM, Ben Gamari <b...@smart-cactus.org> wrote:

> Ryan Scott <ryan.gl.sc...@gmail.com> writes:
>
> > Sorry for not including the full context on that link. It's part of a
> > Summer of Haskell 2016 project called Native Metaprogramming in
> > Haskell [1] (a.k.a. Introspective Template Haskell [2]), aiming to fix
> > Trac #11081 [3].
> >
> On this note, it would be great to hear a bit about the state of this
> project. Shayan, have you written anything describing how things are
> going? It would be great if you could update the Wiki page [2]
> describing a bit about the