RE: [commit: ghc] master: Module reexports, fixing #8407. (7f5c1086)

[Simon Peyton Jones]

Edward

Great stuff.  Is this documented somewhere, notably in 
http://www.haskell.org/ghc/docs/latest/html/users_guide/packages.html
for GHC, and somewhere in Cabal?

And perhaps somewhere on the wiki
https://ghc.haskell.org/trac/ghc/wiki/Commentary/Packages


Thanks

Simon

| -Original Message-
| From: ghc-commits [mailto:ghc-commits-boun...@haskell.org] On Behalf Of
| g...@git.haskell.org
| Sent: 26 July 2014 02:08
| To: ghc-comm...@haskell.org
| Subject: [commit: ghc] master: Module reexports, fixing #8407.
| (7f5c1086)
| 
| Repository : ssh://g...@git.haskell.org/ghc
| 
| On branch  : master
| Link   :
| http://ghc.haskell.org/trac/ghc/changeset/7f5c10864e7c26b90c7ff4ed09d00
| c8a09aa4349/ghc
| 
| >---
| 
| commit 7f5c10864e7c26b90c7ff4ed09d00c8a09aa4349
| Author: Edward Z. Yang 
| Date:   Fri Jul 4 17:01:08 2014 +0100
| 
| Module reexports, fixing #8407.
| 
| The general approach is to add a new field to the package database,
| reexported-modules, which considered by the module finder as
| possible
| module declarations.  Unlike declaring stub module files, multiple
| reexports of the same physical package at the same name do not
| result in an ambiguous import.
| 
| Has submodule updates for Cabal and haddock.
| 
| NB: When a reexport renames a module, that renaming is *not*
| accessible
| from inside the package.  This is not so much a deliberate design
| choice
| as for implementation expediency (reexport resolution happens only
| when
| a package is in the package database.)
| 
| TODO: Error handling when there are duplicate reexports/etc is not
| very
| well tested.
| 
| Signed-off-by: Edward Z. Yang 
| 
| Conflicts:
|   compiler/main/HscTypes.lhs
|   testsuite/.gitignore
|   utils/haddock
| 
| 
| >---
| 
| 7f5c10864e7c26b90c7ff4ed09d00c8a09aa4349
|  compiler/main/DynFlags.hs  |   1 +
|  compiler/main/Finder.lhs   |  25 +++--
|  compiler/main/GHC.hs   |  12 ++-
|  compiler/main/HscTypes.lhs |   6 +-
|  compiler/main/PackageConfig.hs |   4 +
|  compiler/main/Packages.lhs | 109
| -
|  ghc/InteractiveUI.hs   |   8 +-
|  libraries/Cabal|   2 +-
|  .../Distribution/InstalledPackageInfo/Binary.hs|   8 ++
|  testsuite/.gitignore   |   8 ++
|  testsuite/tests/cabal/Makefile |  15 +++
|  testsuite/tests/cabal/all.T|   6 ++
|  testsuite/tests/cabal/cabal05/Makefile |  69 +
|  .../{driver/T3007/A => cabal/cabal05}/Setup.hs |   0
|  testsuite/tests/cabal/{cabal03 => cabal05}/all.T   |   4 +-
|  .../tests/cabal/cabal05/p/LICENSE  |   0
|  testsuite/tests/cabal/cabal05/p/P.hs   |   3 +
|  testsuite/tests/cabal/cabal05/p/P2.hs  |   1 +
|  .../{driver/T3007/A => cabal/cabal05/p}/Setup.hs   |   0
|  testsuite/tests/cabal/cabal05/p/p.cabal|  11 +++
|  .../tests/cabal/cabal05/q/LICENSE  |   0
|  testsuite/tests/cabal/cabal05/q/Q.hs   |   4 +
|  .../{driver/T3007/A => cabal/cabal05/q}/Setup.hs   |   0
|  testsuite/tests/cabal/cabal05/q/q.cabal|  29 ++
|  .../tests/cabal/cabal05/r/LICENSE  |   0
|  testsuite/tests/cabal/cabal05/r/R.hs   |  11 +++
|  .../{driver/T3007/A => cabal/cabal05/r}/Setup.hs   |   0
|  testsuite/tests/cabal/cabal05/r/r.cabal|  32 ++
|  .../tests/cabal/cabal05/s/LICENSE  |   0
|  testsuite/tests/cabal/cabal05/s/S.hs   |  18 
|  .../{driver/T3007/A => cabal/cabal05/s}/Setup.hs   |   0
|  testsuite/tests/cabal/cabal05/s/s.cabal|  11 +++
|  testsuite/tests/cabal/ghcpkg07.stdout  |  11 +++
|  .../{test4.pkg => recache_reexport_db/a.conf}  |  20 ++--
|  testsuite/tests/cabal/{test4.pkg => test7a.pkg}|  20 ++--
|  testsuite/tests/cabal/test7b.pkg   |  17 
|  utils/ghc-cabal/ghc-cabal.cabal|   3 +-
|  utils/ghc-pkg/Main.hs  |  55 ++-
|  utils/ghc-pkg/ghc-pkg.cabal|   4 +-
|  utils/ghctags/ghctags.cabal|   3 +-
|  utils/haddock  |   2 +-
|  41 files changed, 453 insertions(+), 79 deletions(-)
| 
| Diff suppressed because of size. To see it, use:
| 
| git diff-tree --root --patch-with-stat --no-color --find-copies-
| harder --ignore-space-at-eol --cc
| 7f5c10864e7c26b90c7ff4ed09d00c8a09aa4349
| ___
| ghc-commits mailing list
| ghc-comm...@haskell.org
| http://www.ha

Re: Broken Data.Data instances

[Edward Kmett]

Im mostly looking at the Data.Data stuff as "nice to have" at this point.
Well, it really is need to have for some users, but they can typically get
by by writing a few hundred lines of boilerplate when its not there.

If you need to break something internally and it costs us a Data instance
for something? Have at it.

If we can still hack around the changes with Data then great.

Otherwise the Data machinery has always been for expert users who already
deal with a great deal of breakage anyways, so thrashing on that API seems
fine to me. Not desirable, but not unexpected.

-Edward


On Sun, Jul 27, 2014 at 9:49 PM, Richard Eisenberg 
wrote:

> What if there is a good reason for a missing/broken Data.Data instance?
> I'm specifically thinking of GADTs. There are few currently, but I, for
> one, have toyed with the idea of adding more. My recollection is that
> Data.Data doesn't work with GADTs. As a concrete, existent example, see
> CoAxiom.BranchList, which allows for type-level reification of singleton
> lists as distinct from other, not-necessarily-singleton lists.
>
> I would very much like to support API usage that would benefit from
> working Data.Data instances, but I also want to be sure we're not
> eliminating other possible futures without due discussion.
>
> Richard
>
> On Jul 27, 2014, at 2:04 PM, "Alan & Kim Zimmerman" 
> wrote:
>
> Philip
>
> How would you like to take this forward? From my side I would appreciate
> all guidance/help to get it resolved, it is a huge hindrance for HaRe.
>
> Alan
>
>
> On Sun, Jul 27, 2014 at 7:27 PM, Edward Kmett  wrote:
>
>> Philip, Alan,
>>
>> If you need a hand, I'm happy to pitch in guidance.
>>
>> I've had to mangle a bunch of hand-written Data instances and push out
>> patches to a dozen packages that used to be built this way before I
>> convinced the authors to switch to safer versions of Data. Using virtual
>> smart constructors like we do now in containers and Text where needed can
>> be used to preserve internal invariants, etc.
>>
>> This works far better for users of the API than just randomly throwing
>> them a live hand grenade. As I recall, these little grenades in generic
>> programming over the GHC API have been a constant source of pain for
>> libraries like haddock.
>>
>> Simon,
>>
>> It seems to me that regarding circular data structures, nothing prevents
>> you from walking a circular data structure with Data.Data. You can generate
>> a new one productively that looks just like the old with the contents
>> swapped out, it is indistinguishable to an observer if the fixed point is
>> lost, and a clever observer can use observable sharing to get it back,
>> supposing that they are allowed to try.
>>
>> Alternately, we could use the 'virtual constructor' trick there to break
>> the cycle and reintroduce it, but I'm less enthusiastic about that idea,
>> even if it is simpler in many ways.
>>
>> -Edward
>>
>>
>> On Sun, Jul 27, 2014 at 10:17 AM,  wrote:
>>
>>>  Alan,
>>>
>>> In that case, let's have a short feedback-loop between the two of us. It
>>> seems many of these files (Name.lhs, for example) are really stable through
>>> the repo-history. It would be nice to have one bigger refactoring all in
>>> one go (some of the code could use a polish, a lot of code seems removable).
>>>
>>> Regards,
>>> Philip
>>>
>>>  --
>>> *Van:* Alan & Kim Zimmerman [alan.z...@gmail.com]
>>> *Verzonden:* vrijdag 25 juli 2014 13:44
>>> *Aan:* Simon Peyton Jones
>>> *CC:* Holzenspies, P.K.F. (EWI); ghc-devs@haskell.org
>>> *Onderwerp:* Re: Broken Data.Data instances
>>>
>>>   By the way, I would be happy to attempt this task, if the concept is
>>> viable.
>>>
>>>
>>> On Thu, Jul 24, 2014 at 11:23 PM, Alan & Kim Zimmerman <
>>> alan.z...@gmail.com> wrote:
>>>
While we are talking about fixing traversals, how about getting rid
 of the phase specific panic initialisers for placeHolderType,
 placeHolderKind and friends?

  In order to safely traverse with SYB, the following needs to be
 inserted into all the SYB schemes (see

 https://github.com/alanz/HaRe/blob/master/src/Language/Haskell/Refact/Utils/GhcUtils.hs
 )

 -- Check the Typeable items
 checkItemStage1 :: (Typeable a) => SYB.Stage -> a -> Bool
 checkItemStage1 stage x = (const False `SYB.extQ` postTcType `SYB.extQ`
 fixity `SYB.extQ` nameSet) x
   where nameSet = const (stage `elem` [SYB.Parser,SYB.TypeChecker])
 :: GHC.NameSet   -> Bool
 postTcType  = const (stage < SYB.TypeChecker  )
 :: GHC.PostTcType-> Bool
 fixity  = const (stage < SYB.Renamer  )
 :: GHC.Fixity-> Bool

  And in addition HsCmdTop and ParStmtBlock are initialised with
 explicit 'undefined values.

  Perhaps use an initialiser that can have its panic turned off when
 called via the GHC API?

  Regards
Alan

>>

Re: Broken Data.Data instances

[Richard Eisenberg]

What if there is a good reason for a missing/broken Data.Data instance? I'm 
specifically thinking of GADTs. There are few currently, but I, for one, have 
toyed with the idea of adding more. My recollection is that Data.Data doesn't 
work with GADTs. As a concrete, existent example, see CoAxiom.BranchList, which 
allows for type-level reification of singleton lists as distinct from other, 
not-necessarily-singleton lists.

I would very much like to support API usage that would benefit from working 
Data.Data instances, but I also want to be sure we're not eliminating other 
possible futures without due discussion.

Richard

On Jul 27, 2014, at 2:04 PM, "Alan & Kim Zimmerman"  wrote:

> Philip
> 
> How would you like to take this forward? From my side I would appreciate all 
> guidance/help to get it resolved, it is a huge hindrance for HaRe.
> 
> Alan
> 
> 
> On Sun, Jul 27, 2014 at 7:27 PM, Edward Kmett  wrote:
> Philip, Alan, 
> 
> If you need a hand, I'm happy to pitch in guidance. 
> 
> I've had to mangle a bunch of hand-written Data instances and push out 
> patches to a dozen packages that used to be built this way before I convinced 
> the authors to switch to safer versions of Data. Using virtual smart 
> constructors like we do now in containers and Text where needed can be used 
> to preserve internal invariants, etc.
> 
> This works far better for users of the API than just randomly throwing them a 
> live hand grenade. As I recall, these little grenades in generic programming 
> over the GHC API have been a constant source of pain for libraries like 
> haddock.
> 
> Simon,
> 
> It seems to me that regarding circular data structures, nothing prevents you 
> from walking a circular data structure with Data.Data. You can generate a new 
> one productively that looks just like the old with the contents swapped out, 
> it is indistinguishable to an observer if the fixed point is lost, and a 
> clever observer can use observable sharing to get it back, supposing that 
> they are allowed to try.
> 
> Alternately, we could use the 'virtual constructor' trick there to break the 
> cycle and reintroduce it, but I'm less enthusiastic about that idea, even if 
> it is simpler in many ways.
> 
> -Edward
> 
> 
> On Sun, Jul 27, 2014 at 10:17 AM,  wrote:
> Alan,
> 
> In that case, let's have a short feedback-loop between the two of us. It 
> seems many of these files (Name.lhs, for example) are really stable through 
> the repo-history. It would be nice to have one bigger refactoring all in one 
> go (some of the code could use a polish, a lot of code seems removable).
> 
> Regards,
> Philip
> 
> Van: Alan & Kim Zimmerman [alan.z...@gmail.com]
> Verzonden: vrijdag 25 juli 2014 13:44
> Aan: Simon Peyton Jones
> CC: Holzenspies, P.K.F. (EWI); ghc-devs@haskell.org
> Onderwerp: Re: Broken Data.Data instances
> 
> By the way, I would be happy to attempt this task, if the concept is viable.
> 
> 
> On Thu, Jul 24, 2014 at 11:23 PM, Alan & Kim Zimmerman  
> wrote:
> While we are talking about fixing traversals, how about getting rid of the 
> phase specific panic initialisers for placeHolderType, placeHolderKind and 
> friends?
> 
> In order to safely traverse with SYB, the following needs to be inserted into 
> all the SYB schemes (see 
> https://github.com/alanz/HaRe/blob/master/src/Language/Haskell/Refact/Utils/GhcUtils.hs)
> 
> -- Check the Typeable items
> checkItemStage1 :: (Typeable a) => SYB.Stage -> a -> Bool
> checkItemStage1 stage x = (const False `SYB.extQ` postTcType `SYB.extQ` 
> fixity `SYB.extQ` nameSet) x
>   where nameSet = const (stage `elem` [SYB.Parser,SYB.TypeChecker]) :: 
> GHC.NameSet   -> Bool
> postTcType  = const (stage < SYB.TypeChecker  ) :: 
> GHC.PostTcType-> Bool
> fixity  = const (stage < SYB.Renamer  ) :: 
> GHC.Fixity-> Bool
> 
> And in addition HsCmdTop and ParStmtBlock are initialised with explicit 
> 'undefined values.
> 
> Perhaps use an initialiser that can have its panic turned off when called via 
> the GHC API?
> 
> Regards
>   Alan
> 
> 
> 
> On Thu, Jul 24, 2014 at 11:06 PM, Simon Peyton Jones  
> wrote:
> So... does anyone object to me changing these "broken" instances with the 
> ones given by DeriveDataTypeable?
> 
> 
> That’s fine with me provided (a) the default behaviour is not immediate 
> divergence (which it might well be), and (b) the pitfalls are documented.
> 
>  
> 
> Simon
> 
>  
> 
> From: "Philip K.F. Hölzenspies" [mailto:p.k.f.holzensp...@utwente.nl] 
> Sent: 24 July 2014 18:42
> To: Simon Peyton Jones
> Cc: ghc-devs@haskell.org
> Subject: Re: Broken Data.Data instances
> 
>  
> 
> Dear Simon, et al,
> 
> These are very good points to make for people writing such traversals and 
> queries. I would be more than happy to write a page on the pitfalls etc. on 
> the wiki, but in my experience so far, exploring the innards of GHC is 
> tremendously helped by trying small things out an

Re: Broken Data.Data instances

[Alan & Kim Zimmerman]

Philip

How would you like to take this forward? From my side I would appreciate
all guidance/help to get it resolved, it is a huge hindrance for HaRe.

Alan


On Sun, Jul 27, 2014 at 7:27 PM, Edward Kmett  wrote:

> Philip, Alan,
>
> If you need a hand, I'm happy to pitch in guidance.
>
> I've had to mangle a bunch of hand-written Data instances and push out
> patches to a dozen packages that used to be built this way before I
> convinced the authors to switch to safer versions of Data. Using virtual
> smart constructors like we do now in containers and Text where needed can
> be used to preserve internal invariants, etc.
>
> This works far better for users of the API than just randomly throwing
> them a live hand grenade. As I recall, these little grenades in generic
> programming over the GHC API have been a constant source of pain for
> libraries like haddock.
>
> Simon,
>
> It seems to me that regarding circular data structures, nothing prevents
> you from walking a circular data structure with Data.Data. You can generate
> a new one productively that looks just like the old with the contents
> swapped out, it is indistinguishable to an observer if the fixed point is
> lost, and a clever observer can use observable sharing to get it back,
> supposing that they are allowed to try.
>
> Alternately, we could use the 'virtual constructor' trick there to break
> the cycle and reintroduce it, but I'm less enthusiastic about that idea,
> even if it is simpler in many ways.
>
> -Edward
>
>
> On Sun, Jul 27, 2014 at 10:17 AM,  wrote:
>
>>  Alan,
>>
>> In that case, let's have a short feedback-loop between the two of us. It
>> seems many of these files (Name.lhs, for example) are really stable through
>> the repo-history. It would be nice to have one bigger refactoring all in
>> one go (some of the code could use a polish, a lot of code seems removable).
>>
>> Regards,
>> Philip
>>
>>  --
>> *Van:* Alan & Kim Zimmerman [alan.z...@gmail.com]
>> *Verzonden:* vrijdag 25 juli 2014 13:44
>> *Aan:* Simon Peyton Jones
>> *CC:* Holzenspies, P.K.F. (EWI); ghc-devs@haskell.org
>> *Onderwerp:* Re: Broken Data.Data instances
>>
>>   By the way, I would be happy to attempt this task, if the concept is
>> viable.
>>
>>
>> On Thu, Jul 24, 2014 at 11:23 PM, Alan & Kim Zimmerman <
>> alan.z...@gmail.com> wrote:
>>
>>>While we are talking about fixing traversals, how about getting rid
>>> of the phase specific panic initialisers for placeHolderType,
>>> placeHolderKind and friends?
>>>
>>>  In order to safely traverse with SYB, the following needs to be
>>> inserted into all the SYB schemes (see
>>>
>>> https://github.com/alanz/HaRe/blob/master/src/Language/Haskell/Refact/Utils/GhcUtils.hs
>>> )
>>>
>>> -- Check the Typeable items
>>> checkItemStage1 :: (Typeable a) => SYB.Stage -> a -> Bool
>>> checkItemStage1 stage x = (const False `SYB.extQ` postTcType `SYB.extQ`
>>> fixity `SYB.extQ` nameSet) x
>>>   where nameSet = const (stage `elem` [SYB.Parser,SYB.TypeChecker])
>>> :: GHC.NameSet   -> Bool
>>> postTcType  = const (stage < SYB.TypeChecker  )
>>> :: GHC.PostTcType-> Bool
>>> fixity  = const (stage < SYB.Renamer  )
>>> :: GHC.Fixity-> Bool
>>>
>>>  And in addition HsCmdTop and ParStmtBlock are initialised with explicit
>>> 'undefined values.
>>>
>>>  Perhaps use an initialiser that can have its panic turned off when
>>> called via the GHC API?
>>>
>>>  Regards
>>>Alan
>>>
>>>
>>>
>>>  On Thu, Jul 24, 2014 at 11:06 PM, Simon Peyton Jones <
>>> simo...@microsoft.com> wrote:
>>>
So... does anyone object to me changing these "broken" instances
 with the ones given by DeriveDataTypeable?

  That’s fine with me provided (a) the default behaviour is not
 immediate divergence (which it might well be), and (b) the pitfalls are
 documented.



 Simon



 *From:* "Philip K.F. Hölzenspies" [mailto:p.k.f.holzensp...@utwente.nl]

 *Sent:* 24 July 2014 18:42
 *To:* Simon Peyton Jones
 *Cc:* ghc-devs@haskell.org
 *Subject:* Re: Broken Data.Data instances



 Dear Simon, et al,

 These are very good points to make for people writing such traversals
 and queries. I would be more than happy to write a page on the pitfalls
 etc. on the wiki, but in my experience so far, exploring the innards of GHC
 is tremendously helped by trying small things out and showing (bits of) the
 intermediate structures. For me, personally, this has always been hindered
 by the absence of good instances of Data and/or Show (not having to bring
 DynFlags and not just visualising with the pretty printer are very 
 helpful).

 So... does anyone object to me changing these "broken" instances with
 the ones given by DeriveDataTypeable?

 Also, many of these internal data structures could be provided with
 u

Re: Broken Data.Data instances

[Edward Kmett]

Philip, Alan,

If you need a hand, I'm happy to pitch in guidance.

I've had to mangle a bunch of hand-written Data instances and push out
patches to a dozen packages that used to be built this way before I
convinced the authors to switch to safer versions of Data. Using virtual
smart constructors like we do now in containers and Text where needed can
be used to preserve internal invariants, etc.

This works far better for users of the API than just randomly throwing them
a live hand grenade. As I recall, these little grenades in generic
programming over the GHC API have been a constant source of pain for
libraries like haddock.

Simon,

It seems to me that regarding circular data structures, nothing prevents
you from walking a circular data structure with Data.Data. You can generate
a new one productively that looks just like the old with the contents
swapped out, it is indistinguishable to an observer if the fixed point is
lost, and a clever observer can use observable sharing to get it back,
supposing that they are allowed to try.

Alternately, we could use the 'virtual constructor' trick there to break
the cycle and reintroduce it, but I'm less enthusiastic about that idea,
even if it is simpler in many ways.

-Edward


On Sun, Jul 27, 2014 at 10:17 AM,  wrote:

>  Alan,
>
> In that case, let's have a short feedback-loop between the two of us. It
> seems many of these files (Name.lhs, for example) are really stable through
> the repo-history. It would be nice to have one bigger refactoring all in
> one go (some of the code could use a polish, a lot of code seems removable).
>
> Regards,
> Philip
>
>  --
> *Van:* Alan & Kim Zimmerman [alan.z...@gmail.com]
> *Verzonden:* vrijdag 25 juli 2014 13:44
> *Aan:* Simon Peyton Jones
> *CC:* Holzenspies, P.K.F. (EWI); ghc-devs@haskell.org
> *Onderwerp:* Re: Broken Data.Data instances
>
>   By the way, I would be happy to attempt this task, if the concept is
> viable.
>
>
> On Thu, Jul 24, 2014 at 11:23 PM, Alan & Kim Zimmerman <
> alan.z...@gmail.com> wrote:
>
>>While we are talking about fixing traversals, how about getting rid
>> of the phase specific panic initialisers for placeHolderType,
>> placeHolderKind and friends?
>>
>>  In order to safely traverse with SYB, the following needs to be inserted
>> into all the SYB schemes (see
>>
>> https://github.com/alanz/HaRe/blob/master/src/Language/Haskell/Refact/Utils/GhcUtils.hs
>> )
>>
>> -- Check the Typeable items
>> checkItemStage1 :: (Typeable a) => SYB.Stage -> a -> Bool
>> checkItemStage1 stage x = (const False `SYB.extQ` postTcType `SYB.extQ`
>> fixity `SYB.extQ` nameSet) x
>>   where nameSet = const (stage `elem` [SYB.Parser,SYB.TypeChecker])
>> :: GHC.NameSet   -> Bool
>> postTcType  = const (stage < SYB.TypeChecker  )
>> :: GHC.PostTcType-> Bool
>> fixity  = const (stage < SYB.Renamer  )
>> :: GHC.Fixity-> Bool
>>
>>  And in addition HsCmdTop and ParStmtBlock are initialised with explicit
>> 'undefined values.
>>
>>  Perhaps use an initialiser that can have its panic turned off when
>> called via the GHC API?
>>
>>  Regards
>>Alan
>>
>>
>>
>>  On Thu, Jul 24, 2014 at 11:06 PM, Simon Peyton Jones <
>> simo...@microsoft.com> wrote:
>>
>>>So... does anyone object to me changing these "broken" instances
>>> with the ones given by DeriveDataTypeable?
>>>
>>>  That’s fine with me provided (a) the default behaviour is not
>>> immediate divergence (which it might well be), and (b) the pitfalls are
>>> documented.
>>>
>>>
>>>
>>> Simon
>>>
>>>
>>>
>>> *From:* "Philip K.F. Hölzenspies" [mailto:p.k.f.holzensp...@utwente.nl]
>>> *Sent:* 24 July 2014 18:42
>>> *To:* Simon Peyton Jones
>>> *Cc:* ghc-devs@haskell.org
>>> *Subject:* Re: Broken Data.Data instances
>>>
>>>
>>>
>>> Dear Simon, et al,
>>>
>>> These are very good points to make for people writing such traversals
>>> and queries. I would be more than happy to write a page on the pitfalls
>>> etc. on the wiki, but in my experience so far, exploring the innards of GHC
>>> is tremendously helped by trying small things out and showing (bits of) the
>>> intermediate structures. For me, personally, this has always been hindered
>>> by the absence of good instances of Data and/or Show (not having to bring
>>> DynFlags and not just visualising with the pretty printer are very helpful).
>>>
>>> So... does anyone object to me changing these "broken" instances with
>>> the ones given by DeriveDataTypeable?
>>>
>>> Also, many of these internal data structures could be provided with
>>> useful lenses to improve such traversals further. Anyone ever go at that?
>>> Would be people be interested?
>>>
>>> Regards,
>>> Philip
>>>
>>>
>>>  *Simon Peyton Jones* 
>>>
>>> 24 Jul 2014 18:22
>>>
>>> GHC’s data structures are often mutually recursive. e.g.
>>>
>>> ·The TyCon for Maybe contains the DataCon for Just
>>>
>>> ·The DataCon For just contai

Re: Broken Data.Data instances

[Alan & Kim Zimmerman]

To my knowledge there is no trac ticket to make the AST safe. Is this
correct? Can I make one?

Alan


On Sun, Jul 27, 2014 at 4:28 PM, Alan & Kim Zimmerman 
wrote:

> My intention would be to simply change the placeholders into something
> that would not blow up during a normal traversal, preferably something that
> still gives the required behaviour when invoked normally by GHC, to
> indicate a bug that needs fixing, but that can somehow be turned off at
> other tiimes.
>
> I am open to suggestions as to a mechanism  that can achieve this, I
> thought of some kind of setting via Dynamic Flags.
>
>
> Alan
>
>
> On Sun, Jul 27, 2014 at 4:17 PM,  wrote:
>
>>  Alan,
>>
>> In that case, let's have a short feedback-loop between the two of us. It
>> seems many of these files (Name.lhs, for example) are really stable through
>> the repo-history. It would be nice to have one bigger refactoring all in
>> one go (some of the code could use a polish, a lot of code seems removable).
>>
>> Regards,
>> Philip
>>
>>  --
>> *Van:* Alan & Kim Zimmerman [alan.z...@gmail.com]
>> *Verzonden:* vrijdag 25 juli 2014 13:44
>> *Aan:* Simon Peyton Jones
>> *CC:* Holzenspies, P.K.F. (EWI); ghc-devs@haskell.org
>> *Onderwerp:* Re: Broken Data.Data instances
>>
>>   By the way, I would be happy to attempt this task, if the concept is
>> viable.
>>
>>
>> On Thu, Jul 24, 2014 at 11:23 PM, Alan & Kim Zimmerman <
>> alan.z...@gmail.com> wrote:
>>
>>>While we are talking about fixing traversals, how about getting rid
>>> of the phase specific panic initialisers for placeHolderType,
>>> placeHolderKind and friends?
>>>
>>>  In order to safely traverse with SYB, the following needs to be
>>> inserted into all the SYB schemes (see
>>>
>>> https://github.com/alanz/HaRe/blob/master/src/Language/Haskell/Refact/Utils/GhcUtils.hs
>>> )
>>>
>>> -- Check the Typeable items
>>> checkItemStage1 :: (Typeable a) => SYB.Stage -> a -> Bool
>>> checkItemStage1 stage x = (const False `SYB.extQ` postTcType `SYB.extQ`
>>> fixity `SYB.extQ` nameSet) x
>>>   where nameSet = const (stage `elem` [SYB.Parser,SYB.TypeChecker])
>>> :: GHC.NameSet   -> Bool
>>> postTcType  = const (stage < SYB.TypeChecker  )
>>> :: GHC.PostTcType-> Bool
>>> fixity  = const (stage < SYB.Renamer  )
>>> :: GHC.Fixity-> Bool
>>>
>>>  And in addition HsCmdTop and ParStmtBlock are initialised with explicit
>>> 'undefined values.
>>>
>>>  Perhaps use an initialiser that can have its panic turned off when
>>> called via the GHC API?
>>>
>>>  Regards
>>>Alan
>>>
>>>
>>>
>>>  On Thu, Jul 24, 2014 at 11:06 PM, Simon Peyton Jones <
>>> simo...@microsoft.com> wrote:
>>>
So... does anyone object to me changing these "broken" instances
 with the ones given by DeriveDataTypeable?

  That’s fine with me provided (a) the default behaviour is not
 immediate divergence (which it might well be), and (b) the pitfalls are
 documented.



 Simon



 *From:* "Philip K.F. Hölzenspies" [mailto:p.k.f.holzensp...@utwente.nl]

 *Sent:* 24 July 2014 18:42
 *To:* Simon Peyton Jones
 *Cc:* ghc-devs@haskell.org
 *Subject:* Re: Broken Data.Data instances



 Dear Simon, et al,

 These are very good points to make for people writing such traversals
 and queries. I would be more than happy to write a page on the pitfalls
 etc. on the wiki, but in my experience so far, exploring the innards of GHC
 is tremendously helped by trying small things out and showing (bits of) the
 intermediate structures. For me, personally, this has always been hindered
 by the absence of good instances of Data and/or Show (not having to bring
 DynFlags and not just visualising with the pretty printer are very 
 helpful).

 So... does anyone object to me changing these "broken" instances with
 the ones given by DeriveDataTypeable?

 Also, many of these internal data structures could be provided with
 useful lenses to improve such traversals further. Anyone ever go at that?
 Would be people be interested?

 Regards,
 Philip


  *Simon Peyton Jones* 

 24 Jul 2014 18:22

 GHC’s data structures are often mutually recursive. e.g.

 ·The TyCon for Maybe contains the DataCon for Just

 ·The DataCon For just contains Just’s type

 ·Just’s type contains the TyCon for Maybe



 So any attempt to recursively walk over all these structures, as you
 would a tree, will fail.



 Also there’s a lot of sharing.  For example, every occurrence of ‘map’
 is a Var, and inside that Var is map’s type, its strictness, its rewrite
 RULE, etc etc.  In walking over a term you may not want to walk over all
 that stuff at every occurrence of map.



>>>

Re: phabricator issue with git submodules.

[Edward Z . Yang]

That's right.  I am actually not even sure how Harbormaster even
manages to find your commits for the build...

Edward

Excerpts from Karel Gardas's message of 2014-07-27 18:02:24 +0100:
> 
> Hello Edward,
> 
> I've done that, see https://phabricator.haskell.org/D96 -- but now I'm 
> curious but since this is done in this way, basically speaking 
> library/unix + libraries/primitive now points to commits done in my 
> forks of those libs on github.com waiting for approval since I already 
> pushed appropriate pull requests. Now this also means that D96 is 
> probably not includable in GHC HEAD since it points to currently 
> non-existing patches (in public libraries/unix + primitive). Am I right 
> that this works in this way?
> 
> Thanks,
> Karel
> 
> On 07/26/14 09:39 PM, Edward Z. Yang wrote:
> > Hello Karel,
> >
> > When your submodules get updated, you need to add them to your commit
> > (since the parent repository maintains pointers to the submodules).
> > Then they will no longer show up as dirty and you can submit the
> > Phabricator patch.
> >
> > Edward
> >
> > Excerpts from Karel Gardas's message of 2014-07-25 22:48:21 +0100:
> >>
> >> Hi,
> >>
> >> just fixing few warning issues on Solaris/x86. The changes spread over
> >> main ghc tree and libraries/primitive and libraries/unix. I already
> >> commited changes and pushed to my github.com's forks of
> >> libraries/primitive and libraries/unix. The git status looks then:
> >>
> >> $ git status
> >> On branch master
> >> Your branch is ahead of 'origin/master' by 2 commits.
> >> (use "git push" to publish your local commits)
> >>
> >> Changes not staged for commit:
> >> (use "git add..." to update what will be committed)
> >> (use "git checkout --..." to discard changes in working 
> >> directory)
> >>
> >>   modified:   libraries/primitive (new commits)
> >>   modified:   libraries/unix (new commits)
> >>
> >> no changes added to commit (use "git add" and/or "git commit -a")
> >>
> >>
> >> and yet phabricator still complains about it:
> >>
> >> $ arc diff
> >> You have unstaged changes in this working copy.
> >>
> >> Working copy: /export/home/karel/vcs/ghc-src/validate-fixes/
> >>
> >> Unstaged changes in working copy:
> >>   libraries/primitive
> >>   libraries/unix
> >>
> >>
> >>   Do you want to amend these files to the commit? [y/N]
> >>
> >> Usage Exception: Stage and commit (or revert) them before proceeding.
> >>
> >> I pressed enter in question above.
> >>
> >> Is that a known issue or am I doing something wrong here?
> >>
> >> Thanks!
> >> Karel
> >
___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs

Re: phabricator issue with git submodules.

[Karel Gardas]

Hello Edward,

I've done that, see https://phabricator.haskell.org/D96 -- but now I'm 
curious but since this is done in this way, basically speaking 
library/unix + libraries/primitive now points to commits done in my 
forks of those libs on github.com waiting for approval since I already 
pushed appropriate pull requests. Now this also means that D96 is 
probably not includable in GHC HEAD since it points to currently 
non-existing patches (in public libraries/unix + primitive). Am I right 
that this works in this way?


Thanks,
Karel

On 07/26/14 09:39 PM, Edward Z. Yang wrote:

Hello Karel,

When your submodules get updated, you need to add them to your commit
(since the parent repository maintains pointers to the submodules).
Then they will no longer show up as dirty and you can submit the
Phabricator patch.

Edward

Excerpts from Karel Gardas's message of 2014-07-25 22:48:21 +0100:


Hi,

just fixing few warning issues on Solaris/x86. The changes spread over
main ghc tree and libraries/primitive and libraries/unix. I already
commited changes and pushed to my github.com's forks of
libraries/primitive and libraries/unix. The git status looks then:

$ git status
On branch master
Your branch is ahead of 'origin/master' by 2 commits.
(use "git push" to publish your local commits)

Changes not staged for commit:
(use "git add..." to update what will be committed)
(use "git checkout --..." to discard changes in working directory)

  modified:   libraries/primitive (new commits)
  modified:   libraries/unix (new commits)

no changes added to commit (use "git add" and/or "git commit -a")


and yet phabricator still complains about it:

$ arc diff
You have unstaged changes in this working copy.

Working copy: /export/home/karel/vcs/ghc-src/validate-fixes/

Unstaged changes in working copy:
  libraries/primitive
  libraries/unix


  Do you want to amend these files to the commit? [y/N]

Usage Exception: Stage and commit (or revert) them before proceeding.

I pressed enter in question above.

Is that a known issue or am I doing something wrong here?

Thanks!
Karel




___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs

Re: Broken Data.Data instances

[Alan & Kim Zimmerman]

My intention would be to simply change the placeholders into something that
would not blow up during a normal traversal, preferably something that
still gives the required behaviour when invoked normally by GHC, to
indicate a bug that needs fixing, but that can somehow be turned off at
other tiimes.

I am open to suggestions as to a mechanism  that can achieve this, I
thought of some kind of setting via Dynamic Flags.


Alan


On Sun, Jul 27, 2014 at 4:17 PM,  wrote:

>  Alan,
>
> In that case, let's have a short feedback-loop between the two of us. It
> seems many of these files (Name.lhs, for example) are really stable through
> the repo-history. It would be nice to have one bigger refactoring all in
> one go (some of the code could use a polish, a lot of code seems removable).
>
> Regards,
> Philip
>
>  --
> *Van:* Alan & Kim Zimmerman [alan.z...@gmail.com]
> *Verzonden:* vrijdag 25 juli 2014 13:44
> *Aan:* Simon Peyton Jones
> *CC:* Holzenspies, P.K.F. (EWI); ghc-devs@haskell.org
> *Onderwerp:* Re: Broken Data.Data instances
>
>   By the way, I would be happy to attempt this task, if the concept is
> viable.
>
>
> On Thu, Jul 24, 2014 at 11:23 PM, Alan & Kim Zimmerman <
> alan.z...@gmail.com> wrote:
>
>>While we are talking about fixing traversals, how about getting rid
>> of the phase specific panic initialisers for placeHolderType,
>> placeHolderKind and friends?
>>
>>  In order to safely traverse with SYB, the following needs to be inserted
>> into all the SYB schemes (see
>>
>> https://github.com/alanz/HaRe/blob/master/src/Language/Haskell/Refact/Utils/GhcUtils.hs
>> )
>>
>> -- Check the Typeable items
>> checkItemStage1 :: (Typeable a) => SYB.Stage -> a -> Bool
>> checkItemStage1 stage x = (const False `SYB.extQ` postTcType `SYB.extQ`
>> fixity `SYB.extQ` nameSet) x
>>   where nameSet = const (stage `elem` [SYB.Parser,SYB.TypeChecker])
>> :: GHC.NameSet   -> Bool
>> postTcType  = const (stage < SYB.TypeChecker  )
>> :: GHC.PostTcType-> Bool
>> fixity  = const (stage < SYB.Renamer  )
>> :: GHC.Fixity-> Bool
>>
>>  And in addition HsCmdTop and ParStmtBlock are initialised with explicit
>> 'undefined values.
>>
>>  Perhaps use an initialiser that can have its panic turned off when
>> called via the GHC API?
>>
>>  Regards
>>Alan
>>
>>
>>
>>  On Thu, Jul 24, 2014 at 11:06 PM, Simon Peyton Jones <
>> simo...@microsoft.com> wrote:
>>
>>>So... does anyone object to me changing these "broken" instances
>>> with the ones given by DeriveDataTypeable?
>>>
>>>  That’s fine with me provided (a) the default behaviour is not
>>> immediate divergence (which it might well be), and (b) the pitfalls are
>>> documented.
>>>
>>>
>>>
>>> Simon
>>>
>>>
>>>
>>> *From:* "Philip K.F. Hölzenspies" [mailto:p.k.f.holzensp...@utwente.nl]
>>> *Sent:* 24 July 2014 18:42
>>> *To:* Simon Peyton Jones
>>> *Cc:* ghc-devs@haskell.org
>>> *Subject:* Re: Broken Data.Data instances
>>>
>>>
>>>
>>> Dear Simon, et al,
>>>
>>> These are very good points to make for people writing such traversals
>>> and queries. I would be more than happy to write a page on the pitfalls
>>> etc. on the wiki, but in my experience so far, exploring the innards of GHC
>>> is tremendously helped by trying small things out and showing (bits of) the
>>> intermediate structures. For me, personally, this has always been hindered
>>> by the absence of good instances of Data and/or Show (not having to bring
>>> DynFlags and not just visualising with the pretty printer are very helpful).
>>>
>>> So... does anyone object to me changing these "broken" instances with
>>> the ones given by DeriveDataTypeable?
>>>
>>> Also, many of these internal data structures could be provided with
>>> useful lenses to improve such traversals further. Anyone ever go at that?
>>> Would be people be interested?
>>>
>>> Regards,
>>> Philip
>>>
>>>
>>>  *Simon Peyton Jones* 
>>>
>>> 24 Jul 2014 18:22
>>>
>>> GHC’s data structures are often mutually recursive. e.g.
>>>
>>> ·The TyCon for Maybe contains the DataCon for Just
>>>
>>> ·The DataCon For just contains Just’s type
>>>
>>> ·Just’s type contains the TyCon for Maybe
>>>
>>>
>>>
>>> So any attempt to recursively walk over all these structures, as you
>>> would a tree, will fail.
>>>
>>>
>>>
>>> Also there’s a lot of sharing.  For example, every occurrence of ‘map’
>>> is a Var, and inside that Var is map’s type, its strictness, its rewrite
>>> RULE, etc etc.  In walking over a term you may not want to walk over all
>>> that stuff at every occurrence of map.
>>>
>>>
>>>
>>> Maybe that’s it; I’m not certain since I did not write the Data
>>> instances for any of GHC’s types
>>>
>>>
>>>
>>> Simon
>>>
>>>
>>>
>>> *From:* ghc-devs [mailto:ghc-devs-boun...@haskell.org
>>> ] *On Behalf Of *
>>> p.k.f.holzensp...@utwente.nl
>>> *Sent:* 24 July 2014 16:42
>>> *To:* ghc-devs@haskell.org
>>> *Subject:* Br

RE: Broken Data.Data instances

[p.k.f.holzenspies]

Alan,

In that case, let's have a short feedback-loop between the two of us. It seems 
many of these files (Name.lhs, for example) are really stable through the 
repo-history. It would be nice to have one bigger refactoring all in one go 
(some of the code could use a polish, a lot of code seems removable).

Regards,
Philip


Van: Alan & Kim Zimmerman [alan.z...@gmail.com]
Verzonden: vrijdag 25 juli 2014 13:44
Aan: Simon Peyton Jones
CC: Holzenspies, P.K.F. (EWI); ghc-devs@haskell.org
Onderwerp: Re: Broken Data.Data instances

By the way, I would be happy to attempt this task, if the concept is viable.


On Thu, Jul 24, 2014 at 11:23 PM, Alan & Kim Zimmerman 
mailto:alan.z...@gmail.com>> wrote:
While we are talking about fixing traversals, how about getting rid of the 
phase specific panic initialisers for placeHolderType, placeHolderKind and 
friends?

In order to safely traverse with SYB, the following needs to be inserted into 
all the SYB schemes (see
https://github.com/alanz/HaRe/blob/master/src/Language/Haskell/Refact/Utils/GhcUtils.hs)

-- Check the Typeable items
checkItemStage1 :: (Typeable a) => SYB.Stage -> a -> Bool
checkItemStage1 stage x = (const False `SYB.extQ` postTcType `SYB.extQ` fixity 
`SYB.extQ` nameSet) x
  where nameSet = const (stage `elem` [SYB.Parser,SYB.TypeChecker]) :: 
GHC.NameSet   -> Bool
postTcType  = const (stage < SYB.TypeChecker  ) :: 
GHC.PostTcType-> Bool
fixity  = const (stage < SYB.Renamer  ) :: 
GHC.Fixity-> Bool

And in addition HsCmdTop and ParStmtBlock are initialised with explicit 
'undefined values.

Perhaps use an initialiser that can have its panic turned off when called via 
the GHC API?

Regards
  Alan



On Thu, Jul 24, 2014 at 11:06 PM, Simon Peyton Jones 
mailto:simo...@microsoft.com>> wrote:
So... does anyone object to me changing these "broken" instances with the ones 
given by DeriveDataTypeable?

That’s fine with me provided (a) the default behaviour is not immediate 
divergence (which it might well be), and (b) the pitfalls are documented.

Simon

From: "Philip K.F. Hölzenspies" 
[mailto:p.k.f.holzensp...@utwente.nl]
Sent: 24 July 2014 18:42
To: Simon Peyton Jones
Cc: ghc-devs@haskell.org
Subject: Re: Broken Data.Data instances

Dear Simon, et al,

These are very good points to make for people writing such traversals and 
queries. I would be more than happy to write a page on the pitfalls etc. on the 
wiki, but in my experience so far, exploring the innards of GHC is tremendously 
helped by trying small things out and showing (bits of) the intermediate 
structures. For me, personally, this has always been hindered by the absence of 
good instances of Data and/or Show (not having to bring DynFlags and not just 
visualising with the pretty printer are very helpful).

So... does anyone object to me changing these "broken" instances with the ones 
given by DeriveDataTypeable?

Also, many of these internal data structures could be provided with useful 
lenses to improve such traversals further. Anyone ever go at that? Would be 
people be interested?

Regards,
Philip


[cid:image001.jpg@01CFA78B.7D356DE0]
Simon Peyton Jones
24 Jul 2014 18:22
GHC’s data structures are often mutually recursive. e.g.

•The TyCon for Maybe contains the DataCon for Just

•The DataCon For just contains Just’s type

•Just’s type contains the TyCon for Maybe

So any attempt to recursively walk over all these structures, as you would a 
tree, will fail.

Also there’s a lot of sharing.  For example, every occurrence of ‘map’ is a 
Var, and inside that Var is map’s type, its strictness, its rewrite RULE, etc 
etc.  In walking over a term you may not want to walk over all that stuff at 
every occurrence of map.

Maybe that’s it; I’m not certain since I did not write the Data instances for 
any of GHC’s types

Simon

From: ghc-devs [mailto:ghc-devs-boun...@haskell.org] On Behalf Of 
p.k.f.holzensp...@utwente.nl
Sent: 24 July 2014 16:42
To: ghc-devs@haskell.org
Subject: Broken Data.Data instances

Dear GHC-ers,

Is there a reason for explicitly broken Data.Data instances? Case in point:

> instance Data Var where
>   -- don't traverse?
>   toConstr _   = abstractConstr "Var"
>   gunfold _ _  = error "gunfold"
>   dataTypeOf _ = mkNoRepType "Var"

I understand (vaguely) arguments about abstract data types, but this also 
excludes convenient queries that can, e.g. extract all types from a CoreExpr. I 
had hoped to do stuff like this:

> collect :: (Typeable b, Data a, MonadPlus m) => a -> m b
> collect = everything mplus $ mkQ mzero return
>
> allTypes :: CoreExpr -> [Type]
> allTypes = collect

Especially when still exploring (parts of) the GHC API, being able to extract 
things in this fashion is