Thanks for the feedback Richard. It's helpful to know which parts of the implementation you found non-obvious. I will write some more Notes, but first:
* On why we need a RemoteRef of an IORef - because the IORef lives on the server, so GHC has a RemoteRef to it. Did you see Note [External GHCi pointers] in compiler/ghci/GHCi.hs? Does that help? Should I xref it from somewhere else? * We probably could separate Message into two types, it's a good idea. I'll look into doing that. Cheers Simon On 22 June 2016 at 14:36, Richard Eisenberg <e...@cis.upenn.edu> wrote: > It seems there are nice advantages to this approach, and so I'm fine > requiring support from TH. But here is a comment I just posted on D2286 > requesting more comments in the code: > > --- > I've just spent some time staring at GHCi/TH.hs and friends, and it's > starting to make sense. Would it be possible to add some comments/Notes > there? It would be great to know what is run on the client (GHC, right?) > and what on the server (TH, right?). Also, why do we need a RemoteRef of > an IORef? This, at first, strikes me as odd. I'm sure it makes great > sense once I understand all the other moving parts, but it would be helpful > to have a primer. I also find it a bit confusing that the Message type > contains both messages to the server and messages from the server. For > example, I would expect StartTH and Reify to live in different datatypes, > because the two should never mix. Or am I horribly mistaken? > --- > > Before we require -fexternal-interpreter (which seems to be the endgoal > here), has anyone looked at performance implications? All the marshaling > and unmarshaling has to take a toll. > > Richard > > On Jun 22, 2016, at 8:27 AM, Simon Peyton Jones via ghc-devs < > ghc-devs@haskell.org> wrote: > > It’s a great start, thanks > > Simon > > *From:* Simon Marlow [mailto:marlo...@gmail.com] > *Sent:* 22 June 2016 12:32 > *To:* Simon Peyton Jones <simo...@microsoft.com> > *Cc:* ghc-devs@haskell.org > *Subject:* Re: Require -fexternal-interpreter support for future TH > changes? > > > How's this? > https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler/ExternalInterpreter > > I don't want to go into too much detail in the wiki, because details are > more likely to stay current if they're in Notes in the code. There are > already a few Notes (e.g. > https://phabricator.haskell.org/diffusion/GHC/browse/master/compiler/ghci/GHCi.hs;619958832cbe11096cae3dac9a0a7a5591163a00$86) > but if anything is confusing I'll happily add more Notes. > > Cheers > > Simon > > > > On 22 June 2016 at 11:50, Simon Marlow <marlo...@gmail.com> wrote: > > On 22 June 2016 at 11:37, Simon Peyton Jones <simo...@microsoft.com> > wrote: > > I’m ok with this. *It would certainly be great not to support TWO > mechanisms indefinitely*. > > What are the disadvantages to committing to this path? Would anyone even > notice? > > > Yes, people who are making changes to TH will need to ensure that their > changes work with -fexternal-interpreter. In some cases that might mean > extra work, e.g. if we do TemplateHaskell/Introspective > <https://ghc.haskell.org/trac/ghc/wiki/TemplateHaskell/Introspective> then > potentially the whole of HsSyn needs to be binary serializable. That > worries me quite a lot - THSyn is big but tractable, Generic deriving > handled the generation of the binary instances easily enough, but HsSyn is > another matter entirely. > > > There are a lot of moving parts to the implementation, and I for one am > utterly ignorant of how it all works. I would love to see an > implementation overview, either somewhere in the code or on a wiki page. > Things like: > > · How, when, and where in the compiler is the separate process > started? > > · How do the compiler and server communicate? Unix pipes? Is it > the same on Windows and Unix? > > · What is serialised, when, and how? For example, GHC has some > TH code to run. Do we send a syntax tree? Or compile to bytecode and send > that? Or what? > > · How are external references managed. E.g. if the code to be > run refers to ‘map’ I’m sure we don’t serialise the code for ‘map’. > I’m sure there is a lot more. E.g the [wiki:RemoteGHCi wiki page] refers > to “a library implementing a message type…” but I don’t know what that > library is called or where it lives. > > > > Yes, we should really have a page in the commentary with an overview of > the pieces and the main implementation strategy. I'll write one. > Cheers > Simon > > > > Thanks > > Simon > > *From:* ghc-devs [mailto:ghc-devs-boun...@haskell.org] *On Behalf Of *Simon > Marlow > *Sent:* 22 June 2016 09:51 > *To:* ghc-devs@haskell.org > *Subject:* Require -fexternal-interpreter support for future TH changes? > > > *Background* > > A few months ago I added -fexternal-interpreter to GHC: > > - docs: > > http://downloads.haskell.org/~ghc/latest/docs/html/users_guide/ghci.html#ghc-flag--fexternal-interpreter > > <https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fdownloads.haskell.org%2f~ghc%2flatest%2fdocs%2fhtml%2fusers_guide%2fghci.html%23ghc-flag--fexternal-interpreter&data=01%7C01%7Csimonpj%40064d.mgd.microsoft.com%7C5ca29759b5c9483e03fb08d39a7a60ea%7C72f988bf86f141af91ab2d7cd011db47%7C1&sdata=6t6FA3nUkP5FOi%2fqCBSr3GxMH2rwWNk89je1qBH9GfI%3d> > - wiki, rationale: https://ghc.haskell.org/trac/ghc/wiki/RemoteGHCi > > When -fexternal-interpreter is used, GHC runs interpreted code in a > separate subprocess, and communicates with it using binary messages over a > pipe. > > -fexternal-interpreter currently implements all of TH, quasi-quoting, > annotations, and all the GHCi features except for some features of the > debugger. It is also now implemented on Windows, thanks to Tamar Christina. > > *Proposal* > > I'd like to propose that going forward we commit to maintaining full > support for -fexternal-interpreter, with a view to making it the default. > Why? > > - -fexternal-interpreter will be a prerequisite for GHCJS support, so > maintaining full support for TH in -fexternal-interpreter will ensure that > everything that works with GHC works with GHCJS. > - We will be able to make simplifications in GHC and the build system > once -fexternal-interpreter is the default, because when compiling with > -prof or -dynamic we won't have to compile things twice any more. > - Ultimately we don't want to have two ways of doing everything, > because that's harder to maintain. > > How? > > - I'll make all the TH and quasi-quoting tests run with and without > -fexternal-interpreter, so it will break validate if one of these fails. > > *Why now?* > > There are some TH changes in the pipeline that will need special attention > to work with -fexternal-interpreter. e.g. > https://phabricator.haskell.org/D2286 and > https://ghc.haskell.org/trac/ghc/wiki/TemplateHaskell/Introspective, so > I'd like to raise it now so we can keep the issue in mind. > > > > Cheers > > Simon > > > > > _______________________________________________ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > >
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
