Re: Does it sound a good idea to implement "backend plugins"?
> A long time ago, I’ve tried to inject plugin logic to allows some control > over the driver pipeline (phase ordering) and hooking various code gen > related functions. > > See https://phabricator.haskell.org/D535 Cool! I haven't thoroughly read the history of that diff, but allowing manipulation of a Hooks via a Plugin seems overkill in this case, and even if one can do so, it still doesn't lead to the backend IR types; one would need to use runPhaseHook and modify the behavior after a CgGuts is generated, which unfortunately leads to quite some boilerplate code. > At that time I ran into issues that might simply not exist with plugins > anymore today, but I haven’t looked. Interesting. I'll make sure to consult you in case I'm bitten by some hidden issues when I actually implement it :) > The whole design wasn’t quite right and injects everything into the dynflags. > Also ghc wanted to be able to compile the plugin on the fly, but I needed > the plugin to be loaded very early during the startup phase to exert enough > control of the rest of the pipeline through the plugin. Well, in the case of backend plugins, it isn't supposed to be a home plugin to be compiled and used on the fly. A typical use case would be compiling/installing the plugin to a standalone pkgdb, then used to compile other packages. > > On 5 Oct 2018, at 1:52 AM, Shao, Cheng wrote: > > Adding "pluggable backends" to spin up new targets seems to require quite a > bit of additional infrastructure for initialising a library directory and > package database. But there are probably more specific use cases that need > inspecting/modifying STG or Cmm where plugins would already be useful in > practice. > > > I think setting up a new global libdir/pkgdb is beyond the scope of > backend plugins. The user shall implement his/her own boot script to > configure for the new architecture, generate relevant headers, run > Cabal's Setup program to launch GHC with the plugin loaded. > > Hooks (or rather their locations in the pipeline) are rather ad hoc by > nature, but for Asterius a hook that takes Cmm and takes over from there > seems like a reasonable approach given the current state of things. I think > the Cmm hook you implemented (or something similar) would be perfectly > acceptable to use for now. > > > For the use case of asterius itself, indeed Hooks already fit the use > case for now. But since we seek to upstream our newly added features > in our ghc fork back to ghc hq, we should upstream those changes early > and make them more principled. Compared to Hooks, I prefer to move to > Plugins entirely since: > > * Plugins are more composable, you can load multiple plugins in one > ghc invocation. Hooks are not. > * If I implement the same mechanisms in Plugins, this can be > beneficial to other projects. Currently, in asterius, everything works > via a pile of hacks upon hacks in ghc-toolkit, and it's not good for > reuse. > * The newly added backend plugins shouldn't have visible > correctness/performance impact if they're not used, and it's just a > few local modifications in the ghc codebase. > > On Thu, Oct 4, 2018 at 3:56 PM Shao, Cheng wrote: > > > Hi all, > > > I'm thinking of adding "backend plugins" in the current Plugins > > mechanism which allows one to inspect/modify the IRs post simplifier > > pass (STG/Cmm), similar to the recently added source plugins for HsSyn > > IRs. This can be useful for anyone creating a custom GHC backend to > > target an experimental platform (e.g. the Asterius compiler which > > targets WebAssembly), and previously in order to retrieve those IRs > > from the regular pipeline, we need to use Hooks which is somewhat > > hacky. > > > Does this sound a good idea to you? If so, I can open a trac ticket > > and a Phab diff for this feature. > > > Best, > > Shao Cheng > > ___ > > ghc-devs mailing list > > ghc-devs@haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Does it sound a good idea to implement "backend plugins"?
A long time ago, I’ve tried to inject plugin logic to allows some control over the driver pipeline (phase ordering) and hooking various code gen related functions. See https://phabricator.haskell.org/D535 At that time I ran into issues that might simply not exist with plugins anymore today, but I haven’t looked. The whole design wasn’t quite right and injects everything into the dynflags. Also ghc wanted to be able to compile the plugin on the fly, but I needed the plugin to be loaded very early during the startup phase to exert enough control of the rest of the pipeline through the plugin. Cheers, Moritz Sent from my iPhone On 5 Oct 2018, at 1:52 AM, Shao, Cheng wrote: >> Adding "pluggable backends" to spin up new targets seems to require quite a >> bit of additional infrastructure for initialising a library directory and >> package database. But there are probably more specific use cases that need >> inspecting/modifying STG or Cmm where plugins would already be useful in >> practice. > > I think setting up a new global libdir/pkgdb is beyond the scope of > backend plugins. The user shall implement his/her own boot script to > configure for the new architecture, generate relevant headers, run > Cabal's Setup program to launch GHC with the plugin loaded. > >> Hooks (or rather their locations in the pipeline) are rather ad hoc by >> nature, but for Asterius a hook that takes Cmm and takes over from there >> seems like a reasonable approach given the current state of things. I think >> the Cmm hook you implemented (or something similar) would be perfectly >> acceptable to use for now. > > For the use case of asterius itself, indeed Hooks already fit the use > case for now. But since we seek to upstream our newly added features > in our ghc fork back to ghc hq, we should upstream those changes early > and make them more principled. Compared to Hooks, I prefer to move to > Plugins entirely since: > > * Plugins are more composable, you can load multiple plugins in one > ghc invocation. Hooks are not. > * If I implement the same mechanisms in Plugins, this can be > beneficial to other projects. Currently, in asterius, everything works > via a pile of hacks upon hacks in ghc-toolkit, and it's not good for > reuse. > * The newly added backend plugins shouldn't have visible > correctness/performance impact if they're not used, and it's just a > few local modifications in the ghc codebase. > >>> On Thu, Oct 4, 2018 at 3:56 PM Shao, Cheng wrote: >>> >>> Hi all, >>> >>> I'm thinking of adding "backend plugins" in the current Plugins >>> mechanism which allows one to inspect/modify the IRs post simplifier >>> pass (STG/Cmm), similar to the recently added source plugins for HsSyn >>> IRs. This can be useful for anyone creating a custom GHC backend to >>> target an experimental platform (e.g. the Asterius compiler which >>> targets WebAssembly), and previously in order to retrieve those IRs >>> from the regular pipeline, we need to use Hooks which is somewhat >>> hacky. >>> >>> Does this sound a good idea to you? If so, I can open a trac ticket >>> and a Phab diff for this feature. >>> >>> Best, >>> Shao Cheng >>> ___ >>> ghc-devs mailing list >>> ghc-devs@haskell.org >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Does it sound a good idea to implement "backend plugins"?
> Adding "pluggable backends" to spin up new targets seems to require quite a > bit of additional infrastructure for initialising a library directory and > package database. But there are probably more specific use cases that need > inspecting/modifying STG or Cmm where plugins would already be useful in > practice. I think setting up a new global libdir/pkgdb is beyond the scope of backend plugins. The user shall implement his/her own boot script to configure for the new architecture, generate relevant headers, run Cabal's Setup program to launch GHC with the plugin loaded. > Hooks (or rather their locations in the pipeline) are rather ad hoc by > nature, but for Asterius a hook that takes Cmm and takes over from there > seems like a reasonable approach given the current state of things. I think > the Cmm hook you implemented (or something similar) would be perfectly > acceptable to use for now. For the use case of asterius itself, indeed Hooks already fit the use case for now. But since we seek to upstream our newly added features in our ghc fork back to ghc hq, we should upstream those changes early and make them more principled. Compared to Hooks, I prefer to move to Plugins entirely since: * Plugins are more composable, you can load multiple plugins in one ghc invocation. Hooks are not. * If I implement the same mechanisms in Plugins, this can be beneficial to other projects. Currently, in asterius, everything works via a pile of hacks upon hacks in ghc-toolkit, and it's not good for reuse. * The newly added backend plugins shouldn't have visible correctness/performance impact if they're not used, and it's just a few local modifications in the ghc codebase. > On Thu, Oct 4, 2018 at 3:56 PM Shao, Cheng wrote: >> >> Hi all, >> >> I'm thinking of adding "backend plugins" in the current Plugins >> mechanism which allows one to inspect/modify the IRs post simplifier >> pass (STG/Cmm), similar to the recently added source plugins for HsSyn >> IRs. This can be useful for anyone creating a custom GHC backend to >> target an experimental platform (e.g. the Asterius compiler which >> targets WebAssembly), and previously in order to retrieve those IRs >> from the regular pipeline, we need to use Hooks which is somewhat >> hacky. >> >> Does this sound a good idea to you? If so, I can open a trac ticket >> and a Phab diff for this feature. >> >> Best, >> Shao Cheng >> ___ >> ghc-devs mailing list >> ghc-devs@haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Does it sound a good idea to implement "backend plugins"?
I think it sounds like a potentially good idea in general, but I agree with Ben and Matthew here that a more concrete plan of intended use is needed. Adding "pluggable backends" to spin up new targets seems to require quite a bit of additional infrastructure for initialising a library directory and package database. But there are probably more specific use cases that need inspecting/modifying STG or Cmm where plugins would already be useful in practice. Hooks (or rather their locations in the pipeline) are rather ad hoc by nature, but for Asterius a hook that takes Cmm and takes over from there seems like a reasonable approach given the current state of things. I think the Cmm hook you implemented (or something similar) would be perfectly acceptable to use for now. I don't think it's a problem if a hook exists for some time, and at some point it's superseded by a more general plugin mechanism. Especially with the GHC 6 month release cycle there's not much need for future proofing. Luite On Thu, Oct 4, 2018 at 3:56 PM Shao, Cheng wrote: > Hi all, > > I'm thinking of adding "backend plugins" in the current Plugins > mechanism which allows one to inspect/modify the IRs post simplifier > pass (STG/Cmm), similar to the recently added source plugins for HsSyn > IRs. This can be useful for anyone creating a custom GHC backend to > target an experimental platform (e.g. the Asterius compiler which > targets WebAssembly), and previously in order to retrieve those IRs > from the regular pipeline, we need to use Hooks which is somewhat > hacky. > > Does this sound a good idea to you? If so, I can open a trac ticket > and a Phab diff for this feature. > > Best, > Shao Cheng > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Does it sound a good idea to implement "backend plugins"?
"Shao, Cheng" writes: > Hi all, > > I'm thinking of adding "backend plugins" in the current Plugins > mechanism which allows one to inspect/modify the IRs post simplifier > pass (STG/Cmm), similar to the recently added source plugins for HsSyn > IRs. This can be useful for anyone creating a custom GHC backend to > target an experimental platform (e.g. the Asterius compiler which > targets WebAssembly), and previously in order to retrieve those IRs > from the regular pipeline, we need to use Hooks which is somewhat > hacky. > > Does this sound a good idea to you? If so, I can open a trac ticket > and a Phab diff for this feature. > Yes, during the Implementors' Workshop this year it seemed like there was considerable interest in such a mechanism. However, as Matthew said, the devil is in the details; before starting an implementation I would recommend that you open a ticket describing the specifics of the proposed interface. It also wouldn't hurt to motivate the proposal with a discussion of the concrete use-cases that the interface is meant to address. Cheers, - Ben signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Does it sound a good idea to implement "backend plugins"?
Sounds like a reasonable idea to me. However, you should take some time to propose a concrete interface as this part was not obvious in the design of source plugins. Matt On Thu, Oct 4, 2018 at 2:56 PM Shao, Cheng wrote: > > Hi all, > > I'm thinking of adding "backend plugins" in the current Plugins > mechanism which allows one to inspect/modify the IRs post simplifier > pass (STG/Cmm), similar to the recently added source plugins for HsSyn > IRs. This can be useful for anyone creating a custom GHC backend to > target an experimental platform (e.g. the Asterius compiler which > targets WebAssembly), and previously in order to retrieve those IRs > from the regular pipeline, we need to use Hooks which is somewhat > hacky. > > Does this sound a good idea to you? If so, I can open a trac ticket > and a Phab diff for this feature. > > Best, > Shao Cheng > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
