Re: Proposed gcc plugin plugin API mk 2 (this time without camel case!)
Hi, (Sorry for the delay.) I suppose the proposed API doesn’t cover all the needs of your Python bindings and their applications, does it? How do you plan to export the GIMPLE and tree.h APIs? Regarding iterators, there are things like: GCC_IMPLEMENT_PUBLIC_API(bool) gcc_cfg_for_each_block(gcc_cfg cfg, bool (*cb)(gcc_cfg_block block, void *user_data), void *user_data) { int i; for (i = 0; i < cfg.inner->x_n_basic_blocks; i++) { Instead of the callback API, could the details of the containers be exposed a little more, like: /* Return the number of basic blocks in CFG. */ unsigned int gcc_cfg_block_count (gcc_cfg cfg); /* Return a pointer to the array of basic blocks in CFG. */ gcc_cfg_block *gcc_cfg_blocks (gcc_cfg cfg); That would allow higher-level language bindings to provide idiomatic ways to iterate over basic blocks, such as (lazy) list iterators. WDYT? Thanks, Ludo’.
Re: Proposed gcc plugin plugin API mk 2 (this time without camel case!)
On Tue, Apr 3, 2012 at 6:02 PM, David Malcolm wrote: > On Tue, 2012-04-03 at 15:23 +0200, Richard Guenther wrote: >> On Tue, Apr 3, 2012 at 12:03 PM, Richard Guenther >> wrote: >> > On Mon, Apr 2, 2012 at 7:21 PM, David Malcolm wrote: >> >> I wrote a script and ported my proposed API for GCC plugins from my >> >> CamelCase naming convention to an underscore_based_convention (and >> >> manually fixed up things in a few places also). >> >> >> >> The result compiles and passes the full test suite for the Python >> >> plugin; that said, I'm still breaking the encapsulation in quite a few >> >> places (hey, this is an experimental prototype). >> >> >> >> You can see the latest version of it within the "proposed-plugin-api" >> >> branch of the Python plugin here: >> >> http://git.fedorahosted.org/git/?p=gcc-python-plugin.git;a=shortlog;h=refs/heads/proposed-plugin-api >> >> >> >> within the "proposed-plugin-api" subdirectory. >> > >> > Hmm, how do I get it? I did >> > >> > git clone http://git.fedorahosted.org/git/proposed-plugin-api >> > >> > but there is nothing in gcc-python-plugin/. And >> > >> > git checkout proposed-plugin-api >> > >> > says I'm already there ...? >> >> Meanwhile the directory magically appeared (heh ...). > > [The ways of git are something of a mystery to me: 95% of the time it's > the best revision control system I've ever used, but 5% of the time the > most obtuse] > >> Overall it looks good > Thanks for taking a look. > >> - but it seems to leak GCC headers into the >> plugin API (via gcc-plugin.h and input.h inclusion). Thus, it >> lacks separating the plugin API headers from the plugin API implementation >> headers? > That's true. The big information "leak" happens inside > gcc-semiprivate-types.h, which defines the various structs that act like > pointers, each with a decl like this: > > struct gcc_something { > some_private_gcc_pointer_type inner; > }; > > It would be possible to make this more opaque like this: > > struct gcc_something { > struct some_private_gcc_struct *inner; > }; > > given that you then don't need a full definition of that inner struct > visible to users. Though location_t is leaked, and in this approach, > there isn't a way around that, I think. See the reply by Roman. > >> Or maybe I'm not looking at the right place. >> I also dislike the use of GCC_PUBLIC_API, etc. - everything in >> the plugin API headers should be obviously public - and the implementation >> detail should be an implementation detail that should not need to care. > > I added that macro based on the example of other plugin/embedding > systems I've seen (e.g. Python), where it's handy to make that macro be > non-trivial on some platforms. See e.g. CPython's pyport.h: > http://hg.python.org/cpython/file/9599f091faa6/Include/pyport.h > where the macros PyAPI_FUNC and PyAPI_DATA have a 44-line definition, > handling various different compatibility cases. > > For example, GCC_PRIVATE_API could have some magic linker flag that lets > us automatically strip out those symbols so that they're not visible > externally after the compiler has been linked. That's for the implementation side - the public API side by definition has only declarations considered public. >> >> If this landed e.g. in GCC 4.8, would this be usable by other plugins? >> >> For sure. I'd say get the copyright stuff sorted out and start pushing >> things >> into the main GCC repository - where the obvious starting point is to >> get the makefile, configure and install parts correct. >> >> I'd concentrate on providing enough API for introspection, like simple >> things, counting basic-blocks, counting calls, etc. I'm not sure we >> want to expose gcc_gimple_walk_tree or gcc_gimple_print (or >> the gcc_printers for a start) - the latter would something that the >> python plugin >> would provide, resorting to introspecting the stmt itself. > FWIW the Python plugin already heavily uses gcc's pretty-printer code, > so that *is* something I'd want to wrap (but it's fairly easy to do so). Ok, fair enough. >> I also wonder about gcc_gimple_phi_upcast - why would you specialize >> PHI nodes but not any others? I'd have exposed gcc_gimple_code. >> In general the plugin API should provide exactly one (and the most flexible) >> way to do a thing (thus, not have gcc_gimple_assign_single_p) and >> the high-level consumers like the python plugin should provide >> nice-to-use interfaces. > > This touches on the biggest thing that's missing in the API: the > multitude of tree types, gimple statement types, and rtl types. All I > added were the "base classes", and gimple-phi is the only instance so > far in the API of a subclass. Well, one way is surely to view it as sort-of "classes", the other way is to view it as tagged structures which you can simply expose. Of course if there will be a day where somebody converts all of gimple and tree to a C++ class hierarchy the class way would be a better representation. > I woul
Re: Proposed gcc plugin plugin API mk 2 (this time without camel case!)
On Tue, Apr 3, 2012 at 8:15 PM, Romain Geissler wrote: > > Le 3 avr. 2012 à 18:02, David Malcolm a écrit : > >> On Tue, 2012-04-03 at 15:23 +0200, Richard Guenther wrote: >>> On Tue, Apr 3, 2012 at 12:03 PM, Richard Guenther >>> wrote: On Mon, Apr 2, 2012 at 7:21 PM, David Malcolm wrote: > I wrote a script and ported my proposed API for GCC plugins from my > CamelCase naming convention to an underscore_based_convention (and > manually fixed up things in a few places also). > > The result compiles and passes the full test suite for the Python > plugin; that said, I'm still breaking the encapsulation in quite a few > places (hey, this is an experimental prototype). > > You can see the latest version of it within the "proposed-plugin-api" > branch of the Python plugin here: > http://git.fedorahosted.org/git/?p=gcc-python-plugin.git;a=shortlog;h=refs/heads/proposed-plugin-api > > within the "proposed-plugin-api" subdirectory. Hmm, how do I get it? I did git clone http://git.fedorahosted.org/git/proposed-plugin-api but there is nothing in gcc-python-plugin/. And git checkout proposed-plugin-api says I'm already there ...? >>> >>> Meanwhile the directory magically appeared (heh ...). >> >> [The ways of git are something of a mystery to me: 95% of the time it's >> the best revision control system I've ever used, but 5% of the time the >> most obtuse] >> >>> Overall it looks good >> Thanks for taking a look. >> >>> - but it seems to leak GCC headers into the >>> plugin API (via gcc-plugin.h and input.h inclusion). Thus, it >>> lacks separating the plugin API headers from the plugin API implementation >>> headers? >> That's true. The big information "leak" happens inside >> gcc-semiprivate-types.h, which defines the various structs that act like >> pointers, each with a decl like this: >> >> struct gcc_something { >> some_private_gcc_pointer_type inner; >> }; >> >> It would be possible to make this more opaque like this: >> >> struct gcc_something { >> struct some_private_gcc_struct *inner; >> }; >> >> given that you then don't need a full definition of that inner struct >> visible to users. Though location_t is leaked, and in this approach, >> there isn't a way around that, I think. > > Well i think we you should define a public type like this : > > typedef struct some_private_gcc_struct *gcc_something; > > extern some_type retrieve_some_value(gcc_something); Indeed. I'd go one step further and do typedef struct gcc_something_s *gcc_something; and in the implementations only expose struct some_private_gcc_struct by means of casting the pointer. For the public API gcc_something is a pointer to an opaque thing, never dereferenced. > Also, nothing should be noted public or private : all definitions > that will appear in a header installed in > $(gcc -print-file-name=plugin)/include will be public by definition. Right. Richard. > Any additional header that would be needed to implement the > API should be kept separate (like the actual *.c implementing it) > and placed in the gcc/ directory in the trunk (or better something > like gcc/plugin-impl/ to start being modular). Any definition defined > in those additional headers are private. > > Thus, you should define two sets of headers files (public and private ones), > plus body c files, and import only public header files from public header > files. > > Do you have any plan on starting integrating it to the trunk (or at least on > an > new branch on the official gcc repository) soon, like suggested by Richard ? > I might help setting up the configure/makefile and later writing some > wrappers. > (although i don't have write permission). > Cheers > > Romain Geissler
Re: Proposed gcc plugin plugin API mk 2 (this time without camel case!)
Le 3 avr. 2012 à 18:02, David Malcolm a écrit : > On Tue, 2012-04-03 at 15:23 +0200, Richard Guenther wrote: >> On Tue, Apr 3, 2012 at 12:03 PM, Richard Guenther >> wrote: >>> On Mon, Apr 2, 2012 at 7:21 PM, David Malcolm wrote: I wrote a script and ported my proposed API for GCC plugins from my CamelCase naming convention to an underscore_based_convention (and manually fixed up things in a few places also). The result compiles and passes the full test suite for the Python plugin; that said, I'm still breaking the encapsulation in quite a few places (hey, this is an experimental prototype). You can see the latest version of it within the "proposed-plugin-api" branch of the Python plugin here: http://git.fedorahosted.org/git/?p=gcc-python-plugin.git;a=shortlog;h=refs/heads/proposed-plugin-api within the "proposed-plugin-api" subdirectory. >>> >>> Hmm, how do I get it? I did >>> >>> git clone http://git.fedorahosted.org/git/proposed-plugin-api >>> >>> but there is nothing in gcc-python-plugin/. And >>> >>> git checkout proposed-plugin-api >>> >>> says I'm already there ...? >> >> Meanwhile the directory magically appeared (heh ...). > > [The ways of git are something of a mystery to me: 95% of the time it's > the best revision control system I've ever used, but 5% of the time the > most obtuse] > >> Overall it looks good > Thanks for taking a look. > >> - but it seems to leak GCC headers into the >> plugin API (via gcc-plugin.h and input.h inclusion). Thus, it >> lacks separating the plugin API headers from the plugin API implementation >> headers? > That's true. The big information "leak" happens inside > gcc-semiprivate-types.h, which defines the various structs that act like > pointers, each with a decl like this: > > struct gcc_something { > some_private_gcc_pointer_type inner; > }; > > It would be possible to make this more opaque like this: > > struct gcc_something { > struct some_private_gcc_struct *inner; > }; > > given that you then don't need a full definition of that inner struct > visible to users. Though location_t is leaked, and in this approach, > there isn't a way around that, I think. Well i think we you should define a public type like this : typedef struct some_private_gcc_struct *gcc_something; extern some_type retrieve_some_value(gcc_something); Also, nothing should be noted public or private : all definitions that will appear in a header installed in $(gcc -print-file-name=plugin)/include will be public by definition. Any additional header that would be needed to implement the API should be kept separate (like the actual *.c implementing it) and placed in the gcc/ directory in the trunk (or better something like gcc/plugin-impl/ to start being modular). Any definition defined in those additional headers are private. Thus, you should define two sets of headers files (public and private ones), plus body c files, and import only public header files from public header files. Do you have any plan on starting integrating it to the trunk (or at least on an new branch on the official gcc repository) soon, like suggested by Richard ? I might help setting up the configure/makefile and later writing some wrappers. (although i don't have write permission). Cheers Romain Geissler
Re: Proposed gcc plugin plugin API mk 2 (this time without camel case!)
On Tue, 2012-04-03 at 15:23 +0200, Richard Guenther wrote: > On Tue, Apr 3, 2012 at 12:03 PM, Richard Guenther > wrote: > > On Mon, Apr 2, 2012 at 7:21 PM, David Malcolm wrote: > >> I wrote a script and ported my proposed API for GCC plugins from my > >> CamelCase naming convention to an underscore_based_convention (and > >> manually fixed up things in a few places also). > >> > >> The result compiles and passes the full test suite for the Python > >> plugin; that said, I'm still breaking the encapsulation in quite a few > >> places (hey, this is an experimental prototype). > >> > >> You can see the latest version of it within the "proposed-plugin-api" > >> branch of the Python plugin here: > >> http://git.fedorahosted.org/git/?p=gcc-python-plugin.git;a=shortlog;h=refs/heads/proposed-plugin-api > >> > >> within the "proposed-plugin-api" subdirectory. > > > > Hmm, how do I get it? I did > > > > git clone http://git.fedorahosted.org/git/proposed-plugin-api > > > > but there is nothing in gcc-python-plugin/. And > > > > git checkout proposed-plugin-api > > > > says I'm already there ...? > > Meanwhile the directory magically appeared (heh ...). [The ways of git are something of a mystery to me: 95% of the time it's the best revision control system I've ever used, but 5% of the time the most obtuse] > Overall it looks good Thanks for taking a look. > - but it seems to leak GCC headers into the > plugin API (via gcc-plugin.h and input.h inclusion). Thus, it > lacks separating the plugin API headers from the plugin API implementation > headers? That's true. The big information "leak" happens inside gcc-semiprivate-types.h, which defines the various structs that act like pointers, each with a decl like this: struct gcc_something { some_private_gcc_pointer_type inner; }; It would be possible to make this more opaque like this: struct gcc_something { struct some_private_gcc_struct *inner; }; given that you then don't need a full definition of that inner struct visible to users. Though location_t is leaked, and in this approach, there isn't a way around that, I think. > Or maybe I'm not looking at the right place. > I also dislike the use of GCC_PUBLIC_API, etc. - everything in > the plugin API headers should be obviously public - and the implementation > detail should be an implementation detail that should not need to care. I added that macro based on the example of other plugin/embedding systems I've seen (e.g. Python), where it's handy to make that macro be non-trivial on some platforms. See e.g. CPython's pyport.h: http://hg.python.org/cpython/file/9599f091faa6/Include/pyport.h where the macros PyAPI_FUNC and PyAPI_DATA have a 44-line definition, handling various different compatibility cases. For example, GCC_PRIVATE_API could have some magic linker flag that lets us automatically strip out those symbols so that they're not visible externally after the compiler has been linked. > >> If this landed e.g. in GCC 4.8, would this be usable by other plugins? > > For sure. I'd say get the copyright stuff sorted out and start pushing things > into the main GCC repository - where the obvious starting point is to > get the makefile, configure and install parts correct. > > I'd concentrate on providing enough API for introspection, like simple > things, counting basic-blocks, counting calls, etc. I'm not sure we > want to expose gcc_gimple_walk_tree or gcc_gimple_print (or > the gcc_printers for a start) - the latter would something that the > python plugin > would provide, resorting to introspecting the stmt itself. FWIW the Python plugin already heavily uses gcc's pretty-printer code, so that *is* something I'd want to wrap (but it's fairly easy to do so). > I also wonder about gcc_gimple_phi_upcast - why would you specialize > PHI nodes but not any others? I'd have exposed gcc_gimple_code. > In general the plugin API should provide exactly one (and the most flexible) > way to do a thing (thus, not have gcc_gimple_assign_single_p) and > the high-level consumers like the python plugin should provide > nice-to-use interfaces. This touches on the biggest thing that's missing in the API: the multitude of tree types, gimple statement types, and rtl types. All I added were the "base classes", and gimple-phi is the only instance so far in the API of a subclass. I would like the API to expose these (my plugin uses them to good effect), but doing it in C is likely to be messy: lots of upcasting and downcasting functions. An alternative approach might be to bite the bullet and go to C++; maybe something like this (caveat: I last did any C++ about a decade ago, no idea if the following compiles): // Everything exposed in the public headers are pure virtual functions // leading to abstract classes: no public data namespace gcc { // Abstract base class for objects managed by GCC's garbage-collector class gcmanaged { public: virtual void mark_in_use() const = 0; }; cla
Re: Proposed gcc plugin plugin API mk 2 (this time without camel case!)
On Tue, 2012-04-03 at 12:03 +0200, Richard Guenther wrote: > On Mon, Apr 2, 2012 at 7:21 PM, David Malcolm wrote: > > I wrote a script and ported my proposed API for GCC plugins from my > > CamelCase naming convention to an underscore_based_convention (and > > manually fixed up things in a few places also). > > > > The result compiles and passes the full test suite for the Python > > plugin; that said, I'm still breaking the encapsulation in quite a few > > places (hey, this is an experimental prototype). > > > > You can see the latest version of it within the "proposed-plugin-api" > > branch of the Python plugin here: > > http://git.fedorahosted.org/git/?p=gcc-python-plugin.git;a=shortlog;h=refs/heads/proposed-plugin-api > > > > within the "proposed-plugin-api" subdirectory. > > Hmm, how do I get it? I did > > git clone http://git.fedorahosted.org/git/proposed-plugin-api > > but there is nothing in gcc-python-plugin/. And > > git checkout proposed-plugin-api > > says I'm already there ...? I'm sorry that this was unclear. To checkout the source, on the "proposed-plugin-api" branch: $ git clone git://git.fedorahosted.org/gcc-python-plugin.git -b proposed-plugin-api Cloning into gcc-python-plugin... remote: Counting objects: 8375, done. [...snip...] Resolving deltas: 100% (5774/5774), done. Go into the new working copy: $ cd gcc-python-plugin/ Verify that you're on the experimental branch: $ git branch * proposed-plugin-api The code in the root directory is specific to my Python plugin. The proposed generic plugin API is in a subdirectory which confusingly has the same name as the branch (sorry): $ cd proposed-plugin-api (If you're interested in the python plugin, detailed instructions on building from source can be seen at [1]) Sorry about the confusion Dave [1] http://gcc-python-plugin.readthedocs.org/en/latest/basics.html#building-the-plugin-from-source
Re: Proposed gcc plugin plugin API mk 2 (this time without camel case!)
On Tue, Apr 3, 2012 at 12:03 PM, Richard Guenther wrote: > On Mon, Apr 2, 2012 at 7:21 PM, David Malcolm wrote: >> I wrote a script and ported my proposed API for GCC plugins from my >> CamelCase naming convention to an underscore_based_convention (and >> manually fixed up things in a few places also). >> >> The result compiles and passes the full test suite for the Python >> plugin; that said, I'm still breaking the encapsulation in quite a few >> places (hey, this is an experimental prototype). >> >> You can see the latest version of it within the "proposed-plugin-api" >> branch of the Python plugin here: >> http://git.fedorahosted.org/git/?p=gcc-python-plugin.git;a=shortlog;h=refs/heads/proposed-plugin-api >> >> within the "proposed-plugin-api" subdirectory. > > Hmm, how do I get it? I did > > git clone http://git.fedorahosted.org/git/proposed-plugin-api > > but there is nothing in gcc-python-plugin/. And > > git checkout proposed-plugin-api > > says I'm already there ...? Meanwhile the directory magically appeared (heh ...). Overall it looks good - but it seems to leak GCC headers into the plugin API (via gcc-plugin.h and input.h inclusion). Thus, it lacks separating the plugin API headers from the plugin API implementation headers? Or maybe I'm not looking at the right place. I also dislike the use of GCC_PUBLIC_API, etc. - everything in the plugin API headers should be obviously public - and the implementation detail should be an implementation detail that should not need to care. >> If this landed e.g. in GCC 4.8, would this be usable by other plugins? For sure. I'd say get the copyright stuff sorted out and start pushing things into the main GCC repository - where the obvious starting point is to get the makefile, configure and install parts correct. I'd concentrate on providing enough API for introspection, like simple things, counting basic-blocks, counting calls, etc. I'm not sure we want to expose gcc_gimple_walk_tree or gcc_gimple_print (or the gcc_printers for a start) - the latter would something that the python plugin would provide, resorting to introspecting the stmt itself. I also wonder about gcc_gimple_phi_upcast - why would you specialize PHI nodes but not any others? I'd have exposed gcc_gimple_code. In general the plugin API should provide exactly one (and the most flexible) way to do a thing (thus, not have gcc_gimple_assign_single_p) and the high-level consumers like the python plugin should provide nice-to-use interfaces. Thanks, Richard. >> Hope this is helpful >> Dave >>
Re: Proposed gcc plugin plugin API mk 2 (this time without camel case!)
On Mon, Apr 2, 2012 at 7:21 PM, David Malcolm wrote: > I wrote a script and ported my proposed API for GCC plugins from my > CamelCase naming convention to an underscore_based_convention (and > manually fixed up things in a few places also). > > The result compiles and passes the full test suite for the Python > plugin; that said, I'm still breaking the encapsulation in quite a few > places (hey, this is an experimental prototype). > > You can see the latest version of it within the "proposed-plugin-api" > branch of the Python plugin here: > http://git.fedorahosted.org/git/?p=gcc-python-plugin.git;a=shortlog;h=refs/heads/proposed-plugin-api > > within the "proposed-plugin-api" subdirectory. Hmm, how do I get it? I did git clone http://git.fedorahosted.org/git/proposed-plugin-api but there is nothing in gcc-python-plugin/. And git checkout proposed-plugin-api says I'm already there ...? Richard. > How is this looking? > > If this landed e.g. in GCC 4.8, would this be usable by other plugins? > > Hope this is helpful > Dave >