Re: plugin hooks for plugin-provided builtins?
Given the current limitations of Gimple, another area to focus on could be task parallelism (rather than data parallelism). In that case a language like [Google] Go (via GCC) might make a better talking point than C or Fortran. An even better starting point would be Ada which has built-in multi-tasking constructs. Arno
Re: plugin hooks for plugin-provided builtins?
On 09/15/2010 08:54 AM, Arnaud Charlet wrote: Given the current limitations of Gimple, another area to focus on could be task parallelism (rather than data parallelism). In that case a language like [Google] Go (via GCC) might make a better talking point than C or Fortran. An even better starting point would be Ada which has built-in multi-tasking constructs. Arno Hi everybody, At the last GROW workshop in Pisa, I presented Gomet, an extension to GCC which (among other things) performs GIMPLE-to-C translation for parallelization purpose. http://ctuning.org/dissemination/grow10-02.pdf The paper is out of date now with respect to the progress made in Gomet this year, but you may find some information in there anyway. It is definitely possible to achieve such translation, and the information available in the middle-end are sufficient. If the input source code is C, then the biggest required transformation is to adapt the SSA form and lower PHI nodes. Additional glitches (coming from floating point types, array and type sizes in pointer arithmetics, includes) can be overcome. If the input source code is not C, additional work is required to map specific language constructs to C. In Gomet, a subset of Fortran and C++ is supported for free. I did not try Ada, Go or other input languages. Regards, N. BENOIT
Re: plugin hooks for plugin-provided builtins?
On Tue, 14 Sep 2010, Steven Bosscher wrote: On Tue, Sep 14, 2010 at 5:39 PM, Marcus Daniels mdani...@lanl.gov wrote: On 9/14/10 8:46 AM, Basile Starynkevitch wrote: My current work aims to translate some Gimple into OpenCL source code, thus providing GCC with the ability to take advantage of GPU running their proprietary OpenCL compilers without asking the user to learn OpenCL. My understanding is that Gimple does not have the notion of data parallel operations. There was a prototype patch with middle-end arrays from Richard Guenther (see http://gcc.gnu.org/ml/gcc-patches/2008-04/msg01123.html for the latest revision I'm aware of). I suppose the usual things (other priorities, funding) stand in the way of completing this work. Richi probably can tell more. The most important thing that made me stop working on this (besides priorities, funding, time, etc.) is that for good test coverage the Fortran frontend would need to start using this representation. Which I didn't get around to even try to implement myself. But this is indeed something I like to come back to at some point ... Richard.
Re: plugin hooks for plugin-provided builtins?
On Tue, Sep 14, 2010 at 3:36 PM, Basile Starynkevitch
bas...@starynkevitch.net wrote:
Hello All,
I was thinking of adding a new plugin hook for builtins.
The intuition is that some plugins could be pleased if they could add
their own plugins (much like today's plugins can add their own pragmas
or attributes).
I imagine several use cases for such a feature, for example
* a builtin which generate at compile time a random number, e.g. for
hashing purpose. [this could also be delt by preprocessor magic,
for instance giving the ability for plugins to extend the behavior
of libcpp by adding new plugin-specific predefined macros like
__COUNTER__, so we could have a __RANDOM__ macro which would
expand, at cpp expansion time, to a random number. From what I
understood adding new predefined macros from plugins seems harder
than adding num builtins]
* a builtin which calls a foreign function in some language
implementation (e.g. SBCL Common Lisp or native Ocaml) which have
incompatible calling conventions w.r.t. of those of GCC
* a builtin which simplifies a call to some complex external
function into something simpler. __builtin_printf seems already to
do that, and I could imagine many other cases.
* a builtin to help a precise garbage collector by flushing every
live variable (which might have been declared with a special
attribute) out of registers.
* etc...
My intuition, by first looking into the gcc/builtins.c file, is that
it is mostly a matter of providing a way for the plugin to register
its plugin specific builtins into some hashtable, and then having the
expand_builtin of gcc/builtins.c end with some code similar to
case BUILT_IN_FREE:
maybe_emit_free_warning (exp);
break;
default:
{
/* the pointer to an expander provided by a plugin. */
rtx (*plugin_builtin_expanderfun) (int, tree, rtx, rtx);
plugin_builtin_expanderfun = find_plugin_expander_for (fcode);
if (plugin_builtin_expanderfun)
{
target = plugin_builtin_expanderfun (fcode, exp, target, subtarget);
if (target)
return target;
}
/* just do library call, if unknown builtin */
break;
}
/* The switch statement above can drop through to cause the function
to be called normally. */
return expand_call (exp, target, ignore);
}
I actually intend to code something more flexible, by having plugin
able to register a builtin expanding function with some client data.
But is the overall idea enough, or did I misunderstood builtins?
Builtins use a fixed code (in DECL_FUNCTION_CODE) and have
a class (BUILT_IN_MD, BUILT_IN_NORMAL, etc.). Thus without
making the code assigning dynamic this won't work.
Could such a small patch be accepted before end of stage 1?
No. We don't want new plugin hooks without an implemented use-case.
Richard.
Cheers
--
Basile STARYNKEVITCH http://starynkevitch.net/Basile/
email: basileatstarynkevitchdotnet mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***
Re: plugin hooks for plugin-provided builtins?
On Tue, Sep 14, 2010 at 09:36, Basile Starynkevitch bas...@starynkevitch.net wrote: I was thinking of adding a new plugin hook for builtins. We need to have a good use case before adding any more plugin hooks. In the case of this proposal, you also need a fixed code and a class for the builtins to work. In general, plugin support is in a state of flux, because we still have not agreed on a general set of APIs that we want to export to plugins. Instead of thinking of new places to add callbacks, I would like to take a couple of steps back and solve the API issue. What should plugins be allowed to see? Currently, we have an arbitrary collection of internal header files that we just expose to 3rd parties. This is sub-optimal. I would like to have a limited set of headers that contain exactly the functions and data structures that we allow plugin code to interact with. I believe that this problem will be largely solved when we make more progress in solving the internal modularity problem. So, I think we still have to solve that problem before we can think about the interfaces we want to export. Plugin hooks should only be added when an actual need arises. Adding hooks for the sake of adding hooks is not a good way to evolve this functionality. Diego.
Re: plugin hooks for plugin-provided builtins?
On Tue, Sep 14, 2010 at 3:36 PM, Basile Starynkevitch bas...@starynkevitch.net wrote: I was thinking of adding a new plugin hook for builtins. Shouldn't there be a final consensus about the existing hooks, and actual users of them, before adding more and more and more plugin hooks? Ciao! Steven
Re: plugin hooks for plugin-provided builtins?
On Tue, Sep 14, 2010 at 10:12:18AM -0400, Diego Novillo wrote:
On Tue, Sep 14, 2010 at 09:36, Basile Starynkevitch
bas...@starynkevitch.net wrote:
I was thinking of adding a new plugin hook for builtins.
Plugin hooks should only be added when an actual need arises. Adding
hooks for the sake of adding hooks is not a good way to evolve this
functionality.
Whatr do you call an actual need? I did gave some interesting use
cases for plugin providing builtins. Are'nt their enough? Should I
elaborate more?
I don't understand at all this actual need thinking. You seems to
suggest that first a real plugin should be developped which shows the
interest of a hook (and developping such a real interesting plugin is
a year's work at least), but this won't be a plugin, it would be an
entire GCC branch and won't be able to work as a plugin into any
future GCC, and then only the idea of adding such and such hook can be
considered.
I don't buy that way of thinking. What is the difference between doing
that and working on one's branch?
My current work aims to translate some Gimple into OpenCL source
code, thus providing GCC with the ability to take advantage of GPU
running their proprietary OpenCL compilers without asking the user to
learn OpenCL. In principle the idea would be to generate some OpenCL
from Gimple, and Jeremie Salvucci experimented the idea of generating
plain C code from Gimple (his code is not fully working, but under the
GCC MELT branch file gcc/melt/xtramelt-c-generator.melt). Generating
C code from Gimple could also help to use GCC [e.g. when compiling
Fortran] with opensource C code analyzers like Frama-C
http://frama-c.com so having plugins which spit Gimple into C is not a
stupid idea (and for sure that functionality will never make into core
GCC, it should stay as a plugin).
I am not in the position of understanding how the GPU processors work
internally in detail and writing a GCC backend for them (this is a job
for American people at AMD/ATI or Nvidia; it cannot be done in Europe
where no GPU are designed and where the internals of GPUs is an
American industry secret. However, OpenCL specification is public and
readable even in France).
However, the project [opengpu.net] currently funding me requires me to
be able to show a plugin [it actually will be a MELT module, not just
a plugin coded in C] which translates some code (using Gimple) in
OpenCL source code feeded to some proprietary (AMD or Nvidia)
implementation.
Very probably, I will need plugin-specific #pragma-s; we already have
hooks for them, and plugin-specific builtin-s (for instance, builtin
translated to code transferring data from the CPU to the GPU in my case).
Given that adding a hook for plugin specific builtin is simple, what
should I do concretely? We also have the dreadful end of stage 1 for
4.6 deadline in end of october! So I try to figure out what few hooks
are missing to be able to work ...
Remember that for geographical political reasons all my GCC work is
more source to source that source to machine oriented. I don't
have the expertise, and I am not legitimate (internally in my CEA LIST
organization at least, and also w.r.t. of funding French government
agencies) to work on anything close to the target processor or silicon
in GCC.
So could you all explain what is an actual need for plugin hooks for
you? I don't understand you, so please take time to explain.
Cheers
--
Basile STARYNKEVITCH http://starynkevitch.net/Basile/
email: basileatstarynkevitchdotnet mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***
Re: plugin hooks for plugin-provided builtins?
On 9/14/10 8:46 AM, Basile Starynkevitch wrote: My current work aims to translate some Gimple into OpenCL source code, thus providing GCC with the ability to take advantage of GPU running their proprietary OpenCL compilers without asking the user to learn OpenCL. My understanding is that Gimple does not have the notion of data parallel operations. For example, in Fortran, array operators are lowered to scalarized form. OpenCL does have these semantics. kernels enqueued as an NDRanges are item-by-item data parallel and there are often not any loops expressed in the kernel itself. And kernels that do have lots of control logic, and use lots of registers, global memory, etc. will tend not to work well on GPUs. Remember that for geographical political reasons all my GCC work is more source to source that source to machine oriented. I don't have the expertise, and I am not legitimate (internally in my CEA LIST organization at least, and also w.r.t. of funding French government agencies) to work on anything close to the target processor or silicon in GCC. It seems to me a source to source compiler should definitely retain high level constructs like array operators, DO ALL, OpenMP directives, etc. Marcus
Re: plugin hooks for plugin-provided builtins?
On Tue, Sep 14, 2010 at 5:39 PM, Marcus Daniels mdani...@lanl.gov wrote: On 9/14/10 8:46 AM, Basile Starynkevitch wrote: My current work aims to translate some Gimple into OpenCL source code, thus providing GCC with the ability to take advantage of GPU running their proprietary OpenCL compilers without asking the user to learn OpenCL. My understanding is that Gimple does not have the notion of data parallel operations. There was a prototype patch with middle-end arrays from Richard Guenther (see http://gcc.gnu.org/ml/gcc-patches/2008-04/msg01123.html for the latest revision I'm aware of). I suppose the usual things (other priorities, funding) stand in the way of completing this work. Richi probably can tell more. Ciao! Steven
Re: plugin hooks for plugin-provided builtins?
On Tue, 14 Sep 2010 09:39:21 -0600
Marcus Daniels mdani...@lanl.gov wrote:
On 9/14/10 8:46 AM, Basile Starynkevitch wrote:
My current work aims to translate some Gimple into OpenCL source
code, thus providing GCC with the ability to take advantage of GPU
running their proprietary OpenCL compilers without asking the user to
learn OpenCL.
My understanding is that Gimple does not have the notion of data
parallel operations.
For example, in Fortran, array operators are lowered to scalarized form.
OpenCL does have these semantics. kernels enqueued as an NDRanges are
item-by-item data parallel and there are often not any loops expressed
in the kernel itself. And kernels that do have lots of control logic,
and use lots of registers, global memory, etc. will tend not to work
well on GPUs.
Remember that for geographical political reasons all my GCC work is
more source to source that source to machine oriented. I don't
have the expertise, and I am not legitimate (internally in my CEA LIST
organization at least, and also w.r.t. of funding French government
agencies) to work on anything close to the target processor or silicon
in GCC.
It seems to me a source to source compiler should definitely retain
high level constructs like array operators, DO ALL, OpenMP directives, etc.
One can use #pragma-s builtin-s attributes for these. This is why I was
trying to push the idea of plugin hooks for builtins.
Marcus
--
Basile STARYNKEVITCH http://starynkevitch.net/Basile/
email: basileatstarynkevitchdotnet mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mine, sont seulement les miennes} ***
Re: plugin hooks for plugin-provided builtins?
On Tue, 14 Sep 2010 16:01:34 +0200
Richard Guenther richard.guent...@gmail.com wrote:
But is the overall idea enough, or did I misunderstood builtins?
Builtins use a fixed code (in DECL_FUNCTION_CODE) and have
a class (BUILT_IN_MD, BUILT_IN_NORMAL, etc.). Thus without
making the code assigning dynamic this won't work.
Could such a small patch be accepted before end of stage 1?
No. We don't want new plugin hooks without an implemented use-case.
What exactly is an implemented use-case? Is it just a test code? Or is
it an entire big 20KLOC plugin demonstrating the usefulness of the hook?
If it is a big plugin requiring a whole year of development effort, how
can that be developped without the hook?
Cheers.
--
Basile STARYNKEVITCH http://starynkevitch.net/Basile/
email: basileatstarynkevitchdotnet mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mine, sont seulement les miennes} ***
Re: plugin hooks for plugin-provided builtins?
On 9/14/10 10:58 AM, Basile Starynkevitch wrote: It seems to me a source to source compiler should definitely retain high level constructs like array operators, DO ALL, OpenMP directives, etc One can use #pragma-s builtin-s attributes for these. This is why I was trying to push the idea of plugin hooks for builtins. In this use case, what is the GCC middle-end *for* if it does not understand data parallel operations? Even GENERIC lacks the notion, right? (We've been working directly from the Fortran parse tree.) Marcus
Re: plugin hooks for plugin-provided builtins?
On Tue, 14 Sep 2010 11:21:51 -0600
Marcus Daniels mdani...@lanl.gov wrote:
On 9/14/10 10:58 AM, Basile Starynkevitch wrote:
It seems to me a source to source compiler should definitely retain
high level constructs like array operators, DO ALL, OpenMP directives, etc
One can use #pragma-s builtin-s attributes for these. This is why I was
trying to push the idea of plugin hooks for builtins.
In this use case, what is the GCC middle-end *for* if it does not
understand data parallel operations? Even GENERIC lacks the notion,
right? (We've been working directly from the Fortran parse tree.)
The GCC middle end use is for me mandatory (since it is contractual). I
am expecting to work on Gimple to OpenCL translation, whatever that
means. The saling point it that starting from GCC gimple gives the
hypothetical enduser all the power of GCC.
I cannot answer precisely more today to that question. However, Albert
Cohen, IIRC, told me that some future improved Graphite-type polyhedric
optimisation might perhaps help.
My builtin plugin hook was from the intuition that to make such
parallel operations, I'll probably need some _Pragma-s builtin-s
(very suitably hidden in macros or whatever).
I'm just trying to figure out what are the features in 4.6 which will
be useful to my work. I know that in a couple of weeks, they are frozen
(since 4.6 is ending stage 1). The gengtype patch series
http://gcc.gnu.org/ml/gcc-patches/2010-09/msg01029.html I am trying to
push with Jeremie Salvucci is also almost needed for MELT (and for any
plugin using GTY). I have no idea if it will make into 4.6, because I
still don't understand which *reviewer* is interested by them.
So my current feeling is that within a two year timeframe, no more
plugins hooks (even small ones like the builtin hooks I was thinking
about) are possible in practice. This is probably bad news for my work,
but I will have to do without! I am apparently the only one thinking
that there are not enough plugin hooks [the popular opinion being that
there are too much of them; I am in the minority against that opinion],
and that they structure wisely the GCC programming interface. Besides
plugins hook have at least a tiny documentation, which most of GCC API
don't have yet.
I really do not understand the philosophy of: work hard for one or two
years making a plugin, and the GCC community will perhaps eventually
consider -and only after the work is completed- adding the tiny hook
which would have saved you 6 months of work. [I am speaking of course
of small hooks which can be incorporated into the trunk by a single
small patch and which have no measurable effect on GCC performance when
not used]
Some nice people privately emailed me that plugin don't catch since
they are disabled by many Linux distributions (however it seems to me
that Ubuntu/Maeverick, i.e. the future 10.10 currently in beta, has GCC
4.5 with plugins enabled and provide even a gcc-4.5-plugin-dev
package, and so does Debian/Sid which is the real source of many Ubuntu
packages), but I cannot understand how plugins could catch if adding a
small hook is nearly impossible. This is a typical chicken egg
situation or catch-22.
Cheers
--
Basile STARYNKEVITCH http://starynkevitch.net/Basile/
email: basileatstarynkevitchdotnet mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mine, sont seulement les miennes} ***
Re: plugin hooks for plugin-provided builtins?
On Tue, Sep 14, 2010 at 15:22, Basile Starynkevitch bas...@starynkevitch.net wrote: I'm just trying to figure out what are the features in 4.6 which will be useful to my work. I know that in a couple of weeks, they are frozen (since 4.6 is ending stage 1). The gengtype patch series No. End of October. http://gcc.gnu.org/ml/gcc-patches/2010-09/msg01029.html I am trying to push with Jeremie Salvucci is also almost needed for MELT (and for any plugin using GTY). I have no idea if it will make into 4.6, because I still don't understand which *reviewer* is interested by them. If you are trying to install a patch, that makes *you* the interested party. Laurynas has been making great review work for the patches you've posted so far. Although I have not reviewed the patches, I have not heard anything against them. Incidentally, this is an issue I would like to address. We need someone interested in maintaining the GC machinery. Any volunteers? Laurynas? I really do not understand the philosophy of: work hard for one or two years making a plugin, and the GCC community will perhaps eventually consider -and only after the work is completed- adding the tiny hook which would have saved you 6 months of work. [I am speaking of course of small hooks which can be incorporated into the trunk by a single small patch and which have no measurable effect on GCC performance when not used] You are blowing things out of proportion. You also belong to this community, so do not make it an us-vs-them problem. You started the thread with a hypothetical use case scenario. We clearly we don't want to add these hooks just because they may be useful. It is true that you are perhaps the biggest user of plugins at a time where everything related to internal APIs is in a state of flux. So, you can imagine what the external APIs look like. You *must* expect challenges here. This work that you are doing, must it be based on plugins for a fundamental reason? Given your interest in plugins, why not do the following: 1- Do your work in a branch, adding all the hooks and facilities that you think are needed. 2- Develop the plugin against that branch and solve the API issue in that branch. 3- Keep the rest of the community in the loop. In particular, I am interested in having a working plugin facility. But I *know* that we cannot get to that without solving our API issues. Want to work with me on that? Diego.
Re: plugin hooks for plugin-provided builtins?
On Tue, 14 Sep 2010 16:40:59 -0400
Diego Novillo dnovi...@google.com wrote:
Incidentally, this is an issue I would like to address. We need
someone interested in maintaining the GC machinery. Any volunteers?
Laurynas?
What do you mean by maintaining the GC machinery?
What is not working in the current GC (I suppose you mean gcc/ggc*.c files).
Is it becoming a GC or gengtype reviewer?
[I will answer to other points in your email later]
Cheers.
--
Basile STARYNKEVITCH http://starynkevitch.net/Basile/
email: basileatstarynkevitchdotnet mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mine, sont seulement les miennes} ***
Re: plugin hooks for plugin-provided builtins?
On Tue, Sep 14, 2010 at 16:48, Basile Starynkevitch bas...@starynkevitch.net wrote: Is it becoming a GC or gengtype reviewer? Yes. Diego.
Re: plugin hooks for plugin-provided builtins?
The GCC middle end use is for me mandatory (since it is contractual). I am expecting to work on Gimple to OpenCL translation, whatever that means. The saling point it that starting from GCC gimple gives the hypothetical enduser all the power of GCC. Given the current limitations of Gimple, another area to focus on could be task parallelism (rather than data parallelism). In that case a language like [Google] Go (via GCC) might make a better talking point than C or Fortran. That is, some very multithreaded application that touches data in an irregular and unpredictable fashion. The OpenCL angle would be to facilitate multiplexing the coroutines on GPU hardware.
Re: plugin hooks for plugin-provided builtins?
2010/9/14 Diego Novillo dnovi...@google.com: Incidentally, this is an issue I would like to address. We need someone interested in maintaining the GC machinery. Any volunteers? Laurynas? Thanks for the suggestion. In fact, I was meaning to apply. But I can see a few things that need to be considered: - Most of my work was done on GC interface, not implementation, so I am not that familiar with ggc-page.c implementation details. - PCH. I tried to stay away from it as much as possible, but I had to debug some PCH stuff then and now, so I guess I am as familiar with it as most of the now-active global maintainers are (which is to say that not very much :) But maybe I'm raising the bar too high for myself if I wait to be familiar with every single line of code before I apply for maintainership, and probably it would be helpful to free up global maintainers a little bit to review more interesting stuff :) So I would like to maintain the GC machinery. How do I proceed? -- Laurynas
