Re: plugin includes for MELT
* Basile STARYNKEVITCH wrote on Thu, Feb 28, 2008 at 06:56:35PM CET: Ralf Wildenhues wrote: * Basile STARYNKEVITCH wrote on Thu, Feb 28, 2008 at 05:39:47PM CET: run-basilys.d: run-basilys.h \ $(CONFIG_H) $(SYSTEM_H) $(TIMEVAR_H) $(TM_H) $(TREE_H) $(GGC_H) \ tree-pass.h basilys.h gt-basilys.h $(CC) -MT run-basilys-deps -MMD $(ALL_CFLAGS) $(ALL_CPPFLAGS) $ The build compiler may not be gcc and may not understand -MT and -MMD. Yes, I know. But how can I avoid that? Use depcomp. Rather than explaining how to do that, it's easier if you wait until Tom puts the depcomp support code in the tree, then use it the same way the other stuff will. Wasn't there a proposal to use depcomp in gcc a while ago? What is depcomp exactly? A compiler wrapper that helps to do dependency computation as a side-effect of compilation. IIRC it first appeared in Automake (though its authors are also GCC developers, I'm not sure who initiated it). mkdir -p $(melt_build_include_dir); \ mkdir -p is not portable, use $(mkinstalldirs). Is $(mkinstalldirs) usable for non-installed directories? Yes. (in other words, it does not do any chown or chmod? Exactly. Cheers, Ralf
Re: plugin includes for MELT
Hello Basile, A mere quick portability review: * Basile STARYNKEVITCH wrote on Thu, Feb 28, 2008 at 05:39:47PM CET: compile-basilys-defs: echo '#generated compile-basilys-defs' $@ echo 'ALL_CFLAGS=' $(ALL_CFLAGS) '' $@ echo 'ALL_CPPFLAGS=' -I$(PWD) $(ALL_CPPFLAGS) '' $@ You should output to a temporary file and mv that to the target. That avoids incomplete (yet seemingly up to date) files when the user aborts with CTRL-\. See other targets that do similar. run-basilys.d: run-basilys.h \ $(CONFIG_H) $(SYSTEM_H) $(TIMEVAR_H) $(TM_H) $(TREE_H) $(GGC_H) \ tree-pass.h basilys.h gt-basilys.h $(CC) -MT run-basilys-deps -MMD $(ALL_CFLAGS) $(ALL_CPPFLAGS) $ The build compiler may not be gcc and may not understand -MT and -MMD. Wasn't there a proposal to use depcomp in gcc a while ago? run-basilys-deps: mkdir -p $(melt_build_include_dir); \ mkdir -p is not portable, use $(mkinstalldirs). for f in $^ ; do \ cf=$(echo $$f | sed -q :^$(srcdir)/config/:$(melt_build_include_dir)/config:); \ Did you mean `echo ... ` here? make will interpret $(echo ...) wrongly, I think this command will do nothing. Also, I think two spaces extra indentation is normal in the GCC Makefile command lines. sed -q is not portable, but sed -n is. Also, your sed script looks very weird. Did you mean to use the 's' command? If yes, you rely on $(srcdir)/config/ not containing any colons nor any other characters special to sed. You might want to use '|' as delimiter instead of ':'. if [ -n $$cf ] ; then \ cp -va $$f $$cf ; \ cp -va is not portable. You can use cp -pR. else \ cp -va $$f $(melt_build_include_dir)/ ; \ Likewise. fi; \ done Cheers, Ralf
Re: plugin includes for MELT
Ralf Wildenhues wrote:
Hello Basile,
A mere quick portability review:
A big thanks
* Basile STARYNKEVITCH wrote on Thu, Feb 28, 2008 at 05:39:47PM CET:
compile-basilys-defs:
echo '#generated compile-basilys-defs' $@
echo 'ALL_CFLAGS=' $(ALL_CFLAGS) '' $@
echo 'ALL_CPPFLAGS=' -I$(PWD) $(ALL_CPPFLAGS) '' $@
You should output to a temporary file and mv that to the target.
That avoids incomplete (yet seemingly up to date) files when the
user aborts with CTRL-\. See other targets that do similar.
run-basilys.d: run-basilys.h \
$(CONFIG_H) $(SYSTEM_H) $(TIMEVAR_H) $(TM_H) $(TREE_H) $(GGC_H) \
tree-pass.h basilys.h gt-basilys.h
$(CC) -MT run-basilys-deps -MMD $(ALL_CFLAGS) $(ALL_CPPFLAGS) $
The build compiler may not be gcc and may not understand -MT and -MMD.
Yes, I know. But how can I avoid that?
Wasn't there a proposal to use depcomp in gcc a while ago?
What is depcomp exactly? Apparently there is a depcomp file at top of
GCC sourcedir. What exactly for? It appears in libcpp/Makefile.in but I
am not sure to understand fully how to use it.
Anyway, I do not plain to run my MELT when host compiler is not GCC...
Actually, it could happen that MELT generated C files uses GCC extension
(they don't use them yet)
run-basilys-deps:
mkdir -p $(melt_build_include_dir); \
mkdir -p is not portable, use $(mkinstalldirs).
Is $(mkinstalldirs) usable for non-installed directories? (in other
words, it does not do any chown or chmod?
for f in $^ ; do \
cf=$(echo $$f | sed -q
:^$(srcdir)/config/:$(melt_build_include_dir)/config:); \
Did you mean `echo ... ` here? make will interpret $(echo ...) wrongly,
Yes!
I think this command will do nothing. Also, I think two spaces extra
indentation is normal in the GCC Makefile command lines.
sed -q is not portable, but sed -n is. Also, your sed script looks
very weird. Did you mean to use the 's' command? If yes, you rely on
$(srcdir)/config/ not containing any colons nor any other characters
special to sed. You might want to use '|' as delimiter instead of ':'.
if [ -n $$cf ] ; then \
cp -va $$f $$cf ; \
cp -va is not portable. You can use cp -pR.
Actually -p is enough; I'm copying files only!
else \
cp -va $$f $(melt_build_include_dir)/ ; \
Likewise.
Thanks for your input. It helps a lot.
Regards.
--
Basile STARYNKEVITCH http://starynkevitch.net/Basile/
email: basileatstarynkevitchdotnet mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***
Re: plugin includes for MELT
Basile STARYNKEVITCH wrote:
Ralf Wildenhues wrote:
Hello Basile,
A mere quick portability review:
A big thanks
Commited into MELT branch rev 132754.
--
Basile STARYNKEVITCH http://starynkevitch.net/Basile/
email: basileatstarynkevitchdotnet mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***
Re: plugin includes for MELT
Ralf == Ralf Wildenhues [EMAIL PROTECTED] writes: [snip] Ralf Wasn't there a proposal to use depcomp in gcc a while ago? Yes. I'm planning to submit the automatic dependencies patch for real in a week or two. I'm working through my backlog of 4.4 patches and that is the last one. Tom
plugin includes for MELT
Hello All,
{sent to the gcc@ mailing list and Bcc- to GlobalGCC partners}
This email is related to the plugin includes question
http://gcc.gnu.org/ml/gcc/2008-02/msg00373.html
http://gcc.gnu.org/ml/gcc/2008-02/msg00376.html within (in particular)
the MELT branch http://gcc.gnu.org/ml/gcc/2008-02/msg00256.html
http://gcc.gnu.org/ml/gcc/2008-02/msg00355.html
http://gcc.gnu.org/wiki/MiddleEndLispTranslator
funded thru the GGCC http://ggcc.info/ ITEA http://itea2.org/ project
My MELT branch [originally I called it basilys] is (currently is not but
should) generate C code during the cc1 execution from some LISP dialect
(either in memory, or in *.bysl files); the generated C code is then
compiled as a plugin into a shared object which is dynamically loaded
thru dlopen (actually thru lt_dlopenext from ltdl.h - the libtool
dynamic loader wrapper). I don't care much about the perennity of the
generated *.c or *.so files; they might need to be regenerated when
bumping the gcc version (from 4.4 to 4.5 for example)
The point is that every MELT generated C file is a plugin to the
middle-end hence depends upon all the middle-end stuff notably tree.h
and many many others.
In its current (sad buggy) state, MELT is not able to work without a
GCC build and source trees (and I am using scripts which I uploaded to
the Wiki page)! This is not acceptable, it should be able to run on a
system without any of them (but of course some additional files,
describing the internals of GCC used by MELT plugins, are required)
Practically, every MELT generated file has exactly one include directive:
#include run-basilys.h
the gcc/run-basilys.h is in the MELT branch and of course include many
other files eg
#include config.h
#include system.h
#include coretypes.h
#include tree.h
#include target.h
#include cgraph.h
#include ipa-prop.h
#include tree-flow.h
#include tree-pass.h
#include flags.h
#include timevar.h
#include diagnostic.h
#include tree-dump.h
#include tree-inline.h
#include compiler-probe.h
#include ppl_c.h
#include ltdl.h
#include basilys.h
So far, my thoughts about all this is:
* some of the *.h are host- specific, but many are target- specific and
I have hard time to understand which files exactly are host- specific
and which one are target- specific
* some of the *.h are generated, hence in the build tree (not in the
source dir from SVN)
* disk space is cheap, but huge -I... include options are messy so I am
thinking of having a single *generated* directory, e.g. in the build
directory include/gcc-melt-plugin-$(host)--$(target) which is later
installed in $(DESTDIR)$(includedir)/gcc-melt-plugin-$(host)--$(target)/
and which contains all the relevant *.h files needed to run-basilys.h
(directly or indirectly included by it)
Does all the above make sense?
My understanding (which is poor regarding the gcc/Makefile.in) is that
sys-include/ is not relevant in this discussion
Regards.
--
Basile STARYNKEVITCH http://starynkevitch.net/Basile/
email: basileatstarynkevitchdotnet mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***
RE: plugin includes for MELT
On 27 February 2008 12:57, Basile STARYNKEVITCH wrote: My MELT branch [originally I called it basilys] is (currently is not but should) generate C code during the cc1 execution The point is that every MELT generated C file is a plugin to the middle-end hence depends upon all the middle-end stuff notably tree.h and many many others. In its current (sad buggy) state, MELT is not able to work without a GCC build and source trees Practically, every MELT generated file has exactly one include directive: #include run-basilys.h the gcc/run-basilys.h is in the MELT branch and of course include many other files eg So far, my thoughts about all this is: * some of the *.h are host- specific, but many are target- specific and I have hard time to understand which files exactly are host- specific and which one are target- specific Does this matter? Any given compiler only has one combination of target and host; are you hoping the plugins will be swappable between differently-configured compilers? * some of the *.h are generated, hence in the build tree (not in the source dir from SVN) * disk space is cheap, but huge -I... include options are messy so I am thinking of having a single *generated* directory, e.g. in the build directory include/gcc-melt-plugin-$(host)--$(target) which is later installed in $(DESTDIR)$(includedir)/gcc-melt-plugin-$(host)--$(target)/ and which contains all the relevant *.h files needed to run-basilys.h (directly or indirectly included by it) It might be easiest to just generate a single pre-preprocessed .i file from run-basilys.h (using -dD) as part of building the compiler, and install it to the libexec include dir (or a 'melt/' subdir thereof, mightn't it? cheers, DaveK -- Can't think of a witty .sigline today
Re: plugin includes for MELT
Hello All,
A big thanks to Dave Korn, who wrote:
On 27 February 2008 12:57, Basile STARYNKEVITCH wrote:
Practically, every MELT generated file has exactly one include directive:
#include run-basilys.h
the gcc/run-basilys.h is in the MELT branch and of course include many
other files eg
So far, my thoughts about all this is:
* some of the *.h are host- specific, but many are target- specific and
I have hard time to understand which files exactly are host- specific
and which one are target- specific
Does this matter? Any given compiler only has one combination of target and
host; are you hoping the plugins will be swappable between
differently-configured compilers?
Of course not, I explained incorrectly. The plugin is heavily dependent
on the actual cc1 program dlopen-ing it. The plugin depend on all the
*.h used in this particular cc1 and useful for MELT.
* some of the *.h are generated, hence in the build tree (not in the
source dir from SVN)
* disk space is cheap, but huge -I... include options are messy so I am
thinking of having a single *generated* directory, e.g. in the build
directory include/gcc-melt-plugin-$(host)--$(target) which is later
installed in $(DESTDIR)$(includedir)/gcc-melt-plugin-$(host)--$(target)/
and which contains all the relevant *.h files needed to run-basilys.h
(directly or indirectly included by it)
It might be easiest to just generate a single pre-preprocessed .i file from
run-basilys.h (using -dD) as part of building the compiler, and install it to
the libexec include dir (or a 'melt/' subdir thereof, mightn't it?
A big thanks for the suggestion! I am a little bit concerned about
scenario like the following (this happens often on Debian/Sid)
the system had some external library, like libc going from 2.7.0 to
2.7.1) or libtldl, upgraded in a minor way (no API change). The library
deep internal stuff (like the /usr/include/bits/*.h files on Debian)
updated a bit. The gcc compiler did not change at all (same version).
I'm trying to understand how other plugin related effort deals with
this. Perhaps nobody really cares, but I tend to believe that any plugin
effort should install the right *.h files outside of the source or build
directories, for plugins...
Of course, I do know that plugin might mean to some people (not to me) a
stability in the GCC in the internal API level. I don't care about this yet.
--
Basile STARYNKEVITCH http://starynkevitch.net/Basile/
email: basileatstarynkevitchdotnet mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***
Re: plugin includes for MELT
Basile STARYNKEVITCH wrote: I'm trying to understand how other plugin related effort deals with this. In an ideal world, you create a plugin API/ABI that is decoupled from any of the internals of the main program and which has its own headers and interface. Plugin authors simply code to that API without involving any gcc headers. As a plugin author, this is ideal as you simply grab this SDK and code your plugin without having to ever touch gcc sources, and your plugin just works with any gcc. This of course requires a ton more work to create and maintain, since you have to a) invent a new API that is flexible enough to do everything that a plugin ever might want to do, and in a way that does not introduce too many target- or architecture-specific details; b) code wrappers in gcc that translate the plugin API into the internal representation; c) maintain those wrappers in the face of changing gcc internals. It's been my observation that whenever plugins are discussed, the majority of gcc maintainers do not want to bear the maintenance and support burden of this level of decoupling, so it's kind of a pie in the sky position I think. Brian
RE: plugin includes for MELT
On 27 February 2008 18:26, Basile STARYNKEVITCH wrote: I'm trying to understand how other plugin related effort deals with this. Perhaps nobody really cares, but I tend to believe that any plugin effort should install the right *.h files outside of the source or build directories, for plugins... Of course, I do know that plugin might mean to some people (not to me) a stability in the GCC in the internal API level. I don't care about this yet. I think you already have the answer. Usually public plugin-APIs have to go to great lengths to make the interface completely stable, then the plugins can be distributed as object files. On the other hand, you aren't planning on offering a stable API (and that would be quite difficult considering how quickly and significantly gcc internals change from version to version), so there is no advantage in distributing the headers separately from the specific gcc that they were used to build. So, since you are planning to compile the plugin during cc1 execution anyway, why not just say that - plugins are distributed as source - the compiler keeps the gcc-private headers in its private libexec include subdir, thus automatically making the correct headers go along with the correct version+host+target compiler - when cc1 runs, it compiles any plugins needed (possibly caching the compiled objects in another libexec subdir) and dlopens them. Then, as you gain experience with MELT and it becomes clearer to you which parts of gcc's internal api are stable enough to be exposed, you can gradually build up to a public, SDK-like header file, which exposes only those details of the internals that you are either confident enough will not change, or that you feel would not be too difficult to provide a backward-compatibility shim layer for if they do change. cheers, DaveK -- Can't think of a witty .sigline today
Re: plugin includes for MELT
Hello All,
Dave Korn wrote:
On 27 February 2008 18:26, Basile STARYNKEVITCH wrote:
So, since you are planning to compile the plugin during cc1 execution
anyway, why not just say that
- plugins are distributed as source
Yes, exactly. And to be more precise, all MELT plugin C code is
generated (from some MELT Lisp code). This leaves the GCC API dependency
to the generating stuff.
Of course I have to deal with the dependency of the MELT Lisp code on
the GCC internals, but about this I have some ideas about (basically
stay at the highest level possible).
- the compiler keeps the gcc-private headers in its private libexec include
subdir, thus automatically making the correct headers go along with the
correct version+host+target compiler
I think it should be (in gcc/Makefile.in parlance)
$(DESTDIR)$(libexecsubdir)/melt-private-include/ and I should have some
Makefile.in trick to copy the relevant *.h there perhaps thru a
install-melt-includes target
- when cc1 runs, it compiles any plugins needed (possibly caching the
compiled objects in another libexec subdir) and dlopens them.
Then, as you gain experience with MELT and it becomes clearer to you which
parts of gcc's internal api are stable enough to be exposed, you can gradually
build up to a public, SDK-like header file, which exposes only those details
of the internals that you are either confident enough will not change, or that
you feel would not be too difficult to provide a backward-compatibility shim
layer for if they do change.
Yes exactly. But this SDK-like header does not really exist. all the
coding in MELT is done in Lisp (and some of this Lisp code would change
as GCC evolves).
Actually, my current concern is just the part of MELT which translates
LISP into C (the warm-basilys.bysl file, coded in my Lisp dialect, and
[almost] able to translate itself into C), and as you might guess this
part does not depend much on all the (unstable) GCC API, just on a more
stable subset (mostly ggc.h basilys.h and the fact that tree [and some
very few others GCC types] are opaque pointers).
Actually the whole MELT idea is to separate as much as possible stuff
heavily depending on GCC unstable API to those depending on more stable
stuff (like ggc.h basilys.h):
the MELT translator warm-basilys.bysl and the MELT related runtime
basilys.c does not depend much on GCC unstable API. For instance, it
does know about tree typename, but does not care about the details of
tree.h. Regarding tree-s (in GCC sense) it only depends upon the fact
that tree is a GGC-ed pointer.
more specific stuff are other files (mostly to be written still)
Dave a big thanks for your insights help. It was unvaluable!
--
Basile STARYNKEVITCH http://starynkevitch.net/Basile/
email: basileatstarynkevitchdotnet mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***
Re: plugin includes for MELT
Hello All,
Basile STARYNKEVITCH wrote:
I think it should be (in gcc/Makefile.in parlance)
$(DESTDIR)$(libexecsubdir)/melt-private-include/ and I should have some
Makefile.in trick to copy the relevant *.h there perhaps thru a
install-melt-includes target
The one detail I don't understand yet is the link between the -B option
to gcc and this $(DESTDIR)$(libexecsubdir)/ are they somehow equal (or
is the -B value some initial prefix of $(DESTDIR)$(libexecsubdir) ?
Regards.
--
Basile STARYNKEVITCH http://starynkevitch.net/Basile/
email: basileatstarynkevitchdotnet mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***
