Re: GCC + libJIT instead of LLVM

[Kirill Kononenko]

And no this is not a 1st April joke :-)



Thanks,
Kirill

2009/4/1 Kirill Kononenko :
> Hello Dear GCC Developers,
>
>
>
> I would like to ask your opinion about possibility for integration of
> the libJIT Just-In-Time compilation library and GCC. For example, the
> same way as libffi is integrated within gcc source tree. It seems to
> me that LLVM solves many goals that are already complete and solved in
> GCC. So I think libJIT potentially is more useful for GCC and software
> developers.
>
>
> What is your opinion about this idea? How this should be done and what
> improvements could be made to both libJIT and GCC for this kind of
> integration?
>
>
> I was also very surprised to read this interesting discussion by Rhys
> Weatherley (libJIT) and Chris Lattner (LLVM) (libJIT vs LLVM):
> http://lists.gnu.org/archive/html/dotgnu-libjit/2004-05/index.html
>
>
> It talks more about differences in design and goals between libJIT and LLVM.
>
>
> Please let me know your ideas and thoughts. No flaming please. I think
> everyone benefits if we can gather enough thoughts about JITing in
> GCC.
>
>
> I also would like to point to my personal project about improvement of
> libJIT: http://code.google.com/p/libjit-linear-scan-register-allocator/
> . It studies how to make efficient code generation for Common
> Intermediate Language and JIT compilers, and implements a full linear
> scan register allocator and bin packing register allocator within
> libJIT. It also has support of XMM registers for floating point
> operations. The experimental target is the Microsoft Common
> Intermediate Language. There is various data-flow and control-flow
> analysis, including various liveness analysis(fast and full),
> dead-code elimination etc with various optimization level. For
> example, this code generator gives the following benchmark on my pc
> for pnetmark at the moment, with Portable.NET Just-In-Time compiler
> 0.8.0:
>
> LibJIT Linear Scan Register Allocator
> Current  libJIT 0.1.2
>
> Sieve      20439      17499
> Loop       28311      24976
> Logic      57311      55647
> String     16586      16651
> Float       2642       1940
> Method   32426      30401
>
> PnetMark(Average Score) 19005    16970
>
> The average improvement is 12% or so in pnetmark comparing to a global
> register allocator in libJIT which uses number of usage of a variable
> as an euristic.  With dead-code analysis enabled it jumps another 12%
> or so. For example, the logic benchmark jumps over 70%. Other
> optimizations enabled can give more improvement. It was remarkably
> easy to add dead-code elimination in libJIT for example I spent less
> than a week for this. But I am rather interested in the code
> generation and linear scan register allocation at the moment and
> Common Intermediate Language. I think GCC could benefit a lot if an
> integration of both libJIT and GCC could be considered, and there was
> cooperation about this.
>
>
>
> Thanks,
> Kirill
>

Re: GCC + libJIT instead of LLVM

[Andrew Haley]

Kirill Kononenko wrote:

> I would like to ask your opinion about possibility for integration of
> the libJIT Just-In-Time compilation library and GCC. For example, the
> same way as libffi is integrated within gcc source tree. It seems to
> me that LLVM solves many goals that are already complete and solved in
> GCC. So I think libJIT potentially is more useful for GCC and software
> developers.

Useful for what?  I think you have to tell us how this will improve the
experience of gcc users .

Andrew.

Re: GCC + libJIT instead of LLVM

[Kirill Kononenko]

More useful in implementation of Just-In-Time compilation in Virtual
Machine runtimes. For example, for Microsoft Common Intermediate
Language (.NET).


Thanks,
Kirill

2009/4/1 Andrew Haley :
> Kirill Kononenko wrote:
>
>> I would like to ask your opinion about possibility for integration of
>> the libJIT Just-In-Time compilation library and GCC. For example, the
>> same way as libffi is integrated within gcc source tree. It seems to
>> me that LLVM solves many goals that are already complete and solved in
>> GCC. So I think libJIT potentially is more useful for GCC and software
>> developers.
>
> Useful for what?  I think you have to tell us how this will improve the
> experience of gcc users .
>
> Andrew.
>

Re: GCC + libJIT instead of LLVM

[Andrew Haley]

> 2009/4/1 Andrew Haley :
>> Kirill Kononenko wrote:
>>
>>> I would like to ask your opinion about possibility for integration of
>>> the libJIT Just-In-Time compilation library and GCC. For example, the
>>> same way as libffi is integrated within gcc source tree. It seems to
>>> me that LLVM solves many goals that are already complete and solved in
>>> GCC. So I think libJIT potentially is more useful for GCC and software
>>> developers.
>> Useful for what?  I think you have to tell us how this will improve the
>> experience of gcc users .

Kirill Kononenko wrote:
> More useful in implementation of Just-In-Time compilation in Virtual
> Machine runtimes. For example, for Microsoft Common Intermediate
> Language (.NET).

We already know what a JIT does, thank you.  I think you have to tell us how
having libjit integrated in gcc will improve the experience of gcc users.
As opposed to them, say, using libjit as a library separate from gcc.

Andrew.

Re: GCC + libJIT instead of LLVM

[Kirill Kononenko]

>> 2009/4/1 Andrew Haley :
>>> Kirill Kononenko wrote:
>>>
 I would like to ask your opinion about possibility for integration of
 the libJIT Just-In-Time compilation library and GCC. For example, the
 same way as libffi is integrated within gcc source tree. It seems to
 me that LLVM solves many goals that are already complete and solved in
 GCC. So I think libJIT potentially is more useful for GCC and software
 developers.
>>> Useful for what?  I think you have to tell us how this will improve the
>>> experience of gcc users .
>
> Kirill Kononenko wrote:
>> More useful in implementation of Just-In-Time compilation in Virtual
>> Machine runtimes. For example, for Microsoft Common Intermediate
>> Language (.NET).
>
> We already know what a JIT does, thank you.  I think you have to tell us how

Please could you specify more precisely the 'we' here?

I really don't understand what you exactly mean here.

> having libjit integrated in gcc will improve the experience of gcc users.
> As opposed to them, say, using libjit as a library separate from gcc.
>

Just read my first post.


Thanks,
Kirill

Re: GCC + libJIT instead of LLVM

[Andrew Haley]

Kirill Kononenko wrote:
>>> 2009/4/1 Andrew Haley :
 Kirill Kononenko wrote:

> I would like to ask your opinion about possibility for integration of
> the libJIT Just-In-Time compilation library and GCC. For example, the
> same way as libffi is integrated within gcc source tree. It seems to
> me that LLVM solves many goals that are already complete and solved in
> GCC. So I think libJIT potentially is more useful for GCC and software
> developers.
 Useful for what?  I think you have to tell us how this will improve the
 experience of gcc users .
>> Kirill Kononenko wrote:
>>> More useful in implementation of Just-In-Time compilation in Virtual
>>> Machine runtimes. For example, for Microsoft Common Intermediate
>>> Language (.NET).
>> We already know what a JIT does, thank you.  I think you have to tell us how
> 
> Please could you specify more precisely the 'we' here?

Everybody who reads the gcc list knows what a JIT does.

> I really don't understand what you exactly mean here.
> 
>> having libjit integrated in gcc will improve the experience of gcc users.
>> As opposed to them, say, using libjit as a library separate from gcc.
> 
> Just read my first post.

I have, and you gave no reasons why this might help anyone.
I'm beginning to think this is an April Fool "joke".

Andrew.

Re: GCC + libJIT instead of LLVM

[Dave Korn]

Kirill Kononenko wrote:
>>> 2009/4/1 Andrew Haley:
 Kirill Kononenko wrote:

> I would like to ask your opinion about possibility for integration of
> the libJIT Just-In-Time compilation library and GCC. For example, the
> same way as libffi is integrated within gcc source tree. It seems to
> me that LLVM solves many goals that are already complete and solved in
> GCC. So I think libJIT potentially is more useful for GCC and software
> developers.
 Useful for what?  I think you have to tell us how this will improve the
 experience of gcc users .
>> Kirill Kononenko wrote:
>>> More useful in implementation of Just-In-Time compilation in Virtual
>>> Machine runtimes. For example, for Microsoft Common Intermediate
>>> Language (.NET).
>> We already know what a JIT does, thank you.  I think you have to tell us how
> 
> Please could you specify more precisely the 'we' here?

  I think he means "Every single reader of the GCC mailing list, or as close
as makes no difference".

> I really don't understand what you exactly mean here.
> 
>> having libjit integrated in gcc will improve the experience of gcc users.
>> As opposed to them, say, using libjit as a library separate from gcc.
>>
> 
> Just read my first post.

  Hi, I also read your first post.  It says

> It seems to
> me that LLVM solves many goals that are already complete and solved in
> GCC. So I think libJIT potentially is more useful for GCC and software
> developers.

but you don't say what libjit would be more useful than, or how this overlap
between "solved goals" between gcc and llvm implies that.  Do you simply mean
that, because llvm and gcc both cover similar areas, and because libjit is
useful for llvm, it must also be useful for gcc?  Could you list some of these
goals and explain how they relate to JIT?

  You also wrote

> I think GCC could benefit a lot if an
> integration of both libJIT and GCC could be considered

but you don't explicitly describe any of the benefits you think will arise.
Can you enlarge on exactly what effect it would have if libjit was integrated
into GCC (presumably in the runtime support libraries somewhere) rather than
just existing as a separate installation?  I can't see how it would perform
any differently.

cheers,
  DaveK

Re: GCC + libJIT instead of LLVM

[Andrew Haley]

Basile STARYNKEVITCH wrote:

> The second issue (which perhaps Kirill did not thought of) would be to
> accelerate some internal optimisations of GCC by using JIT-code
> generation techniques within the compiler itself. There are several
> occasions within GCC where complex internal processing happens, and one
> could imagine to "partially evaluate" them (w.r.t. to the compiled
> source program) by generating some code specifically tuned to that
> processing. BTW, the MELT branch was designed with such stuff in mind,
> and indeed can generate some code (currently, it generates C code, run
> the host compiler on it, dlopen it, and use it; in principle I could
> have used libjit instead of forking a compilation process from inside cc1).

Yes, that's true.  But it doesn't in any sense require libjit to be integrated
with gcc to achieve this: the jit could just be called as an external library.

> I still don't understand what Kiril is thinking of exactly. In contrast
> to Andrew, I don't believe it is an April Fool's joke, but perhaps a
> language issue: both Kiril & me Basile are not native English speakers,
> and we may have difficulties in finding the right words & express
> ourself fluently in English.

For what it's worth, I didn't really think this is April Fool's joke; I was
just trying to provoke Kirill into explaining his purpose.  I seem to have
failed to do that.

Andrew.

Re: GCC + libJIT instead of LLVM

[Basile STARYNKEVITCH]

Kirill Kononenko wrote (citing me Basile)


The second issue (which perhaps Kirill did not thought of) would be to
accelerate some internal optimisations of GCC by using JIT-code generation
techniques within the compiler itself. There are several occasions within
GCC where complex internal processing happens, and one could imagine to
"partially evaluate" them (w.r.t. to the compiled source program) by
generating some code specifically tuned to that processing. BTW, the MELT
branch was designed with such stuff in mind, and indeed can generate some
code (currently, it generates C code, run the host compiler on it, dlopen
it, and use it; in principle I could have used libjit instead of forking a
compilation process from inside cc1).




Yes, this is exactly one kind of opportunity I see here. Basile is
absolutely correct here.

  


The standard answer to that would be : find an interesting use case (of 
JIT compilation inside GCC, eg inside some complex optimisation pass) 
and implement it in your branch.


I invested too much effort in C code generation in MELT, so I won't 
switch quickly to libjit, even if it could help a lot.


However, once the plugins machinery comes into the trunk, nothing 
prevent Kiril or others to experiment a plugin which uses libjit 
internally. And you (Kiril or others) could even start a branch right 
now (provided you did comply with the legal requirements of 
http://gcc.gnu.org/contribute.html ie the FSF copyright transfer).


--
Basile STARYNKEVITCH http://starynkevitch.net/Basile/
email: basilestarynkevitchnet mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***

Re: GCC + libJIT instead of LLVM

[Kirill Kononenko]

2009/4/1 Dave Korn :
> Kirill Kononenko wrote:
 2009/4/1 Andrew Haley:
> Kirill Kononenko wrote:
>
>> I would like to ask your opinion about possibility for integration of
>> the libJIT Just-In-Time compilation library and GCC. For example, the
>> same way as libffi is integrated within gcc source tree. It seems to
>> me that LLVM solves many goals that are already complete and solved in
>> GCC. So I think libJIT potentially is more useful for GCC and software
>> developers.
> Useful for what?  I think you have to tell us how this will improve the
> experience of gcc users .
>>> Kirill Kononenko wrote:
 More useful in implementation of Just-In-Time compilation in Virtual
 Machine runtimes. For example, for Microsoft Common Intermediate
 Language (.NET).
>>> We already know what a JIT does, thank you.  I think you have to tell us how
>>
>> Please could you specify more precisely the 'we' here?
>
>  I think he means "Every single reader of the GCC mailing list, or as close
> as makes no difference".
>
>> I really don't understand what you exactly mean here.
>>
>>> having libjit integrated in gcc will improve the experience of gcc users.
>>> As opposed to them, say, using libjit as a library separate from gcc.
>>>
>>
>> Just read my first post.
>
>  Hi, I also read your first post.  It says
>
>> It seems to
>> me that LLVM solves many goals that are already complete and solved in
>> GCC. So I think libJIT potentially is more useful for GCC and software
>> developers.
>
> but you don't say what libjit would be more useful than, or how this overlap
> between "solved goals" between gcc and llvm implies that.  Do you simply mean
> that, because llvm and gcc both cover similar areas, and because libjit is
> useful for llvm, it must also be useful for gcc?  Could you list some of these
> goals and explain how they relate to JIT?
>
>  You also wrote
>
>> I think GCC could benefit a lot if an
>> integration of both libJIT and GCC could be considered
>
> but you don't explicitly describe any of the benefits you think will arise.
> Can you enlarge on exactly what effect it would have if libjit was integrated
> into GCC (presumably in the runtime support libraries somewhere) rather than
> just existing as a separate installation?  I can't see how it would perform
> any differently.

LLVM is an overkill for JIT compilation. I think the tasks which LLVM
solves are already solved within GCC transformations, or can be
integrated very easily with libJIT. LibJIT is also much easier in
usage, for ordinary developers. So what I see here, LLVM is rather a
competitor to GCC, considering which tasks it solves. And complexity
of code rises inproportionally to the task it is used for. While both
could be potentionaly useful for their area of expertise. I think an
GCC+libJIT solves 99% of areas of LLVM usage.


Thanks,
Kirill

Re: GCC + libJIT instead of LLVM

[Kirill Kononenko]

>> The second issue (which perhaps Kirill did not thought of) would be to
>> accelerate some internal optimisations of GCC by using JIT-code
>> generation techniques within the compiler itself. There are several
>> occasions within GCC where complex internal processing happens, and one
>> could imagine to "partially evaluate" them (w.r.t. to the compiled
>> source program) by generating some code specifically tuned to that
>> processing. BTW, the MELT branch was designed with such stuff in mind,
>> and indeed can generate some code (currently, it generates C code, run
>> the host compiler on it, dlopen it, and use it; in principle I could
>> have used libjit instead of forking a compilation process from inside cc1).
>
> Yes, that's true.  But it doesn't in any sense require libjit to be integrated
> with gcc to achieve this: the jit could just be called as an external library.
>
>> I still don't understand what Kiril is thinking of exactly. In contrast
>> to Andrew, I don't believe it is an April Fool's joke, but perhaps a
>> language issue: both Kiril & me Basile are not native English speakers,
>> and we may have difficulties in finding the right words & express
>> ourself fluently in English.
>
> For what it's worth, I didn't really think this is April Fool's joke; I was

Everyone who read that, might have thought you really meant that.

> just trying to provoke Kirill into explaining his purpose.  I seem to have
> failed to do that.
>

My explanations seem to have also failed to explain you.
Unfortunately, one really needs have some back group with both
Just-In-Time compilers,Virtual Machines, and Common Intermediate
Language to understand this area. I understand that it is not your
area of expertise, so it is not an issue for me.


Thanks,
Kirill

Re: GCC + libJIT instead of LLVM

[Kirill Kononenko]

> However, I see several interesting issues raised here:
>
> the first is to [re-]use GCC for just in time compilation, for instance to
> JIT-compile CLI or JVM bytecode into machine code, or even C or some
> specialized gimple-like representation into machine code, or CLISP into
> machine code, all this using most of existing code in GCC (certainly the
> middle-end and a back-end). This might be doable, is certainly interesting
> (I heard even that some major GCC contributor[s] wanted that to happen), but
> remains difficult (because GCC was not designed for that). Note that libjit
> per se is not useful in that case, but GCC would be made as a server (or a
> library) which happens to deliver some of libjit or LLVM functionalities
> (the ability to generate code).
>
> The second issue (which perhaps Kirill did not thought of) would be to
> accelerate some internal optimisations of GCC by using JIT-code generation
> techniques within the compiler itself. There are several occasions within
> GCC where complex internal processing happens, and one could imagine to
> "partially evaluate" them (w.r.t. to the compiled source program) by
> generating some code specifically tuned to that processing. BTW, the MELT
> branch was designed with such stuff in mind, and indeed can generate some
> code (currently, it generates C code, run the host compiler on it, dlopen
> it, and use it; in principle I could have used libjit instead of forking a
> compilation process from inside cc1).


Yes, this is exactly one kind of opportunity I see here. Basile is
absolutely correct here.


>
> I still don't understand what Kiril is thinking of exactly. In contrast to
> Andrew, I don't believe it is an April Fool's joke, but perhaps a language
> issue: both Kiril & me Basile are not native English speakers, and we may
> have difficulties in finding the right words & express ourself fluently in
> English. If it is the case please forgive us (or just try to interact on a
> french or russian mailing list!).
>


Thanks,
Kirill

Re: GCC + libJIT instead of LLVM

[Basile STARYNKEVITCH]

Andrew Haley wrote:

Useful for what?  I think you have to tell us how this will improve the
experience of gcc users .
  


Kirill Kononenko wrote:
  

More useful in implementation of Just-In-Time compilation in Virtual
Machine runtimes. For example, for Microsoft Common Intermediate
Language (.NET).



We already know what a JIT does, thank you.  I think you have to tell us how
having libjit integrated in gcc will improve the experience of gcc users.
As opposed to them, say, using libjit as a library separate from gcc.
  


I still do not understand exactly what Kirill Kononenko is thinking about.

However, I see several interesting issues raised here:

the first is to [re-]use GCC for just in time compilation, for instance 
to JIT-compile CLI or JVM bytecode into machine code, or even C or some 
specialized gimple-like representation into machine code, or CLISP into 
machine code, all this using most of existing code in GCC (certainly the 
middle-end and a back-end). This might be doable, is certainly 
interesting (I heard even that some major GCC contributor[s] wanted that 
to happen), but remains difficult (because GCC was not designed for 
that). Note that libjit per se is not useful in that case, but GCC would 
be made as a server (or a library) which happens to deliver some of 
libjit or LLVM functionalities (the ability to generate code).


The second issue (which perhaps Kirill did not thought of) would be to 
accelerate some internal optimisations of GCC by using JIT-code 
generation techniques within the compiler itself. There are several 
occasions within GCC where complex internal processing happens, and one 
could imagine to "partially evaluate" them (w.r.t. to the compiled 
source program) by generating some code specifically tuned to that 
processing. BTW, the MELT branch was designed with such stuff in mind, 
and indeed can generate some code (currently, it generates C code, run 
the host compiler on it, dlopen it, and use it; in principle I could 
have used libjit instead of forking a compilation process from inside cc1).


I still don't understand what Kiril is thinking of exactly. In contrast 
to Andrew, I don't believe it is an April Fool's joke, but perhaps a 
language issue: both Kiril & me Basile are not native English speakers, 
and we may have difficulties in finding the right words & express 
ourself fluently in English. If it is the case please forgive us (or 
just try to interact on a french or russian mailing list!).


Regards.

--
Basile STARYNKEVITCH http://starynkevitch.net/Basile/
email: basilestarynkevitchnet mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***

Re: GCC + libJIT instead of LLVM

[Kirill Kononenko]

2009/4/1 Andrew Haley :
> Kirill Kononenko wrote:
 2009/4/1 Andrew Haley :
> Kirill Kononenko wrote:
>
>> I would like to ask your opinion about possibility for integration of
>> the libJIT Just-In-Time compilation library and GCC. For example, the
>> same way as libffi is integrated within gcc source tree. It seems to
>> me that LLVM solves many goals that are already complete and solved in
>> GCC. So I think libJIT potentially is more useful for GCC and software
>> developers.
> Useful for what?  I think you have to tell us how this will improve the
> experience of gcc users .
>>> Kirill Kononenko wrote:
 More useful in implementation of Just-In-Time compilation in Virtual
 Machine runtimes. For example, for Microsoft Common Intermediate
 Language (.NET).
>>> We already know what a JIT does, thank you.  I think you have to tell us how
>>
>> Please could you specify more precisely the 'we' here?
>
> Everybody who reads the gcc list knows what a JIT does.
>
>> I really don't understand what you exactly mean here.
>>
>>> having libjit integrated in gcc will improve the experience of gcc users.
>>> As opposed to them, say, using libjit as a library separate from gcc.
>>
>> Just read my first post.
>
> I have, and you gave no reasons why this might help anyone.
> I'm beginning to think this is an April Fool "joke".

Please let me know who are the 'we' in your previous post?

I' m beginning to think that you have a personal interest here. I
think you either meant there "I" or your company in which you work
full-time, and get your paycheck. I am correct to suppose this? I am
beginning to think that this is an April Fool "joke" of your part.


Thanks,
Kirill

Re: GCC + libJIT instead of LLVM

[Kirill Kononenko]

2009/4/1 Basile STARYNKEVITCH :
>>>
>>> The second issue (which perhaps Kirill did not thought of) would be to
>>> accelerate some internal optimisations of GCC by using JIT-code
>>> generation
>>> techniques within the compiler itself. There are several occasions within
>>> GCC where complex internal processing happens, and one could imagine to
>>> "partially evaluate" them (w.r.t. to the compiled source program) by
>>> generating some code specifically tuned to that processing. BTW, the MELT
>>> branch was designed with such stuff in mind, and indeed can generate some
>>> code (currently, it generates C code, run the host compiler on it, dlopen
>>> it, and use it; in principle I could have used libjit instead of forking
>>> a
>>> compilation process from inside cc1).
>>>
>>
>>
>> Yes, this is exactly one kind of opportunity I see here. Basile is
>> absolutely correct here.
>>
>>
>
> The standard answer to that would be : find an interesting use case (of JIT
> compilation inside GCC, eg inside some complex optimisation pass) and
> implement it in your branch.
>
> I invested too much effort in C code generation in MELT, so I won't switch
> quickly to libjit, even if it could help a lot.
>
> However, once the plugins machinery comes into the trunk, nothing prevent
> Kiril or others to experiment a plugin which uses libjit internally. And you
> (Kiril or others) could even start a branch right now (provided you did
> comply with the legal requirements of http://gcc.gnu.org/contribute.html ie
> the FSF copyright transfer).
>

Thank you a lot, Basile, for your support.



Thanks,
Kirill

Re: GCC + libJIT instead of LLVM

[Dave Korn]

Kirill Kononenko wrote:

> LLVM is an overkill for JIT compilation. I think the tasks which LLVM
> solves are already solved within GCC transformations, or can be
> integrated very easily with libJIT. LibJIT is also much easier in
> usage, for ordinary developers. So what I see here, LLVM is rather a
> competitor to GCC, considering which tasks it solves. And complexity
> of code rises inproportionally to the task it is used for. While both
> could be potentionaly useful for their area of expertise. I think an
> GCC+libJIT solves 99% of areas of LLVM usage.

  From this, it sounds to me as if the question you should be asking is not so
much "Could integrating libjit be useful for GCC", but more like "Could
integrating GCC be useful for libjit"!

cheers,
  DaveK

Re: GCC + libJIT instead of LLVM

[Dave Korn]

Kirill and Andrew wrote:

>> "April Fool's joke"
> "not your area of expertise"

  Maybe it would be for the best if you two started over, before this turns 
sour.

cheers,
  DaveK

Re: GCC + libJIT instead of LLVM

[Kirill Kononenko]

2009/4/1 Dave Korn :
>> LLVM is an overkill for JIT compilation. I think the tasks which LLVM
>> solves are already solved within GCC transformations, or can be
>> integrated very easily with libJIT. LibJIT is also much easier in
>> usage, for ordinary developers. So what I see here, LLVM is rather a
>> competitor to GCC, considering which tasks it solves. And complexity
>> of code rises inproportionally to the task it is used for. While both
>> could be potentionaly useful for their area of expertise. I think an
>> GCC+libJIT solves 99% of areas of LLVM usage.
>
>  From this, it sounds to me as if the question you should be asking is not so
> much "Could integrating libjit be useful for GCC", but more like "Could
> integrating GCC be useful for libjit"!

LibJIT is a not a competitor to LLVM. Three guys on internet don't
compete with a multi-billion corporation like Apple (?) But you are
correct. LibJIT is only a tool for JITing, and it already does this
job much better than any other tool. LibJIT can benefit even more from
integration with GCC internals.


Thanks,
Kirill

Re: GCC + libJIT instead of LLVM

[Andrew Haley]

Dave Korn wrote:
> Kirill and Andrew wrote:
> 
>>> "April Fool's joke"
>> "not your area of expertise"
> 
>   Maybe it would be for the best if you two started over, before this turns 
> sour.

I'm out of here already!  All I can say is that I hope my boss never finds
out that virtual machines and JITs are not my area of expertise.  :-)

Andrew.

Re: GCC + libJIT instead of LLVM

[Kirill Kononenko]

>> Kirill and Andrew wrote:
>>
 "April Fool's joke"
>>> "not your area of expertise"
>>
>>   Maybe it would be for the best if you two started over, before this turns 
>> sour.
>
> I'm out of here already!  All I can say is that I hope my boss never finds
> out that virtual machines and JITs are not my area of expertise.  :-)

You are lucky :-) Most luckely your company won't assume that it has
spend so much money in LLVM investment without results.

You can just try libJIT if you want to compare. As I understand it is
very easy to use or convert something done for one in another.


Thanks,
Kirill

Re: GCC + libJIT instead of LLVM

[David Edelsohn]

On Wed, Apr 1, 2009 at 8:14 AM, Kirill Kononenko
 wrote:

> My explanations seem to have also failed to explain you.
> Unfortunately, one really needs have some back group with both
> Just-In-Time compilers,Virtual Machines, and Common Intermediate
> Language to understand this area. I understand that it is not your
> area of expertise, so it is not an issue for me.

I think a JIT associated with GCC would be great.  Not everything is
static compilation.  "Split Compilation" combining static compilation
for complicated analysis along with a "Managed Runtime" for
execution and runtime transformations is becoming more important.

libJIT could help with the runtime for a GCC implementation of OpenCL.
It could help with an environment for Java and CIL.

A JIT associated with GCC would allow coordination between GCC
bytecode generation and the JIT.  It also can allow the JIT cache
to be pre-warmed by GCC static code and profiling information.

There are many exciting opportunities.

David

Re: GCC + libJIT instead of LLVM

[Kirill Kononenko]

>
>> My explanations seem to have also failed to explain you.
>> Unfortunately, one really needs have some back group with both
>> Just-In-Time compilers,Virtual Machines, and Common Intermediate
>> Language to understand this area. I understand that it is not your
>> area of expertise, so it is not an issue for me.
>
> I think a JIT associated with GCC would be great.  Not everything is
> static compilation.  "Split Compilation" combining static compilation
> for complicated analysis along with a "Managed Runtime" for
> execution and runtime transformations is becoming more important.
>
> libJIT could help with the runtime for a GCC implementation of OpenCL.
> It could help with an environment for Java and CIL.
>
> A JIT associated with GCC would allow coordination between GCC
> bytecode generation and the JIT.  It also can allow the JIT cache
> to be pre-warmed by GCC static code and profiling information.
>
> There are many exciting opportunities.


Thank you David this excellent explanation. I also think that there
are many exciting opportunities.



Thanks,
Kirill

Re: GCC + libJIT instead of LLVM

[Kirill Kononenko]

>>> My explanations seem to have also failed to explain you.
>>> Unfortunately, one really needs have some back group with both
>>> Just-In-Time compilers,Virtual Machines, and Common Intermediate
>>> Language to understand this area. I understand that it is not your
>>> area of expertise, so it is not an issue for me.
>>
>> I think a JIT associated with GCC would be great.  Not everything is
>> static compilation.  "Split Compilation" combining static compilation
>> for complicated analysis along with a "Managed Runtime" for
>> execution and runtime transformations is becoming more important.
>>
>> libJIT could help with the runtime for a GCC implementation of OpenCL.
>> It could help with an environment for Java and CIL.
>>
>> A JIT associated with GCC would allow coordination between GCC
>> bytecode generation and the JIT.  It also can allow the JIT cache
>> to be pre-warmed by GCC static code and profiling information.
>>
>> There are many exciting opportunities.
>
>

Thank you David for this excellent explanation. I also think that
there are many exciting opportunities.


Thanks,
Kirill

Re: GCC + libJIT instead of LLVM

[Chris Lattner]

On Apr 1, 2009, at 5:09 AM, Dave Korn wrote:

It seems to
me that LLVM solves many goals that are already complete and solved  
in
GCC. So I think libJIT potentially is more useful for GCC and  
software

developers.


but you don't say what libjit would be more useful than, or how this  
overlap
between "solved goals" between gcc and llvm implies that.  Do you  
simply mean
that, because llvm and gcc both cover similar areas, and because  
libjit is
useful for llvm, it must also be useful for gcc?  Could you list  
some of these

goals and explain how they relate to JIT?


Hi Dave,

I don't mean to invade into a pretty amusing thread, but I wanted to  
clarify one important thing: litjit and LLVM have nothing to do with  
each other.


libjit and LLVM have very different design points.  libjit is a bit  
more than a glorified macro assembler with some late "optimizations",  
whereas LLVM is a real compiler that happens to have the ability to  
emit machine code to memory.


To be perfectly clear, LLVM does not use libjit.

-Chris

Re: GCC + libJIT instead of LLVM

[Kirill Kononenko]

>>> It seems to
>>> me that LLVM solves many goals that are already complete and solved in
>>> GCC. So I think libJIT potentially is more useful for GCC and software
>>> developers.
>>
>> but you don't say what libjit would be more useful than, or how this
>> overlap
>> between "solved goals" between gcc and llvm implies that.  Do you simply
>> mean
>> that, because llvm and gcc both cover similar areas, and because libjit is
>> useful for llvm, it must also be useful for gcc?  Could you list some of
>> these
>> goals and explain how they relate to JIT?
>
> Hi Dave,
>
> I don't mean to invade into a pretty amusing thread, but I wanted to clarify
> one important thing: litjit and LLVM have nothing to do with each other.
>
> libjit and LLVM have very different design points.  libjit is a bit more
> than a glorified macro assembler with some late "optimizations", whereas
> LLVM is a real compiler that happens to have the ability to emit machine
> code to memory.
>
> To be perfectly clear, LLVM does not use libjit.
>
> -Chris

This is what Chris Lattner wrote a couple of years ago. Now I see an
exactly contradiction:

"

Hi All,

I was wondering what you could tell me about the differences in goals
between libjit and LLVM.  In particular, on this page:
http://www.southern-storm.com.au/doc/libjit/libjit_1.html

It is stated "Unlike other systems such as the JVM, .NET, Parrot, and
LLVM, libjit is not a virtual machine in its own right. It is the
foundation upon which a number of different virtual machines, dynamic
scripting languages, or customized rendering routines can be built."

I believe that the statement is true about .NET, JVM, and Parrot, but LLVM
is exactly what you describe: a framework that can be used to build a
number of different virtual machines.  In particular, LLVM assumes no
runtime model or library: we currently JIT C,C++,Stacker (a forth-like
language), and have good starts on Java, .NET, O-Caml, and Ruby
front-ends.

If you're not familiar with LLVM, it's actually a lot more than a JIT.
It's also a static, linktime, and offline optimizer that includes dozens
of aggressive SSA-based optimizations.  Another advantage of LLVM is that
it is mature and works well now.  :)  For more info, see:
http://llvm.cs.uiuc.edu/

Can you say a little bit about the advantages of using libjit over LLVM?
Also, can you please correct the quote above to accurately describe the
difference between libjit and LLVM?

Thanks,

-Chris

"

and
"
On Sunday 30 May 2004 12:46 pm, Chris Lattner wrote:

> What do you mean by "real" one?  They are both real.

"Real" in the sense that the other one is discontinued.  Which is essentially
what you want here: for libjit to be discontinued and turned into a thin C
API on top of LLVM.  That just isn't going to happen.  I'm just as attached
to my project as you are to yours.  Sorry.

> Sure, that's fine.  Please change the link to llvm.org though (a recent
> change).  Thanks!

I have updated the documentation in CVS and on the Web site.

Cheers,

Rhys.
"

There are also many other interesting things there:
http://lists.gnu.org/archive/html/dotgnu-libjit/2004-05/index.html


Thanks,
Kirill

Re: GCC + libJIT instead of LLVM

[Manuel López-Ibáñez]

2009/4/1 Kirill Kononenko :
>
> This is what Chris Lattner wrote a couple of years ago. Now I see an
> exactly contradiction:
>

Please, could you pinpoint side-by-side the two sentences that
contradict each other and later give links to (or quote) the context?
I am having troubling identifying the contradiction.

In any case, two years is a long time. Even if LLVM was using libjit
two years ago, it is pretty clear that LLVM is not using libjit
anymore. And that wouldn't be a contradiction.

As a general remark, I find your messages a bit vague and lacking
concreteness. Please take this as a constructive comment with the goal
of making easier: (1) for you to get your point across, and (2) for me
(and I guess others) to quickly understand your point. I think this
may be one reason why your messages are not being (generally) met with
the positive replies that you were surely expecting.

Cheers,

Manuel.

Re: GCC + libJIT instead of LLVM

[Daniel Jacobowitz]

On Wed, Apr 01, 2009 at 06:54:55PM +0200, Manuel López-Ibáñez wrote:
> 2009/4/1 Kirill Kononenko :
> >
> > This is what Chris Lattner wrote a couple of years ago. Now I see an
> > exactly contradiction:
> >
> 
> Please, could you pinpoint side-by-side the two sentences that
> contradict each other and later give links to (or quote) the context?
> I am having troubling identifying the contradiction.

Please, could you not do it on this list?

A discussion about the differences between LLVM and libjit is wildly
off-topic for GCC development.

-- 
Daniel Jacobowitz
CodeSourcery

Re: GCC + libJIT instead of LLVM

[Daniel Berlin]

On Wed, Apr 1, 2009 at 5:33 AM, Kirill Kononenko
 wrote:
> Hello Dear GCC Developers,
>
>
>
> I would like to ask your opinion about possibility for integration of
> the libJIT Just-In-Time compilation library and GCC. For example, the
> same way as libffi is integrated within gcc source tree. It seems to
> me that LLVM solves many goals that are already complete and solved in
> GCC. So I think libJIT potentially is more useful for GCC and software
> developers.
>
Highly disagree.

>
> What is your opinion about this idea? How this should be done and what
> improvements could be made to both libJIT and GCC for this kind of
> integration?

I don't think we should integrate libJIT into GCC. It doesn't solve
any of the *interesting* JIT problems we would have, it only solves
the ones we know how to solve in a fairly short time (given enough
developers).

Re: GCC + libJIT instead of LLVM

[Kirill Kononenko]

2009/4/1 Daniel Berlin :
> On Wed, Apr 1, 2009 at 5:33 AM, Kirill Kononenko
>  wrote:
>> Hello Dear GCC Developers,
>>
>>
>>
>> I would like to ask your opinion about possibility for integration of
>> the libJIT Just-In-Time compilation library and GCC. For example, the
>> same way as libffi is integrated within gcc source tree. It seems to
>> me that LLVM solves many goals that are already complete and solved in
>> GCC. So I think libJIT potentially is more useful for GCC and software
>> developers.
>>
> Highly disagree.
>
>>
>> What is your opinion about this idea? How this should be done and what
>> improvements could be made to both libJIT and GCC for this kind of
>> integration?
>
> I don't think we should integrate libJIT into GCC. It doesn't solve
> any of the *interesting* JIT problems we would have, it only solves

What are exactly the *interesting* JIT problems you have? And who are
exactly "we" again in this case? Is it Google? Or you personally speak
for all GCC user community?

> the ones we know how to solve in a fairly short time (given enough
> developers).

Do you actually have these extra 'enough developers'?




Thanks,
Kirill

Re: GCC + libJIT instead of LLVM

[Kirill Kononenko]

Please, let collect together all useful ideas and concrete thoughts? I
am sure many people already have thought about which JITing support
GCC users need. I also do have my thoughts about this research topic
but I would like also to have useful feedback from people who also
understand this research topic like me and gcc internals. Please also
stop flaming about LLVM.



Thanks,
Kirill


2009/4/1 Kirill Kononenko :
> 2009/4/1 Daniel Berlin :
>> On Wed, Apr 1, 2009 at 5:33 AM, Kirill Kononenko
>>  wrote:
>>> Hello Dear GCC Developers,
>>>
>>>
>>>
>>> I would like to ask your opinion about possibility for integration of
>>> the libJIT Just-In-Time compilation library and GCC. For example, the
>>> same way as libffi is integrated within gcc source tree. It seems to
>>> me that LLVM solves many goals that are already complete and solved in
>>> GCC. So I think libJIT potentially is more useful for GCC and software
>>> developers.
>>>
>> Highly disagree.
>>
>>>
>>> What is your opinion about this idea? How this should be done and what
>>> improvements could be made to both libJIT and GCC for this kind of
>>> integration?
>>
>> I don't think we should integrate libJIT into GCC. It doesn't solve
>> any of the *interesting* JIT problems we would have, it only solves
>
> What are exactly the *interesting* JIT problems you have? And who are
> exactly "we" again in this case? Is it Google? Or you personally speak
> for all GCC user community?
>
>> the ones we know how to solve in a fairly short time (given enough
>> developers).
>
> Do you actually have these extra 'enough developers'?
>
>
>
>
> Thanks,
> Kirill
>

Re: GCC + libJIT instead of LLVM

[Kirill Kononenko]

Hello Everyone


Do I understand it correctly that there are no useful thoughts or
ideas with flamimg and flooding about LLVM?


Thanks,
Kirill

2009/4/1 Kirill Kononenko :
> Please, let collect together all useful ideas and concrete thoughts? I
> am sure many people already have thought about which JITing support
> GCC users need. I also do have my thoughts about this research topic
> but I would like also to have useful feedback from people who also
> understand this research topic like me and gcc internals. Please also
> stop flaming about LLVM.
>
>
>
> Thanks,
> Kirill
>
>
> 2009/4/1 Kirill Kononenko :
>> 2009/4/1 Daniel Berlin :
>>> On Wed, Apr 1, 2009 at 5:33 AM, Kirill Kononenko
>>>  wrote:
 Hello Dear GCC Developers,



 I would like to ask your opinion about possibility for integration of
 the libJIT Just-In-Time compilation library and GCC. For example, the
 same way as libffi is integrated within gcc source tree. It seems to
 me that LLVM solves many goals that are already complete and solved in
 GCC. So I think libJIT potentially is more useful for GCC and software
 developers.

>>> Highly disagree.
>>>

 What is your opinion about this idea? How this should be done and what
 improvements could be made to both libJIT and GCC for this kind of
 integration?
>>>
>>> I don't think we should integrate libJIT into GCC. It doesn't solve
>>> any of the *interesting* JIT problems we would have, it only solves
>>
>> What are exactly the *interesting* JIT problems you have? And who are
>> exactly "we" again in this case? Is it Google? Or you personally speak
>> for all GCC user community?
>>
>>> the ones we know how to solve in a fairly short time (given enough
>>> developers).
>>
>> Do you actually have these extra 'enough developers'?
>>
>>
>>
>>
>> Thanks,
>> Kirill
>>
>

Re: GCC + libJIT instead of LLVM

[Kirill Kononenko]

Do I understand it correctly that there are no useful thoughts or
ideas with-out- flaming and flooding about LLVM?



Thanks,
Kirill

>
> 2009/4/1 Kirill Kononenko :
>> Please, let collect together all useful ideas and concrete thoughts? I
>> am sure many people already have thought about which JITing support
>> GCC users need. I also do have my thoughts about this research topic
>> but I would like also to have useful feedback from people who also
>> understand this research topic like me and gcc internals. Please also
>> stop flaming about LLVM.
>>
>>
>>
>> Thanks,
>> Kirill
>>
>>
>> 2009/4/1 Kirill Kononenko :
>>> 2009/4/1 Daniel Berlin :
 On Wed, Apr 1, 2009 at 5:33 AM, Kirill Kononenko
  wrote:
> Hello Dear GCC Developers,
>
>
>
> I would like to ask your opinion about possibility for integration of
> the libJIT Just-In-Time compilation library and GCC. For example, the
> same way as libffi is integrated within gcc source tree. It seems to
> me that LLVM solves many goals that are already complete and solved in
> GCC. So I think libJIT potentially is more useful for GCC and software
> developers.
>
 Highly disagree.

>
> What is your opinion about this idea? How this should be done and what
> improvements could be made to both libJIT and GCC for this kind of
> integration?

 I don't think we should integrate libJIT into GCC. It doesn't solve
 any of the *interesting* JIT problems we would have, it only solves
>>>
>>> What are exactly the *interesting* JIT problems you have? And who are
>>> exactly "we" again in this case? Is it Google? Or you personally speak
>>> for all GCC user community?
>>>
 the ones we know how to solve in a fairly short time (given enough
 developers).
>>>
>>> Do you actually have these extra 'enough developers'?
>>>
>>>
>>>
>>>
>>> Thanks,
>>> Kirill
>>>
>>
>

Re: GCC + libJIT instead of LLVM

[Dave Korn]

Kirill Kononenko wrote:
> Do I understand it correctly that there are no useful thoughts or
> ideas with-out- flaming and flooding about LLVM?

  I will admit that I don't have any ideas, but I thought there were a couple
of positive suggestions in there.

  It's possible there's a bit of "library fatigue" going on.  GCC does seem to
keep sprouting new ones!

cheers,
  DaveK

Re: GCC + libJIT instead of LLVM

[Kirill Kononenko]

There have been mentioned a couple of ideas indeed. But I would not
like to spend a lot of my precious time on telling my thoughts and
suggestions, if the topic is already decided elsewhere. So I basically
want asking question which exactly JITing support GCC needs, that I
don't spend my time in the rubbish bin. So no productivity of my part
is lost.


Thanks,
Kirill

2009/4/3 Dave Korn :
> Kirill Kononenko wrote:
>> Do I understand it correctly that there are no useful thoughts or
>> ideas with-out- flaming and flooding about LLVM?
>
>  I will admit that I don't have any ideas, but I thought there were a couple
> of positive suggestions in there.
>
>  It's possible there's a bit of "library fatigue" going on.  GCC does seem to
> keep sprouting new ones!
>
>cheers,
>  DaveK
>

Re: GCC + libJIT instead of LLVM

[Ian Lance Taylor]

Kirill Kononenko  writes:

> There have been mentioned a couple of ideas indeed. But I would not
> like to spend a lot of my precious time on telling my thoughts and
> suggestions, if the topic is already decided elsewhere. So I basically
> want asking question which exactly JITing support GCC needs, that I
> don't spend my time in the rubbish bin. So no productivity of my part
> is lost.

It doesn't really work that way.  As far as I know, gcc is not at
present looking for any JITing support.  That is, I don't know of any
active gcc projects which want to use a JIT.  In normal gcc development
practice, the gcc source base would not supply a JIT and see if gcc
developers want to use it.  Instead, somebody with a useful project who
needs a JIT would consider whether to use an existing library, such as
libJIT or LLVM, or whether to invent something new.

In other words, if your goal is to let us know that we can use libJIT,
then: mission accomplished.  If your goal is to include libJIT in the
gcc code base, then you need to 1) identify a gcc project which needs a
JIT, 2) convince people to work on it, 3) convince them that libJIT is
the right solution, 4) convince the gcc maintainers to accept the
project and libJIT into gcc mainline.  No step may be omitted.

Ian

Re: GCC + libJIT instead of LLVM

[Kirill Kononenko]

Hi Ian,


Thank you a lot for your reply.

>
>> There have been mentioned a couple of ideas indeed. But I would not
>> like to spend a lot of my precious time on telling my thoughts and
>> suggestions, if the topic is already decided elsewhere. So I basically
>> want asking question which exactly JITing support GCC needs, that I
>> don't spend my time in the rubbish bin. So no productivity of my part
>> is lost.
>
> It doesn't really work that way.  As far as I know, gcc is not at
> present looking for any JITing support.  That is, I don't know of any
> active gcc projects which want to use a JIT.  In normal gcc development
> practice, the gcc source base would not supply a JIT and see if gcc
> developers want to use it.  Instead, somebody with a useful project who
> needs a JIT would consider whether to use an existing library, such as
> libJIT or LLVM, or whether to invent something new.
>
> In other words, if your goal is to let us know that we can use libJIT,
> then: mission accomplished.  If your goal is to include libJIT in the
> gcc code base, then you need to 1) identify a gcc project which needs a
> JIT, 2) convince people to work on it, 3) convince them that libJIT is
> the right solution, 4) convince the gcc maintainers to accept the
> project and libJIT into gcc mainline.  No step may be omitted.

I think the issue that is there, is that many project want to include
their own source code in the gcc source tree. This is not a case in
this proposal. What I want to identify is how both a VM engine(ILDJIT,
.NET for example, Mono, Portable.NET), gcc and libJIT could be
extended with minimal changes to both, for best user experience for
example, is it speed performance, benchmark, code size, or power
consumption.


Thanks,
Kirill

Re: GCC + libJIT instead of LLVM

[Diego Novillo]

On Fri, Apr 3, 2009 at 10:54, Kirill Kononenko
 wrote:

> What I want to identify is how both a VM engine(ILDJIT,
> .NET for example, Mono, Portable.NET), gcc and libJIT could be
> extended with minimal changes to both, for best user experience for
> example, is it speed performance, benchmark, code size, or power
> consumption.

You still need to do the steps Ian outlined in his message.  For GCC
maintainers to accept changes (minimal as they may be), you have to
justify those changes, propose a patch and submit for review.

Whether the patch is approved or not will depend on whether you
convinced the GCC maintainers that the feature is useful and that it
complies with http://gcc.gnu.org/contribute.html.

It seems like you are still early in your design cycle.  You still
need to complete steps 1-3 in Ian's list.


Diego.

Re: GCC + libJIT instead of LLVM

[Basile STARYNKEVITCH]

Ian Lance Taylor wrote:

Kirill Kononenko  writes:

  

There have been mentioned a couple of ideas indeed. But I would not
like to spend a lot of my precious time on telling my thoughts and
suggestions, if the topic is already decided elsewhere. So I basically
want asking question which exactly JITing support GCC needs, that I
don't spend my time in the rubbish bin. So no productivity of my part
is lost.

In other words, if your goal is to let us know that we can use libJIT,
then: mission accomplished.  If your goal is to include libJIT in the
gcc code base, then you need to 1) identify a gcc project which needs a
JIT, 2) convince people to work on it, 3) convince them that libJIT is
the right solution, 4) convince the gcc maintainers to accept the
project and libJIT into gcc mainline.  No step may be omitted.
  


I believe that Kiril mentioned in a separate post the equation
   (mcs | csc | cscc) & gcc & libJIT == LLVM ?

I tend to believe that Kiril dreams of building a CLI/.NET 
infrastructure and VM which uses all the powerful optimisations of GCC.


For reference to Kiril: do you know that there exist some branches of 
GCC (I believe it was contributed by STMicro guys) which are able to
 1. Parse (as a front-end) the CLI bytecode and generate the Gimple IR 
from it

 2. Parse C# (as a front-end) and generate the Gimple IR from it
 3. Provide a backend which spills CLI bytecode from Gimple IR (without 
using the backend infrastructure of GCC, but "replacing" it).
Maybe these branches could satisfy some of Kiril dreams (which I still 
did not understood exactly).


Perhaps Kiril is dreaming of a compile server built around GCC; some 
people are experimenting that in their branch.


However, as far as I know, nobody uses libJIT inside GCC, and nobody is 
working to incorporate it inside GCC.


Maybe what Kiril is dreaming about is also:
  having a compiler server built around GCC.
  replacing the use of libJIT inside DotGNU by a library which would 
interact with that GCC server, incrementally replacing the CLI bytecode 
by their Gimple representation, invoke the various GCC optimisations, etc...


All this is very ambitious.

Maybe Kiril equation is becoming
(mcs | csc | cscc) & gcc & gcc-server & CLItoGimple  > LLVM  | DotGNU

But I still don't understand what Kiril is dreaming of. It has 
apparently no much in common with libJIT, except perhaps at most keeping 
libJIT API but rewriting it entirely to interact with a future GCC 
server. Perhaps hacking DotGNU to spill Gimple into GCC would be easier.


At last, I am not sure that any kind of JIT compiler would want to use 
expansive optimisations as those provided by GCC. JIT & AOT [ahead of 
time, like GCC usually works] compilation are really different goals, 
with different trade-offs. A JIT wants to translate code quickly into 
machine code. An AOT compiler like GCC wants to generate very efficient 
code, even while taking a lot of time for that generation.


Maybe a more reasonable project for Kiril would be to enhance & merge 
the existing branches for CLI of GCC and the GCC server branch to have 
them interact with DotGNU by providing a mode where  expansive 
compilation is done by GCC. But I am not entirely convinced that GCC is 
the good vehicule for that. A lot of stuff inside GCC is really designed 
for AOT!


Of course, I don't understand all the details involved. I don't know 
much about the various branches I mention in this email.


Regards.

--
Basile STARYNKEVITCH http://starynkevitch.net/Basile/
email: basilestarynkevitchnet mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***

Re: GCC + libJIT instead of LLVM

[Kirill Kononenko]

I have been criticized for inserting a mail from a public mail list
into this mail list. My position is that if a guy contradicts himself
in two different public mail lists, it is a reason to point to this.
Public mail list is a 'public' mail list. It is a source of reference,
for books for example even. It is very different from private
correspondence, citing which is just disgusting when done
intentionally.


Thanks,
Kirill

2009/4/3 Basile STARYNKEVITCH :
> Ian Lance Taylor wrote:
>>
>> Kirill Kononenko  writes:
>>
>>
>>>
>>> There have been mentioned a couple of ideas indeed. But I would not
>>> like to spend a lot of my precious time on telling my thoughts and
>>> suggestions, if the topic is already decided elsewhere. So I basically
>>> want asking question which exactly JITing support GCC needs, that I
>>> don't spend my time in the rubbish bin. So no productivity of my part
>>> is lost.
>>
>> In other words, if your goal is to let us know that we can use libJIT,
>> then: mission accomplished.  If your goal is to include libJIT in the
>> gcc code base, then you need to 1) identify a gcc project which needs a
>> JIT, 2) convince people to work on it, 3) convince them that libJIT is
>> the right solution, 4) convince the gcc maintainers to accept the
>> project and libJIT into gcc mainline.  No step may be omitted.
>>
>
> I believe that Kiril mentioned in a separate post the equation
>   (mcs | csc | cscc) & gcc & libJIT == LLVM ?
>
> I tend to believe that Kiril dreams of building a CLI/.NET infrastructure
> and VM which uses all the powerful optimisations of GCC.
>
> For reference to Kiril: do you know that there exist some branches of GCC (I
> believe it was contributed by STMicro guys) which are able to
>  1. Parse (as a front-end) the CLI bytecode and generate the Gimple IR from
> it
>  2. Parse C# (as a front-end) and generate the Gimple IR from it
>  3. Provide a backend which spills CLI bytecode from Gimple IR (without
> using the backend infrastructure of GCC, but "replacing" it).
> Maybe these branches could satisfy some of Kiril dreams (which I still did
> not understood exactly).
>
> Perhaps Kiril is dreaming of a compile server built around GCC; some people
> are experimenting that in their branch.
>
> However, as far as I know, nobody uses libJIT inside GCC, and nobody is
> working to incorporate it inside GCC.
>
> Maybe what Kiril is dreaming about is also:
>  having a compiler server built around GCC.
>  replacing the use of libJIT inside DotGNU by a library which would interact
> with that GCC server, incrementally replacing the CLI bytecode by their
> Gimple representation, invoke the various GCC optimisations, etc...
>
> All this is very ambitious.
>
> Maybe Kiril equation is becoming
> (mcs | csc | cscc) & gcc & gcc-server & CLItoGimple  > LLVM  | DotGNU
>
> But I still don't understand what Kiril is dreaming of. It has apparently no
> much in common with libJIT, except perhaps at most keeping libJIT API but
> rewriting it entirely to interact with a future GCC server. Perhaps hacking
> DotGNU to spill Gimple into GCC would be easier.
>
> At last, I am not sure that any kind of JIT compiler would want to use
> expansive optimisations as those provided by GCC. JIT & AOT [ahead of time,
> like GCC usually works] compilation are really different goals, with
> different trade-offs. A JIT wants to translate code quickly into machine
> code. An AOT compiler like GCC wants to generate very efficient code, even
> while taking a lot of time for that generation.
>
> Maybe a more reasonable project for Kiril would be to enhance & merge the
> existing branches for CLI of GCC and the GCC server branch to have them
> interact with DotGNU by providing a mode where  expansive compilation is
> done by GCC. But I am not entirely convinced that GCC is the good vehicule
> for that. A lot of stuff inside GCC is really designed for AOT!
>
> Of course, I don't understand all the details involved. I don't know much
> about the various branches I mention in this email.
>
> Regards.
>
> --
> Basile STARYNKEVITCH http://starynkevitch.net/Basile/
> email: basilestarynkevitchnet mobile: +33 6 8501 2359
> 8, rue de la Faiencerie, 92340 Bourg La Reine, France
> *** opinions {are only mines, sont seulement les miennes} ***
>
>

Re: GCC + libJIT instead of LLVM

[Kirill Kononenko]

Hello


After many considerations, I want to let everyone know about a release
of my work done in a package:



0.1.2.5 + / 0.1.2 1/2 version release (code name: "libJIT-ON-TESTOSTERONE")

* main branch + libJIT-linear-scan-register-allocator
* Add optimization levels for IA-32 from 0 to 4
* Add a new specialized ABI called INTERNAL
* Add brand new optimized object code generator (level 1, 2, 3, 4 of
optimization)
* Various low-level machine dependent optimizations and tricks
* Aggressive optimization of division by integer constants as
  by Torbjorn Granlund and Peter L. Montgomery in "Division By
Invariant Integers using Multiplication"
* Add primitive code generators for MMX/Streaming SIMD
Extensions/SSE/SSE2/SSE3 and others
* Use SIMD SSE/SSE2/SSE3 for floating point values and operations
(level 1, 2, 3, 4 of optimization)
* Data-flow and control-flow based analysis (level 1, 2, 3, 4 of
optimization)
* Fast liveness analysis (level 2 of optimization)
* Dead-code elimination (level 4 of optimization)
* Full liveness analysis (level 3, 4 of optimization)
* Linear scan register allocator algorithm (level 2)
* Bin packing register allocator algorithm (level 3, 4)
* Tested on DotGNU Portable.NET Common Language Runtime
  / a Microsoft Common Intermediate Language Virtual Machine

(a special "unofficial" research release version


Download is at:

http://libjit-linear-scan-register-allocator.googlecode.com/files/libjit-0.1.2.5%2B.tar.gz



This work gives a lot of improvement and beats Mono in all benchmarks.

I want to also let know that this is a research project and there is
no target to compete with Novell, or Apple. Simply because it is not
possible. But I am rather looking for cooperation for benefits of all
sides. I will be honored if this work can be a base for a cooperation
with Mono for example.


Thanks,
Kirill


2009/4/3 Diego Novillo :
> On Fri, Apr 3, 2009 at 10:54, Kirill Kononenko
>  wrote:
>
>> What I want to identify is how both a VM engine(ILDJIT,
>> .NET for example, Mono, Portable.NET), gcc and libJIT could be
>> extended with minimal changes to both, for best user experience for
>> example, is it speed performance, benchmark, code size, or power
>> consumption.
>
> You still need to do the steps Ian outlined in his message.  For GCC
> maintainers to accept changes (minimal as they may be), you have to
> justify those changes, propose a patch and submit for review.
>
> Whether the patch is approved or not will depend on whether you
> convinced the GCC maintainers that the feature is useful and that it
> complies with http://gcc.gnu.org/contribute.html.
>
> It seems like you are still early in your design cycle.  You still
> need to complete steps 1-3 in Ian's list.
>
>
> Diego.
>

Re: GCC + libJIT instead of LLVM

[Kirill Kononenko]

About the benchmarks you know what I mean I guess. That they don't
prove anything, no matter how hard other people want to prove the
inverse. Concrete figures will be in incoming research papers I am
working right now. So just don't start right now with saying give us
figures :-)


Thanks,
Kirill

2009/4/5 Kirill Kononenko :
>
>
> After many considerations, I want to let everyone know about a release
> of my work done in a package:
>
>
>
> 0.1.2.5 + / 0.1.2 1/2 version release (code name: "libJIT-ON-TESTOSTERONE")
>
>        * main branch + libJIT-linear-scan-register-allocator
>        * Add optimization levels for IA-32 from 0 to 4
>        * Add a new specialized ABI called INTERNAL
>        * Add brand new optimized object code generator (level 1, 2, 3, 4 of
> optimization)
>        * Various low-level machine dependent optimizations and tricks
>        * Aggressive optimization of division by integer constants as
>          by Torbjorn Granlund and Peter L. Montgomery in "Division By
> Invariant Integers using Multiplication"
>        * Add primitive code generators for MMX/Streaming SIMD
> Extensions/SSE/SSE2/SSE3 and others
>        * Use SIMD SSE/SSE2/SSE3 for floating point values and operations
> (level 1, 2, 3, 4 of optimization)
>        * Data-flow and control-flow based analysis (level 1, 2, 3, 4 of
> optimization)
>        * Fast liveness analysis (level 2 of optimization)
>        * Dead-code elimination (level 4 of optimization)
>        * Full liveness analysis (level 3, 4 of optimization)
>        * Linear scan register allocator algorithm (level 2)
>        * Bin packing register allocator algorithm (level 3, 4)
>        * Tested on DotGNU Portable.NET Common Language Runtime
>          / a Microsoft Common Intermediate Language Virtual Machine
>
> (a special "unofficial" research release version
> 
>
> Download is at:
>
> http://libjit-linear-scan-register-allocator.googlecode.com/files/libjit-0.1.2.5%2B.tar.gz
>
>
>
> This work gives a lot of improvement and beats Mono in all benchmarks.
>
> I want to also let know that this is a research project and there is
> no target to compete with Novell, or Apple. Simply because it is not
> possible. But I am rather looking for cooperation for benefits of all
> sides. I will be honored if this work can be a base for a cooperation
> with Mono for example.
>
>
> Thanks,
> Kirill
>
>
> 2009/4/3 Diego Novillo :
>> On Fri, Apr 3, 2009 at 10:54, Kirill Kononenko
>>  wrote:
>>
>>> What I want to identify is how both a VM engine(ILDJIT,
>>> .NET for example, Mono, Portable.NET), gcc and libJIT could be
>>> extended with minimal changes to both, for best user experience for
>>> example, is it speed performance, benchmark, code size, or power
>>> consumption.
>>
>> You still need to do the steps Ian outlined in his message.  For GCC
>> maintainers to accept changes (minimal as they may be), you have to
>> justify those changes, propose a patch and submit for review.
>>
>> Whether the patch is approved or not will depend on whether you
>> convinced the GCC maintainers that the feature is useful and that it
>> complies with http://gcc.gnu.org/contribute.html.
>>
>> It seems like you are still early in your design cycle.  You still
>> need to complete steps 1-3 in Ian's list.
>>
>>
>> Diego.
>>
>

Re: GCC + libJIT instead of LLVM

[Gabriele SVELTO]

Basile STARYNKEVITCH wrote:


I believe that Kiril mentioned in a separate post the equation
   (mcs | csc | cscc) & gcc & libJIT == LLVM ?

I tend to believe that Kiril dreams of building a CLI/.NET 
infrastructure and VM which uses all the powerful optimisations of GCC.


For reference to Kiril: do you know that there exist some branches of 
GCC (I believe it was contributed by STMicro guys) which are able to
 1. Parse (as a front-end) the CLI bytecode and generate the Gimple IR 
from it

 2. Parse C# (as a front-end) and generate the Gimple IR from it
 3. Provide a backend which spills CLI bytecode from Gimple IR (without 
using the backend infrastructure of GCC, but "replacing" it).
Maybe these branches could satisfy some of Kiril dreams (which I still 
did not understood exactly).


the CLI BE/FE branch (http://gcc.gnu.org/projects/cli.html) has seen little 
progress lately mostly because we have been 'redirected' to another project but 
we're still trying to push it forward and we have recently applied a few patches 
that make it quite usable for compiling C code and running the resulting CLI 
binaries under Mono. We support emitting CLI from GIMPLE and using CLI as a 
source language (though the front-end hasn't been tested for a while now 
unfortunately). We haven't written a C# front-end though, but maybe somebody 
else tried. That would end up as something very different from what we've been 
doing in the CLI branch (i.e. something more like gcj).

Re: GCC + libJIT instead of LLVM

[Mark Mitchell]

David Edelsohn wrote:

>> My explanations seem to have also failed to explain you.
>> Unfortunately, one really needs have some back group with both
>> Just-In-Time compilers,Virtual Machines, and Common Intermediate
>> Language to understand this area. I understand that it is not your
>> area of expertise, so it is not an issue for me.
> 
> I think a JIT associated with GCC would be great.  Not everything is
> static compilation.  "Split Compilation" combining static compilation
> for complicated analysis along with a "Managed Runtime" for
> execution and runtime transformations is becoming more important.

I agree.

I don't know how relevant libJIT might or might not be for this, but the
basic idea of GCC generating deployable code which can be further
optimized at run-time (presumably using some kind of JIT) is certainly a
good one.  I believe that's a usage model that will become increasingly
important over time.

-- 
Mark Mitchell
CodeSourcery
m...@codesourcery.com
(650) 331-3385 x713

Re: GCC + libJIT instead of LLVM

[Kirill Kononenko]

Hello Everyone


I wanted to let you know that if there is someone interested in
working on the libJIT approach instead of using LLVM overkill under a
Google Summer of Code code project and more general on this topic or
as a diploma I am ready to mentor and help with this.


Thanks,
Kirill

2009/4/6 Mark Mitchell :
> David Edelsohn wrote:
>
>>> My explanations seem to have also failed to explain you.
>>> Unfortunately, one really needs have some back group with both
>>> Just-In-Time compilers,Virtual Machines, and Common Intermediate
>>> Language to understand this area. I understand that it is not your
>>> area of expertise, so it is not an issue for me.
>>
>> I think a JIT associated with GCC would be great.  Not everything is
>> static compilation.  "Split Compilation" combining static compilation
>> for complicated analysis along with a "Managed Runtime" for
>> execution and runtime transformations is becoming more important.
>
> I agree.
>
> I don't know how relevant libJIT might or might not be for this, but the
> basic idea of GCC generating deployable code which can be further
> optimized at run-time (presumably using some kind of JIT) is certainly a
> good one.  I believe that's a usage model that will become increasingly
> important over time.
>
> --
> Mark Mitchell
> CodeSourcery
> m...@codesourcery.com
> (650) 331-3385 x713
>



-- 
http://code.google.com/p/libjit-linear-scan-register-allocator/