Hi Adam, Sorry for the long email, I hope I'm making a lot of things clearer. If not, we should chat via IRC.
The primary differences are architectural: There are two compiler classes. A MethodCompiler and a normal Compiler. The primary difference being that Compiler is responsible for an entire Assembly, its Metadata and to create specific instances of the abstract MethodCompiler. The MethodCompiler is a base class for compilers, well uhm Methods. The key idea behind a MethodCompiler being Jit support for single methods and the ability to assign different method compilers to different functions, choosen by settings, heuristics (code complexity), adaptive runtime compiliation models (often called methods receive stronger optimizations) and hot recompilation support. A MethodCompiler doesn't actually know how to compile anything, but uses an instance of CompilerPipeline. The compiler pipeline is built out of one or more instances of objects, which implement the ICompilationStage interface. This interface only defines two things: The name of the stage and a Run(MethodCompiler compiler) method. Usually compilers are built around the following idea: Source reader -> tokenizer -> syntax checker -> IR transform -> simplification -> SSA transform -> Optimizations -> SSA untransform -> Instruction selection -> Register Allocation -> Code output The idea is to split the implementations of everything and make it exchangeable just be redefining the compiler pipeline used. So the above stages would all be represented by individual classes. The benefit of this approach is the ability to plug&play a compiler from a set of individual algorithms. So the AoT could for example use a graph coloring register allocator, because it has the time. And a Jit could use a linear scan. The Jit could use a subset of the optimizations available in the AoT, just by removing the time consuming optimization stages. (All types so far located in SharpOS.Runtime.CompilerServices namespace) So this design does benefit multiple developers: Everyone can work on their stage without breaking anyone else. Buggy stages can simply be removed from the pipeline without affecting other devs and users much. This code includes a full PE file loader, which I think can be improved a bit more and integrated into SharpOS itself. My goal back then was to have a compiler, which shared a lot of infrastructure with the OS itself, in order to get a high degree of code reuse and coverage. The ngen code included uses the PE loader to load an assembly and creates a MethodCompiler for all methods found. It has a full IL decoder, which is able to decode the IL, build the IR and split it into basic blocks including dominator support all in one linear scan, e.g. being very fast. I would like to improve the IL decoder in order to replace Mono.Cecil, as we will not be able to afford loading an Assembly definition using Mono.Cecil with the Jit compiler in SharpOS: It simply takes too long and consumes too much memory in order to use it to compile a single or a set of methods. The PE loader is located in SharpOS.Runtime.Loader.PE. Metadata decoding is included in SharpOS.Runtime.Metadata, but definitely has room for improvement. I'm actually looking for a design using less memory and higher performance here. The key being that in contrast to Mono.Cecil I'm not instantiating classes/structures for metadata right away, but only when it is needed. More changes affect the way the IR is built: The IL Decoder invokes the target Architecture to retrieve the IR of an IL opcodes, which allows the target architecture to filter or override standard IR in order to adjust to special requirements. I did this on purpose in order to have the ability to allow the target architecture to affect IR instruction selection and throw errors for unsupported opcodes. The latter being primarily used with my goal of being able to compile IL not only to CPUs, but also to compile certain IL subsets down to shaders for GPUs. Think of OpenGL, Apples CoreImage or scientific algorithms. The IL decoder is very far and watches the operand stack etc, the IR isn't very far yet. Some things are missing: Callsites and signatures are not decoded, call arguments are not popped of the operand stack. It is thus currently not able to fully decode mscorlib, because of the call signature and operand pop issue. It does support generics fully. The IR is located in SharpOS.Runtime.CompilerServices.Ir and architecture specific code in other namespaces like SharpOS.Runtime.CompilerServices.x86. The IL decoder is in SharpOS.Runtime.CompilerServices.ILDecoder - a single compilation stage available for any compiler using the this framework. One more thing the architecture can influence is the compilation stages. It can add target specific compilation stages in order to provide better instruction scheduling for todays highly parallel CPUs to reduce latencies. And finally I have a slightly different approach to minor runtime related things, but that's no biggie. We're pretty similar there. So no, this is not just a new option switch. This is a very different approach to things. Like I said the AOT is further along with a lot of things implementation wise, but its design could use improvement IMHO. This is not a replacement for it and I do not intend to build a full replacement myself. Its a design prototype with certain goals in mind. I think the current AOT code can be fit in this design and a lot of stuff could benefit. And it certainly is diversified and allows more devs to work on the compiler. grover -----Ursprüngliche Nachricht----- Von: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Im Auftrag von Adam Stevenson Gesendet: Montag, 14. April 2008 08:25 An: [email protected] Betreff: Re: [SharpOS Developers] AoT & Jit compiler architecture Howdy Grover, I am taking a look now at your code. Could you outline though where some of the big changes in ideology are in the code base? Or what features we need to integrate. I have no problem with integrating trying to keep peoples own seperate goals integrated into the main. The worse that ever happens is there is just option switches added. The best what happens is that you have a code base that is very diversified and able to adapt to change well. (and a lot of developers!) -Adam On Sun, Apr 13, 2008 at 6:38 PM, Michael Ruck <[EMAIL PROTECTED]> wrote: > Hi Darx, Hi SharpOS community, > > I've checked in an early draft of the compiler I wrote for my managed > OS into my sandbox. Like I said it is organized differently with > several thoughts in mind. The SVN path is /sandbox/grover/mscoree and > /sandbox/grover/ngen. It is not complete, but you should get an idea. > I'd like to chat with you about this and how we could merge this (at > least the design, as the AOT is further along.) I'd like to add the > GC supporting stuff to the compiler, do the refactoring if we agree > and do the remaining jit/pe loader work for SharpOS. > > I'd propose a time, but I'm not sure what suits you best and which > timezone you're in. I'm in GMT +2 (central europe daylight savings > time) or Dresden, Germany. I could meet with you in IRC any evening from 2100GMT to 2300GMT. > > grover > > > > ---------------------------------------------------------------------- > --- This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > Don't miss this year's exciting event. There's still time to save $100. > Use priority code J8TL2D2. > > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com > /javaone _______________________________________________ > SharpOS-Developers mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/sharpos-developers > ------------------------------------------------------------------------- This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javao ne _______________________________________________ SharpOS-Developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sharpos-developers ------------------------------------------------------------------------- This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone _______________________________________________ SharpOS-Developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sharpos-developers
