Removing all context, so it's clear this is just my opinions and thoughts...
As most of you know, we've opened up our codebase and incorporated the graciously-donated RubyTruffle directly into JRuby. It's available on JRuby master and we are planning to ship Truffle support with JRuby 9000, our next major version (due out in the next couple months). At the same time, we have been developing our own next-gen IR-based compiler, which will run unmodified on any JVM (with or without invokedynamic, though I still have to implement the "without" side). Why are we doing this when Truffle shows such promise? I'll try to enumerate the benefits and problems of Truffle here. * Benefits of using Truffle 1. Simpler implementation. >From day 1, the most obvious benefit of Truffle is that you just have to write an AST interpreter. Anyone who has implemented a programming language can do this easily. This specific benefit doesn't help us implement JRuby, since we already have an AST interpreter, but it did make Chris Seaton's job easier building RubyTruffle initially. This also means a Truffle-based language is more approachable than one with a complicated compiler pipeline of its own. 2. Better communication with the JIT. Truffle, via Graal, has potential to pass much more information on to the JIT. Things like type shape, escaped references, frame access, type specialization, and so on can be communicated directly, rather than hoping and praying they'll be inferred by the shape of bytecodes. This is probably the largest benefit; much of my time optimizing JRuby has been spend trying to "trick" C2 into doing the right thing, since I don't have a direct way to communicate intent. The peak performance numbers for Truffle-based languages have been extremely impressive. If it's possible to get those numbers reasonably quickly and with predictable steady-state behavior in large, heterogeneous codebases, this is definitely the quickest path (on any runtime) to a high-performance language implementation. 3. OSS and pure Java Truffle and Graal are just OpenJDK projects under OpenJDK licenses, and anyone can build, hack, or distribute them. In addition, both Truffle and Graal are 100% Java, so for the first time a plain old Java developer can see (and manipulate) exactly how the JIT works without getting lost in a sea of plus plus. * Problems with Truffle I want to emphasize that regardless of its warts, we love Truffle and Graal and we see great potential here. But we need a dose of reality once in a while, too. 1. AST is not enough. In order to make that AST fly, you can't just implement a dumb generic interpreter. You need to know about (and generously annotate your AST for) many advanced compiler optimization techniques: A. Type specialization plus guarded fallbacks: Truffle will NOT specialize your code for you. You must provide every specialized path in your AST nodes as well as annotating "slow path", "transfer to interpreter", etc. B. Frame access and reification: In order to have cross-call access to frames or to squash frames created for multiple inlined calls, you must use Truffle's representation of a frame. This means loads/stores within your AST must be done against a Truffle object, not against an arbitrary object of your own creation. C. Method invocation and inlining: Up until fairly recently, if you wanted to inline methods you had to essentially build your own call site logic, profiling, deopt paths within your Truffle AST. When I did a little hacking on RubyTruffle around OSS time (December/January) it did *no* inlining of Ruby-to-Ruby calls. I hacked in inlining using existing classes and managed to get it to work, but I was doing all the plumbing myself. I know this has improved in the Truffle codebase since then, but I have my concerns about production readiness when the inlining call site parts of Truffle were just recently added and are still in flux. And there's plenty of other cases. Building a basic language for Truffle is pretty easy (I did a micro-language in about two hours at JVMLS last year), but building a high-performance language for Truffle still takes a fair investment of effort and working knowledge of dynamic compiler optimizations. 2. Long startup and warmup times. As Thomas pointed out in the other thread, because Truffle and Graal are normally run as plain Java libraries, they can actually aggravate startup time issues. Now, not only would all of JRuby have to warm up, but the eventual native code JIT has to warm up too. This is not surprising, really. It is possible to mitigate this by doing some form of AOT against Graal, but for every case I have seen the Truffle/Graal approach makes startup time much, much worse compared to just running atop JVM. Warmup time is also worsened significantly. The AST you create for Truffle must be heavily mutated while running in order to produce a specialized version of that AST. This must happen before the AST is eventually fed into Graal, which means you have a self-modifying interpreter spinning AST objects like mad while executing the early phases of your application. Compare to a dumb interpreter as in JRuby's old AST, where interpreting the AST produces no additional objects other than those necessary for execution of the code. The Truffle approach itself adds overhead too. Until optimized, the fully-reified frame objects, specialization markup (which triggers AST rewriting), deoptimization guards, and so on are all done manually against heap-level data structures. This is in addition to the JVM-level overhead of executing an AST (native frame-per-node, boxing and type-widening, poor inlining profile). Some amount of AOT *might* be applicable here, but the benefit of Truffle and Graal is lost in the AOT case if we're not getting real-world profile information. The Substrate VM has ben brought up to aid startup and warmup too...but that direction produces a closed-world executable based on optimizing all code up front...not exactly what we're looking for in a general-purpose language runtime. 3. Limited concurrency The RubyTruffle runtime currently has to execute code under the watchful eye of a global lock. Yes, you read that right...RubyTruffle is single-threaded right now. I would like to know if there's a deeper reason for this, but the obvious shallow reason is that you can't have multiple threads executing at the same time if they're making thread-unsafe mutations to the executing code. This is similar to the major stumbling block for e.g. Pypy, which rewrites currently-executing assembly instructions at deopt/reopt safe points. I believe once the code has transitioned to native, you can execute that safely across threads...but this is opaque to your Truffle-based language, and it's unclear how you'd manage re-acquiring some sort of lock when transferring back to the interpreter. The fact that concurrency has so far been hand-waved (or so it seems to me from the outside) scares the living hell out of me, especially when there's talk about rolling this stuff into Java 9. Obviously some of this could be mitigated with an immutable AST structure or other thread-friendly tree-transformation algorithm, but making the Truffle AST thread-safe may also make it even more object-heavy during interpretation, aggravating startup time further. 4. Limited availability This is the chicken-and-egg issue. Truffle is just a library, so we can ignore that for the moment (given any JVM, you can run a Truffle language). Graal is required for Truffle to perform well at all. The Truffle interpreter is without a doubt the slowest interpreter we've ever had for JRuby, and that's saying something (there could be startup/warmup effects in play here too). In order for us to go 100% Truffle, we'd need a Graal VM. That limits us to either pre-release or hand-made builds of Graal/OpenJDK. Even if Graal somehow did get into Java 9, we'd still have legions of users on 8, 7, ... even 6 in some cases, though we're probably leaving them behind with JRuby 9000. Ignoring other platforms (non-OpenJDK, Android) and assuming Graal in Java 9, I'd conservatively estimate JRuby could still not go 100% Truffle until 2017 or later. And it gets worse. Graal will probably never exist on other JVMs. Graal will probably never exist in an Android VM. Graal may not even be available in other non-Oracle OpenJDK derivatives for a very long time. We have users on dozens of different platform/JVM combinations, so there's really no practical way for us to abandon our JVM bytecode runtimes in the near future. Now of course if Graal became essential to users, it would be available in more places. We recognize the potential of Truffle and Graal, which is why we've been thrilled to work with Oracle on a RubyTruffle that's part of JRuby. We also recognize that the Truffle/Graal approach has some very compelling features for our users, and that our users may often be comfortable running custom JVMs. We're allowing all flowers to bloom and our users will pick the ones that work for them. 5. Unclear benefits for real-world applications There have been many published microbenchmarks for Truffle-based languages, but very few benchmarks of real-world applications performing significantly better than custom-made VMs (JS versus V8). There have been practically no studies of a Truffle-based language running a large application for a long period of time...and by long I mean server-scale. Chris Seaton has pushed this forward recently for Ruby, getting general-purpose, numeric-heavy libraries to run and optimize very well (a png library and a psd library). Going deeper requires having more of the language's standard libraries to be available, and I believe this is where Chris has spent much of his time (RubyTruffle currently requires mostly-custom versions of JRuby's core classes...versions that Truffle can recognize, specialize, and escape-analyze away). * Conclusion I again want to emphasize that we think Truffle and Graal are really awesome technology. I spent years with my nose smooshed against the glass, watching the Pypy guys add optimizations I wanted and make good on their promise of "just implement an interpreter...we'll do the rest". Finally we have what I wanted: a Pypy for JVM (in Truffle) and an LLVM for JVM (in Graal). These are exciting times indeed. But reality steps in. There's a long road ahead. I think we need to separate the questions about Truffle from questions about Graal. Truffle is ultimately just a library that uses Graal. Graal is promising JIT technology. Graal is simpler than C2 and may be able to match or beat its performance. Graal provides a better way to communicate intent to the JIT. These facts are not in question. However, Graal is not (other than when used as the JVM's JIT) a JVM. Targeting Graal directly acts against the promise of a standard, platform-and-VM-agnostic bytecode -- and that's the promise that brought most of us here. Graal is not yet ready to replace C2, which would mean adding to the size and complexity of Java 9. And Graal is almost completely untested in large production settings. I personally would love to see Graal get into a Java release soon as an experimental feature, but Java 9 seems ambitious but any standard. It *might* be possible/reasonable to include Graal as experimental in 9. Java 10 is certainly feasible for experimental, and may be feasible for product. But even if Graal got into mainstream OpenJDK and Java, there's a very long adoption tail ahead. I'd like to hear more from folks on the Graal and Truffle teams. Prove me wrong :-) - Charlie _______________________________________________ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev