Hey Tai, this is a brilliant report - thanks very much. Even though you haven't done anything but back up the assertions from Clojure/core, I find your report much more reassuring (from a personal perspective) than just blindly believing something.
I think the Clojure community would benefit from further such reports/discussions. Although they might not always directly improve Clojure the language, they clearly help inform and educate Clojure the community - which can only be a good thing. We shouldn't just be satisfied with knowing which direction we're headed - we should be trying our best to collectively help each other understand why we're headed in that direction. This report has definitely given me more understanding of why. Thanks once again, Sam --- http://sam.aaron.name On 25 Aug 2011, at 22:41, Tal Liron wrote: > So, after setting up a JVM 7 environment to play with Clojure, and > enthusiastically rummaging through the codebase, I have good news and bad > news. :) > > (To quickly answer someone's question here -- I do know that it's a low > priority for the Clojure project, but thought the potential benefit could be > worth *my* personal time. A few people suggested that invokedynamic would not > be that useful for Clojure, but I'm a skeptic and wanted to see for myself. I > think other questions posed here are answered in the long report below...) > > The bad news is that I don't think Clojure can benefit from JVM 7's > invokedynamic. The good news is that it is unnecessary, because Clojure's > compiler is already efficient, and entirely bypasses all the pesky problems > other dynamic JVM languages have, for which invokedynamic was introduced. (I > work a lot with Rhino, JRuby, Jython, Groovy and Quercus (a JVM PHP engine), > and can anecdotally say that Clojure and Quercus are "fastest" of their breed > for the kind of web development work I do.) > > The thing is that Clojure's compiler is itself very Clojure-like, in that it > treats functions as a first-class data type, just like maps, sequences, vars, > etc. In the runtime, all of these data types share common interfaces, such as > ISeq, IVar and -- you guessed it -- IFn. This situation is very unlike many > non-Lisp languages, in which functions are often methods bound to classes, > modules, interfaces, etc., all that heavy internal structure that Lisps > implement in Lisp. So, when Clojure calls a function, it either already has > the instance in its entirety (a lambda) or it finds it by dereferencing a > binding. Since all functions are instances that implement IFn, the > implementation can then call invokeinterface, which is very efficient. > > [See clojure.lang.Compiler#InvokeExpr.emitArgsAndCall] > > Clojure can get away with this especially easily because the only variant for > function signatures in Clojure is arity. So, all we need is a simple switch > to call the IFn method with the correct arity. (Interestingly, this means > that Clojure has a fixed set of such signatures, and thus a fixed upper limit > to arity: 20, in case you were wondering.) > > In a language like Ruby, methods are not so free floating, and have much more > complex invocation styles as well as flexible signatures. (Clojure has only > one style: function calls are simple forms.) So, in order to use > invokeinterface, JRuby implementors would have had to create special classes > *per* method *per* invocation style if they wanted to be efficient. But this > is impossible, because so many classes would exhaust the perm-gen. For those > languages, invokedynamic is a terrific solution, because it lets them > dynamically link the implementation to the caller via a flexible > "bootstrapping" mechanism, allowing them to do entirely without the extra > class definition. Since all Clojure functions share essentially the same > class as well as interface, none of these challenges exist. > > Another aspect is the immutability of these IFn instances: you can't refactor > them at runtime, all you can do is change the bindings to refer to new > functions. So, Clojure achieves runtime dynamics by letting you simply rebind > new functions to existing Vars, Refs, Atoms, etc., and the same invocation > route continues as usual. In a language like Ruby, the bootstrapping > mechanism of invokedynamic lets implementors change the base linkage to > reflect a new signature for the method. Again, a terrific JVM 7 feature that > Clojure simply does not need. > > Another issue I examined was how Clojure calls non-Clojure JVM code, thinking > that perhaps there invokedynamic would be useful. But, Clojure again has a > straightforward approach that poses no problems. Here, Clojure very directly > calls non-Clojure code using invokestatic, invokevirtual or invokeinterface > as appropriate. A Clojure form with the "." or ".." notations translates > quite directly to a JVM method call. In fact, there's no significant > difference between how Clojure compiles such code and how a Java compiler > (such as javac) would compile such code. Clojure, too, explicitly handles > boxing and unboxing of primitive types and other conveniences. Where Clojure > slightly differs is in how it picks the correct method to call (it does > method reflection, but only once during the compilation phase), and its > coercion of types to match the called method -- after all, its type system is > dynamic. > > [See clojure.lang.Compiler#InstanceMethodExpr, StaticMethodExpr, etc.] > > Again, this is different from an implementation like JRuby, in which the > design decision was that non-Ruby code would live in Ruby "as if" it were > Ruby code, supporting that Ruby invocation styles. This makes such calls > compile like calls to other Ruby functions, and thus represents the same > challenges mentioned above. > > One thought about this: Actually, this is a place where Clojure *could* be > more like JRuby, and allow for tighter integration with non-Clojure code. For > example, I think it would convenient if, say, a non-static Java method on a > Java class could be extracted from its context and be treated as a simple > form by Clojure, in which case it would carry a class instance with it. This > is called a "delegate" in the Java world. A similar effect is easily achieved > in Clojure by using, well, closures (created by bind, let, etc.), but such a > "delegate" would let the Java method live outside of closures, and behave > like a regular Clojure fn. For example, it can be returned from a closure or > be bound to a Var. Depending on how this feature is implemented, it might > make sense to use invokedynamic for it, thought I'm guessing that it, again, > that would prove unnecessary. > > Another note re: JRuby, is that it also has a very nice implementation of > constants that makes use of invokedynamic, which ends up being somewhat > similar to the problem of switching functions dynamically, but also uses a > "SwitchPoint" mechanism to allow for very efficient concurrent mutability of > constants. Indeed, JRuby makes use of every part of JSR-292! Clojure doesn't > need consts, of course, because everything is immutable. > > In summary, Clojure's dynamics are already handled as well as could be on the > JVM, at least as far as I can see. > > And so my direct contribution to Clojure in this case is not a code patch, > but a research report. :) It's kinda like how the Large Hadron Collider might > end up disproving the existence of the Higgs boson. (Yes, I just compared > myself to the LHC! And also compared invokedynamic to the "God Particle"! > Everything is awesome!) > > If anyone thinks I missed something, please let me know. I've stared at this > problem so much that I may have ended up cross-eyed. > > +++ > > A related issue -- > > Though Clojure may not be able to benefit from invokedynamic, I think it > would be nice if it had support for working with it. For example, a set of > macros that would be able to create MethodHandles from Clojure functions, and > a way to create "bootstrap" functions in Clojure (the latter would need to be > supported in the Clojure runtime). This could be useful if someone would want > to write an implementation of a dynamic language in Clojure, or if Clojure is > to be embedded in another of the dynamic languages. Since Clojure *is* a > dynamic language (albeit one that does not need invokedynamic) it would be a > friendly gesture for it to play nicely with other dynamic languages on JVM 7, > which will probably make heavy use of invokedynamic. > > It might be a good idea to put this on a roadmap somewhere for that day in > the far future where Clojure fully supports JVM 7 features. > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with your > first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
