Thank you very much for the write up. I am going to paste this along
with some additional details on the wiki under Java 7 support.
Aaron
On 08/25/2011 05:41 PM, Tal Liron wrote:
So, after setting up a JVM 7 environment to play with Clojure, and
enthusiastically rummaging through the codebase, I have good news and
bad news. :)
(To quickly answer someone's question here -- I do know that it's a
low priority for the Clojure project, but thought the potential
benefit could be worth *my* personal time. A few people suggested that
invokedynamic would not be that useful for Clojure, but I'm a skeptic
and wanted to see for myself. I think other questions posed here are
answered in the long report below...)
The bad news is that I don't think Clojure can benefit from JVM 7's
invokedynamic. The good news is that it is unnecessary, because
Clojure's compiler is already efficient, and entirely bypasses all the
pesky problems other dynamic JVM languages have, for which
invokedynamic was introduced. (I work a lot with Rhino, JRuby, Jython,
Groovy and Quercus (a JVM PHP engine), and can anecdotally say that
Clojure and Quercus are "fastest" of their breed for the kind of web
development work I do.)
The thing is that Clojure's compiler is itself very Clojure-like, in
that it treats functions as a first-class data type, just like maps,
sequences, vars, etc. In the runtime, all of these data types share
common interfaces, such as ISeq, IVar and -- you guessed it -- IFn.
This situation is very unlike many non-Lisp languages, in which
functions are often methods bound to classes, modules, interfaces,
etc., all that heavy internal structure that Lisps implement in Lisp.
So, when Clojure calls a function, it either already has the instance
in its entirety (a lambda) or it finds it by dereferencing a binding.
Since all functions are instances that implement IFn, the
implementation can then call invokeinterface, which is very efficient.
[See clojure.lang.Compiler#InvokeExpr.emitArgsAndCall]
Clojure can get away with this especially easily because the only
variant for function signatures in Clojure is arity. So, all we need
is a simple switch to call the IFn method with the correct arity.
(Interestingly, this means that Clojure has a fixed set of such
signatures, and thus a fixed upper limit to arity: 20, in case you
were wondering.)
In a language like Ruby, methods are not so free floating, and have
much more complex invocation styles as well as flexible signatures.
(Clojure has only one style: function calls are simple forms.) So, in
order to use invokeinterface, JRuby implementors would have had to
create special classes *per* method *per* invocation style if they
wanted to be efficient. But this is impossible, because so many
classes would exhaust the perm-gen. For those languages, invokedynamic
is a terrific solution, because it lets them dynamically link the
implementation to the caller via a flexible "bootstrapping" mechanism,
allowing them to do entirely without the extra class definition. Since
all Clojure functions share essentially the same class as well as
interface, none of these challenges exist.
Another aspect is the immutability of these IFn instances: you can't
refactor them at runtime, all you can do is change the bindings to
refer to new functions. So, Clojure achieves runtime dynamics by
letting you simply rebind new functions to existing Vars, Refs, Atoms,
etc., and the same invocation route continues as usual. In a language
like Ruby, the bootstrapping mechanism of invokedynamic lets
implementors change the base linkage to reflect a new signature for
the method. Again, a terrific JVM 7 feature that Clojure simply does
not need.
Another issue I examined was how Clojure calls non-Clojure JVM code,
thinking that perhaps there invokedynamic would be useful. But,
Clojure again has a straightforward approach that poses no problems.
Here, Clojure very directly calls non-Clojure code using invokestatic,
invokevirtual or invokeinterface as appropriate. A Clojure form with
the "." or ".." notations translates quite directly to a JVM method
call. In fact, there's no significant difference between how Clojure
compiles such code and how a Java compiler (such as javac) would
compile such code. Clojure, too, explicitly handles boxing and
unboxing of primitive types and other conveniences. Where Clojure
slightly differs is in how it picks the correct method to call (it
does method reflection, but only once during the compilation phase),
and its coercion of types to match the called method -- after all, its
type system is dynamic.
[See clojure.lang.Compiler#InstanceMethodExpr, StaticMethodExpr, etc.]
Again, this is different from an implementation like JRuby, in which
the design decision was that non-Ruby code would live in Ruby "as if"
it were Ruby code, supporting that Ruby invocation styles. This makes
such calls compile like calls to other Ruby functions, and thus
represents the same challenges mentioned above.
One thought about this: Actually, this is a place where Clojure
*could* be more like JRuby, and allow for tighter integration with
non-Clojure code. For example, I think it would convenient if, say, a
non-static Java method on a Java class could be extracted from its
context and be treated as a simple form by Clojure, in which case it
would carry a class instance with it. This is called a "delegate" in
the Java world. A similar effect is easily achieved in Clojure by
using, well, closures (created by bind, let, etc.), but such a
"delegate" would let the Java method live outside of closures, and
behave like a regular Clojure fn. For example, it can be returned from
a closure or be bound to a Var. Depending on how this feature is
implemented, it might make sense to use invokedynamic for it, thought
I'm guessing that it, again, that would prove unnecessary.
Another note re: JRuby, is that it also has a very nice implementation
of constants that makes use of invokedynamic, which ends up being
somewhat similar to the problem of switching functions dynamically,
but also uses a "SwitchPoint" mechanism to allow for very efficient
concurrent mutability of constants. Indeed, JRuby makes use of every
part of JSR-292! Clojure doesn't need consts, of course, because
everything is immutable.
In summary, Clojure's dynamics are already handled as well as could be
on the JVM, at least as far as I can see.
And so my direct contribution to Clojure in this case is not a code
patch, but a research report. :) It's kinda like how the Large Hadron
Collider might end up disproving the existence of the Higgs boson.
(Yes, I just compared myself to the LHC! And also compared
invokedynamic to the "God Particle"! Everything is awesome!)
If anyone thinks I missed something, please let me know. I've stared
at this problem so much that I may have ended up cross-eyed.
+++
A related issue --
Though Clojure may not be able to benefit from invokedynamic, I think
it would be nice if it had support for working with it. For example, a
set of macros that would be able to create MethodHandles from Clojure
functions, and a way to create "bootstrap" functions in Clojure (the
latter would need to be supported in the Clojure runtime). This could
be useful if someone would want to write an implementation of a
dynamic language in Clojure, or if Clojure is to be embedded in
another of the dynamic languages. Since Clojure *is* a dynamic
language (albeit one that does not need invokedynamic) it would be a
friendly gesture for it to play nicely with other dynamic languages on
JVM 7, which will probably make heavy use of invokedynamic.
It might be a good idea to put this on a roadmap somewhere for that
day in the far future where Clojure fully supports JVM 7 features.
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient
with your first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--
Cheers,
Aaron Bedra
--
Clojure/core
http://clojure.com
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
