Re: RFR[9]:Fix java/lang/invoke/MethodHandleImpl's use of Unsafe.defineAnonymousClass()
yes, adding @Hidden solves the first item of my list, but nevertheless changing the behavior of defineAnonymousClass does not solve the other items. That said, i hijack this thread because i have not noticed that defineAnonymousClass behavior was changed. I should have started another thread about that. Rémi Le 11 mai 2016 15:10:02 CEST, Vladimir Ivanova écrit : >Let me clarify: both proposed patches move invoker class out of >java.lang.invoke package, but add @Hidden on invoke_V instead. > >So, JVM should not list it in stack traces and you don't have to filter > >it out on your side. > >Moreover, I think the absence of @Hidden on >j.l.i.MethodHandleImpl.T.invoke_V was an overlook. > >Best regards, >Vladimir Ivanov > >On 5/11/16 3:59 PM, fo...@univ-mlv.fr wrote: >> - Mail original - >>> De: "Vladimir Ivanov" >>> À: "Remi Forax" , "shilpi rastogi" > >>> Cc: core-libs-...@openjdk.java.net, "John Rose" > , "Michael Haupt" , >>> "paul sandoz" , "Da Vinci Machine Project" > >>> Envoyé: Mercredi 11 Mai 2016 14:50:25 >>> Objet: Re: RFR[9]:Fix java/lang/invoke/MethodHandleImpl's use of >Unsafe.defineAnonymousClass() >>> >>> Remi, I'm curious why doesn't @Hidden on the invoker method solve >your >>> problem? >>> >>> Best regards, >>> Vladimir Ivanov >> >> Hi Vladimir, >> as far as i know @Hidden only work on the stackframe that correspond >to a method marked with @Hidden, >> not for the stackframe on top of the stackframe marked. >> So having the invoker marked with @Hidden is not enough, but maybe >i'm wrong. >> >> Rémi >> >>> >>> On 5/11/16 3:44 PM, Remi Forax wrote: Hi all, changing the behavior of defineAnonymousClass in 9 is huge burden >for me and i believe for anybody that maintains a dynamic language >runtime. As an implementer, being able to choose the package of an anonymous >class is an important feature. I use to choose carefully the package name for: - filtering the stack trace element that will be shown or not to >the user. This patch specifically broke the stack trace that my runtime >will emit because it removes "java.lang.invoke". I'm not the only one to filter out stacktrace element that starts >with "java.lang.invoke", Nashorn or JRuby do that too. I can modify the code to use the new StackWalking API if all the >method handle form artifact are marked with an interface or something >like this. - generate proxy in an existing package see https://github.com/forax/proxy2 - generate code specialization (specialization of an existing >method for some primitive types) of an existing class in an existing package (for the specialization, i specialize the constant pool at >runtime so i have no choice but to use defineAnonymousClass). I understand that being able to generate a class in any package do >not work well with the jigsaw view of the world but that's why >defineAnonymousClass is in Unsafe after all. regards, Rémi - Mail original - > De: "shilpi rastogi" > À: core-libs-...@openjdk.java.net, "John Rose" > , > "Michael Haupt" , > "paul sandoz" , "Vladimir Ivanov" > > Envoyé: Mercredi 11 Mai 2016 13:24:09 > Objet: RFR[9]:Fix java/lang/invoke/MethodHandleImpl's use of > Unsafe.defineAnonymousClass() > > Hi All, > > Please review the following- > > https://bugs.openjdk.java.net/browse/JDK-8149574 > > Solution: Changed anonymous class package name with the package >name of > its host class. > > Two approaches to solve this- > 1. Parse .class and get the class name index form constant pool >and > patch it with new name > http://cr.openjdk.java.net/~srastogi/8149574/webrev.05/ > > 2. Create class with new name (With ASM) > http://cr.openjdk.java.net/~srastogi/8149574/webrev.06/ > > Which approach is better? > > Thanks, > Shilpi > > > > >>> ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: JFokus VM Tech Day 2016
:) Do you plan to attend the VM Tech day ? Rémi Le 12 novembre 2015 02:37:04 CET, John Rosea écrit : >On Nov 11, 2015, at 12:35 AM, Marcus Lagergren >wrote: >> >> bare silicone magic > >That extra E moves the venue from Santa Clara to Las Vegas. > > > >___ >mlvm-dev mailing list >mlvm-dev@openjdk.java.net >http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: What can we improve in JSR292 for Java 9?
Hi John, comment inlined Le 26 août 2015 00:30:00 CEST, John Rose john.r.r...@oracle.com a écrit : On Feb 25, 2015, at 6:29 PM, John Rose john.r.r...@oracle.com wrote: Maybe this is general enough: MHs.loop(init, predicate, body)(*a) = { let i = init(*a); while (predicate(i, *a)) { i = body(i, *a); } return i; } ...where the type of i depends on init, and if init returns void then you have a classic side-effect-only loop. FTR, I've been thinking more about this one, with help from Michael Haupt. The above loop is pretty good, except for a couple of things: 'i' ought to be tuple of loop-varying values, and the exit path needs a more general post-processing step, a finalizer that corresponds to the initializers that set up the loop variables. A generalized (linear) loop has two kinds of values flowing within it: loop-invariant and loop-varying. (The control predicate has a special role, but may be classed as loop-varying. The finalizer is also special, and loop-invariant.) Loop-varying values can be inputs, control values (array indices, etc.), and outputs, but can all, I think, be modeled as step functions using a common signature. This takes us to something like the C for-loop, but with a sprinkle of tuples, with a finishing bow around it to tie it up: for ((v1,v2,v3,…) = inits; predicate(v1…); (v1…) = step(v1…)) { … }; return finish(v1…); Loop-local but invariant values can be modeled as non-updated loop variables, as in this C idiom: for (int i, limit = computeLimit(); i limit; i++) … Pure side effects can be modeled as void-valued step functions, so we want to allow void as a corner case for loop variable types. (I.e., where there is a step but not a value.) I don't want to make any more special provision for pure side effects (such as a unique Runnable-like body for a loop), both because it isn't necessary, and because it encourages sloppy side-effect-driven loops. If we allow each loop variable to have its own predicate and finalizer, we can properly model the effects of ordered guards and special early-exit paths. loop({init*}, {pred*}, {step*}, {fini*}) := lambda (arg*) { let var* = init*(arg*) for (;;) { for ((v,p,s,f) in (var*, pred*, step*, fini*)) if (!p(var*, arg*)) return f(var*, arg*) v = s(var*, arg*) // all vars help update each var } } } All invocations are exact. As a bonus we can easily derive post-checked loops (do-while), and time-shifted index variables (p++) simply by adding more variables and/or steps. but i think i prefer to have pre and post updates instead of several predicates/finishers. Like this: loop({init*}, {prestep*}, {pred}, {poststep*}, {fini}) := lambda (arg*) { let var* = init*(arg*) for (;;) { foreach v in vars: v = prestep(var*, args*) if (!pred(var*, arg*)) return fini(var*, arg*) foreach v in vars: v = poststep(var*, args*) } it's less powerful but it seems you can encode all your examples of loops too. Constraints on parameter and return types: - There must be the same number of inits, preds, steps, and finis. - There must be at least one input function (e.g., a pred, so that the loop has a chance to exit). - Parameter lists of the init functions must all be the same. - Parameter lists of the pred, step, and fini functions must all be the same. - Parameter lists of the step functions must be a suffix of the parameter lists of the init functions. - Parameters of the step functions must be the same as the return types of the init functions (with voids removed), followed by the common suffix. - Return types of the predicates must all be boolean. - Return types of the finalizers must all be the same (and can be void). - The return type of each init function must match the return type of the corresponding step function (a var type or void). The result, a looping method handle, has a return type taken from the finalizer function(s), and argument types taken from the init functions. The loop has one iteration variable for each non-void init/step function pair. Each step/pred/fini call takes all of the iteration variable values, followed by all the external (loop-invariant) arguments. No throws are caught by the loop; throws cause the loop to terminate abnormally. In some loops, such as find first match, the predicate naturally follows the step function, because it needs to evaluate the result of the step. To model post-checks (and the classic do/while shape), a second void-valued loop variable is required: loop(...) := lambda (arg*) { let (var, _) = (null, void) for (;;) { if (_) return _ // trivial predicate: exit branch never taken var = s(_, arg*) // pick up next search key if (!p(var, arg*)) // did we find it? return f(var, arg*) // extract value from search key as needed _ = _(var, arg*) // void value, no side effect } } This variable-doubling could be compressed out if there were a further boolean (per pred/step
Re: ClassValue rooting objects after it goes away?
No, it's not expected ! Rémi Le 6 août 2015 20:26:43 CEST, Charles Oliver Nutter head...@headius.com a écrit : Pardon me if this has been discussed before, but we had a bug (with fix) reported today that seems to indicate that the JVM is rooting objects put into a ClassValue even if the ClassValue goes away. Here's the pull request: https://github.com/jruby/jruby/pull/3228 And here's one example of the root trace leading back to our JRuby runtime. All the roots appear to be VM-level code: https://dl.dropboxusercontent.com/u/9213410/class-values-leak.png Is this expected? If we have to stuff a WeakReference into the ClassValue it seriously diminishes its utility to us. - Charlie ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: invokedynamic and subclasses
Hi Mike, i've compiled a small list of patterns, https://code.google.com/p/jsr292-cookbook/ take a look to the first inlining cache example. cheers, Rémi Le 24 juin 2015 14:19:32 CEST, Mike Jarmy mja...@gmail.com a écrit : I've been experimenting with invokedynamic in a compiler for a dynamic language that I'm working on, and I've run into a problem that I'm having difficulty solving. Let's say I have a class called A (Java syntax used for clarity): public class A { } And another called B, that is a subclass of A: public class B extends A { public Value foo; public Value bar; } There will be lots of other subclasses of A as well (let's call them C, D, E, etc), some of which will have a field called foo, and some of which won't. The class A will never have any fields -- its sole purpose is to be the base class of everything else. I have been able to successfully use a static bootstrap method in B, so that I can compile a call to invokedynamic on the imaginary method get_foo() of an instance of B, and then reroute the call inside B's bootstrap method via MethodHandles.Lookup.findGetter(), and finally return the current value of foo. So far, so good. However, at the place where I am compiling the invokedynamic instruction, I have no idea if the target is a B, C, D, or what. All I know is that it must be an A. So what I really want to be able to do (I think) is to use a static bootstrap method in A. I want get_foo() to succeed for every invokedynamic call to an instance of A who's *subclass* really does have a field called foo. Unfortunately there doesn't seem to be a way to make A do what I want. I understand why that is -- foo doesn't exist in A, so there is no way to create a findGetter() call. But I'm hoping there might be some clever way to do it anyway. I've tried all kinds of different approaches (making yet another invokedynamic call from inside A, etc, etc) but I can't come up with anything that works. Any ideas? By the way, I've figured out how to do this the old way, by generating interfaces for each class that has e.g. a foo field, and casting to that interface every time I compile a get_foo() invocation. This works, and its reasonably performant, but using invokedynamic seems like it would be a more elegant and flexible solution. ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Backport 2.0 RC
As traditionally for the JVM Summit, I'm please to announce a new version of the JSR 292 backport, http://code.google.com/p/jvm-language-runtime/downloads/list This version as some speed improvements and numerous bug fixes thanks to the Nashorn Team. I'm still interested by any report of your favorite language running with the backport, Jochen?, Mark?, Duncan?, Charles*? or anyone that develop a language runtime. cheers, Rémi * can I also beg for a special voodoo option of the CLI of JRuby to force invokedynamic bytecode on a 1.6 VM ? ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Backport 2.0 RC
On 07/27/2012 07:56 PM, Jochen Theodorou wrote: Am 27.07.2012 18:44, schrieb Rémi Forax: As traditionally for the JVM Summit, I'm please to announce a new version of the JSR 292 backport, http://code.google.com/p/jvm-language-runtime/downloads/list This version as some speed improvements and numerous bug fixes thanks to the Nashorn Team. I'm still interested by any report of your favorite language running with the backport, Jochen?, Mark?, Duncan?, Charles*? or anyone that develop a language runtime. The Groovy plans for invokedynamic are that version 3 of Groovy, coming next year, will use invokedynamic per default and the backport will be then used as a way to make it run on earlier VMs. I haven't really tried it out yet, instead in Groovy 2.0.1 we support invokedynamic as a Groovy VM plugin for jdk7. But I intend to test your backport against our call site caching after the summit well... I will be in Santa Clara on Sunday already if you want we could meet up and try making an integration of your backport. A meeting with whoever wants to participate in the invokedynamic workshop to speak what to do there is also something that could be done. Otherwise I will have to think of it all by myself. My plane will land Sunday late in the evening, so I think it's better to try to figure out that during the summit. bye blackdrag C U soon, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Backport 2.0 RC
On 07/27/2012 09:49 PM, Mark Roos wrote: Hi Remi How do I run with the backport? I am using OSX Lion. regards mark download the backport, unzip the zip, in lib you have two files one is asm-4 and the other is the backport (jsr292-backport.jar) the jsr292-bkkcport.jar is a Java agent so you can just run with the jdk6 provided by Apple, java -javaagent:lib/jsr292-backport.jar YourMain the only constraint is that because jsr292-backport.jar has a dependency on asm.jar, they should be in the same directory. cheers, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: MethodHandle.bindTo() only for reference types?
On 07/26/2012 11:50 AM, Aleksey Shipilev wrote: Hi John, On 07/24/2012 11:02 PM, John Rose wrote: On Jul 24, 2012, at 11:09 AM, Attila Szegedi wrote: MethodHandle.bind is a less-general primitive. The general API is insertArguments. Good 292 support for primitives requires a signature-polymorphic API. Therefore, we have been considering adding something like this, to fill functionality not covered by bind and insertArguments: Thanks for thinking forward on this. But let's get back to the original issue: inconsistent API between MH.bindTo() and MHs.insertArguments(). If the latter boxes primitive values, why not to allow the former to do the same? Albeit it is not a good support for primitives, but this is the consistency improvement. The idea that that bindTo() should be used as Attila said when you want to de-virtualize a method by fixing the receiver to a particular object. The idea is that when you see bindTo() in a code you know the intent of the programmer without having to track precisely which parameter is bound with which value. Thanks, Aleksey. cheers, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: MethodHandle.bindTo() only for reference types?
On 07/24/2012 09:02 PM, John Rose wrote: On Jul 24, 2012, at 11:09 AM, Attila Szegedi wrote: I don't think technically there'd be a difficulty in having it work on primitives, it's just that it ain't the intent; you still use insertArguments() for uses that are not semantically binds. MethodHandle.bind is a less-general primitive. The general API is insertArguments. Good 292 support for primitives requires a signature-polymorphic API. Therefore, we have been considering adding something like this, to fill functionality not covered by bind and insertArguments: class MethodHandle { /** Binds the given argument(s) to the method handle, but does not invoke it. * Arguments are not boxed, but rather passed in a signature-polymorphic call. * Requires an exact match to the corresponding leading parameter types. * Returns a method handle which accepts any remaining arguments, * and invokes the original on the combined argument lists. * Equivalent to insertArguments(this, 0, args), but may be more efficient. * Equivalent to this.bind(args[0]), but may be more efficient. */ public final native MethodHandle invokePartial(Object… args) throws Throwable; } If we do, we will first experiment internally with it (as a non-public API). invokePartial is in fact, invokedynamic + insertArguments + insertArguments (to insert 0) + asType(asVarargCollector) + escape analysis (to remove the array allocation and boxing). I would prefer a way to gently ask the compiler to substitute a classical Java call to an invokedynamic one. The question is: are VMs able to do the escape analysis here or not. BTW, invokePartial is important because it is the primitive needed when you want to create a lambda that capture some values from the scope so even Java will use it. — John Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: build error on OSX
On 07/22/2012 11:38 AM, Henri Gomez wrote: Hi to all, Even using Java 8 for building MLVM, using stephen scripts or my own, build failed : # Running javac: 43 files; in /Users/henri/Downloads/mlvm/sources/jdk/make/java/invoke /Library/Java/JavaVirtualMachines/1.8.0.jdk/Contents/Home/bin/java -XX:-PrintVMOptions -XX:+UnlockDiagnosticVMOptions -XX:-LogVMOutput -Djava.awt.headless=true -Xmx512m -Xms512m -XX:PermSize=32m -XX:MaxPermSize=160m -Xbootclasspath/p:/Users/henri/Downloads/mlvm/sources/build/macosx-x86_64/langtools/dist/bootstrap/lib/javac.jar -jar /Users/henri/Downloads/mlvm/sources/build/macosx-x86_64/langtools/dist/bootstrap/lib/javac.jar -Xlint:-path -source 7 -target 7 -encoding ascii -Xbootclasspath:/Users/henri/Downloads/mlvm/sources/build/macosx-x86_64/classes -sourcepath /Users/henri/Downloads/mlvm/sources/build/macosx-x86_64/gensrc:::/Users/henri/Downloads/mlvm/sources/jdk/src/macosx/classes:/Users/henri/Downloads/mlvm/sources/jdk/src/solaris/classes:/Users/henri/Downloads/mlvm/sources/jdk/src/share/classes -d /Users/henri/Downloads/mlvm/sources/build/macosx-x86_64/classes @/Users/henri/Downloads/mlvm/sources/build/macosx-x86_64/tmp/java/java.lang.invoke/.classes.list.filtered ../../../src/share/classes/java/lang/invoke/MethodType.java:996: error: cannot find symbol private final ConcurrentHashMapEntry, Entry map = new ConcurrentHashMapEntry, Entry(); ^ symbol: class ConcurrentHashMap location: class WeakInternSet ../../../src/share/classes/java/lang/invoke/MethodType.java:996: error: cannot find symbol private final ConcurrentHashMapEntry, Entry map = new ConcurrentHashMapEntry, Entry(); ^ symbol: class ConcurrentHashMap location: class WeakInternSet 2 errors make[4]: *** [.compile.classlist] Error 1 make[3]: *** [all] Error 1 make[2]: *** [all] Error 1 make[1]: *** [jdk-build] Error 2 make: *** [build_product_image] Error 2 *** create symbolic link to last build and persist 1.8.0_00-mlvm-2012_07_22.jdk into .last_build build/macosx-x86_64/j2sdk-bundle/jdk1.8.0.jdk/Contents/Home/bin/java does not exist, rebuild MLVM with ./update.sh Any ideas ? add import java.util.concurrent.ConcurrentHashMap to java/lang/invoke/MethodType.java. Thanks cheers, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: build error on OSX
On 07/22/2012 07:25 PM, Henri Gomez wrote: Hum, commited without this import ? maybe lost in the patch queue ?? Rémi Le 22 juil. 2012 à 15:46, Rémi Forax fo...@univ-mlv.fr a écrit : On 07/22/2012 11:38 AM, Henri Gomez wrote: Hi to all, Even using Java 8 for building MLVM, using stephen scripts or my own, build failed : # Running javac: 43 files; in /Users/henri/Downloads/mlvm/sources/jdk/make/java/invoke /Library/Java/JavaVirtualMachines/1.8.0.jdk/Contents/Home/bin/java -XX:-PrintVMOptions -XX:+UnlockDiagnosticVMOptions -XX:-LogVMOutput -Djava.awt.headless=true -Xmx512m -Xms512m -XX:PermSize=32m -XX:MaxPermSize=160m -Xbootclasspath/p:/Users/henri/Downloads/mlvm/sources/build/macosx-x86_64/langtools/dist/bootstrap/lib/javac.jar -jar /Users/henri/Downloads/mlvm/sources/build/macosx-x86_64/langtools/dist/bootstrap/lib/javac.jar -Xlint:-path -source 7 -target 7 -encoding ascii -Xbootclasspath:/Users/henri/Downloads/mlvm/sources/build/macosx-x86_64/classes -sourcepath /Users/henri/Downloads/mlvm/sources/build/macosx-x86_64/gensrc:::/Users/henri/Downloads/mlvm/sources/jdk/src/macosx/classes:/Users/henri/Downloads/mlvm/sources/jdk/src/solaris/classes:/Users/henri/Downloads/mlvm/sources/jdk/src/share/classes -d /Users/henri/Downloads/mlvm/sources/build/macosx-x86_64/classes @/Users/henri/Downloads/mlvm/sources/build/macosx-x86_64/tmp/java/java.lang.invoke/.classes.list.filtered ../../../src/share/classes/java/lang/invoke/MethodType.java:996: error: cannot find symbol private final ConcurrentHashMapEntry, Entry map = new ConcurrentHashMapEntry, Entry(); ^ symbol: class ConcurrentHashMap location: class WeakInternSet ../../../src/share/classes/java/lang/invoke/MethodType.java:996: error: cannot find symbol private final ConcurrentHashMapEntry, Entry map = new ConcurrentHashMapEntry, Entry(); ^ symbol: class ConcurrentHashMap location: class WeakInternSet 2 errors make[4]: *** [.compile.classlist] Error 1 make[3]: *** [all] Error 1 make[2]: *** [all] Error 1 make[1]: *** [jdk-build] Error 2 make: *** [build_product_image] Error 2 *** create symbolic link to last build and persist 1.8.0_00-mlvm-2012_07_22.jdk into .last_build build/macosx-x86_64/j2sdk-bundle/jdk1.8.0.jdk/Contents/Home/bin/java does not exist, rebuild MLVM with ./update.sh Any ideas ? add import java.util.concurrent.ConcurrentHashMap to java/lang/invoke/MethodType.java. Thanks cheers, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: State of MLVM and Lambda
On 07/20/2012 11:25 AM, Roman Kennke wrote: Hello, Maybe this is a stupid beginner question. I would like to know what is the status of MLVM with respect to project Lambda. Is it possible to build a JVM that has both (has lambda, or parts of it, already been merged into hotspot-comp?)? Is there any overlap between the two, or are they largely independent (MLVM mostly hotspot-centric vs. Lambda mostly javac centric?)? Cheers, Roman Hi Roman, there are two parts in lambda project, you have lambda and default methods, lambda currently doesn't require runtime support (only javac + JDK) but default methods are resolved by the VM and as far as I know, the default methods part is not merged into hotspot-comp. cheers, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Crash using sun.misc.Unsafe.static
On 07/17/2012 01:06 AM, Krystal Mok wrote: Hi Remi, Looks like it's a trap if you're running this on JDK6's HotSpot VM. If you're running a debug build of the same VM, you should hit an assertion before hitting the ShouldNotReachHere() part. opto/type.cpp:2477 assert(o-is_java_object(), must be java language object); Even though the source code of JDK6u33 is not open sourced, you could try a similar version of HotSpot VM from OpenJDK, namely the hsx/hsx20/baseline branch, or jdk6/jdk6/hotspot. Make a fastdebug build and you should see the assertion. And you're right that this has to do with the PermGen removal project. The difference comes from [1], which is a part of CR 7017732. To be specific, before the 7017732, static fields are stored in the instanceKlass of a Java class; an instanceKlass (or its enclosing klassOopDesc to be exact) is not an Java object. After 7017732, static fields are moved to the tail of java.lang.Class instances, which are Java objects. So to answer your question, you just shouldn't expect the the code to work in JDK6/HotSpot. Hi Krys, Too bad because I wanted to use a similar code in the jsr292 backport (a Java agent that enable dynamic language runtime written with invokedynamic to run on a jdk6 stock VM) This remember me another question, the java agent use java.lang.instrument.Instrumentation.retransform to change code at runtime and it seems that using retransform disable OSR on the retransformed class. Is there a reason for that ? [1]: http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/diff/c7f3d0b4570f/src/share/vm/prims/unsafe.cpp Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Crash using sun.misc.Unsafe.static
On 07/18/2012 12:55 AM, John Rose wrote: On Jul 16, 2012, at 4:06 PM, Krystal Mok wrote: And you're right that this has to do with the PermGen removal project. The difference comes from [1], which is a part of CR 7017732. To be specific, before the 7017732, static fields are stored in the instanceKlass of a Java class; an instanceKlass (or its enclosing klassOopDesc to be exact) is not an Java object. After 7017732, static fields are moved to the tail of java.lang.Class instances, which are Java objects. So to answer your question, you just shouldn't expect the the code to work in JDK6/HotSpot. The API for unsafe access to statics is designed to allow implementations to do one of two things: 1. store a static variable at a fixed offset within a managed object (addressable via Java references) 2. store a static variable at an arbitrary but fixed 64-bit VA (in which case the object base is just null) The JDK 6 system is in a middle state, where the instanceKlass is a managed object and can move is not compatible with Java references. I think there was a time when non-Java oops could be manipulated via Java reference variables, but the practice has always been rather… unsafe. This could be fixed in JDK 6, but it's probably not worth it. I recommend spinning bytecoded accessors for statics on JDK 6. On JDK 7 and later, the Class is the base (as Kris pointed out). This is probably how it's going to stay. I spin bytecode by default if the field is visible from the callsite and wanted to use unsafe with a constant base object if the field is not visible. So to workaround that bug, I store the base object in a non-constant field exactly like reflection does, so it works on jdk6. Exactly, I should try to detect if the VM is hotspot or not because neither JRockit nor J9 have the same issue. Thanks for the report. — John P.S. For multi-tenancy VMs, the addressing arithmetic for statics would need to take into account the task ID. But the above design (baseOrNull + longOffset) still works, since the unsafe API doesn't say how the two components get combined. Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Crash using sun.misc.Unsafe.static
Hi guys, the latest jdk6 (jdk6 update 33) fails hard in the compiler if I try to store the result of unsafe.staticFieldBase() in a static final field, i.e if it's considered as a constant. Neither recent jdk7 nor jdk8 crash, only jdk6 (it also failed with update 21 which is the oldest version I have). I think it doesn't crash with recent VM because the result of unsafe.staticFieldBase() is not a real object with jdk6 but is a real one with jdk7/jdk8 (because of permgen removal) Is this bug can be fixed or should I just do not do that ? cheers, Rémi package sun.misc; import java.lang.reflect.Field; public class UnsafeCrash { private static int value = 1; private static final Unsafe UNSAFE; private static final Object STATIC_FIELD_BASE; private static final long STATIC_FIELD_OFFSET; static { Field field; try { field = UnsafeCrash.class.getDeclaredField(value); } catch (NoSuchFieldException e) { throw new AssertionError(e); } Unsafe unsafe = Unsafe.getUnsafe(); UNSAFE = unsafe; STATIC_FIELD_BASE = unsafe.staticFieldBase(field); STATIC_FIELD_OFFSET = unsafe.staticFieldOffset(field); } private static int test() { return UNSAFE.getInt(STATIC_FIELD_BASE, STATIC_FIELD_OFFSET); } public static void main(String[] args) { int sum = 0; for(int i=0; i10; i++) { sum += test(); } System.out.println(sum); } } [forax@localhost src]$ /usr/jdk/jdk1.6.0_33/bin/java -Xbootclasspath/p:. sun.misc.UnsafeCrash # # A fatal error has been detected by the Java Runtime Environment: # # Internal Error (type.cpp:2520), pid=26155, tid=139712055437056 # Error: ShouldNotReachHere() [thread 139712054384384 also had an error]# # JRE version: 6.0_33-b04 # Java VM: Java HotSpot(TM) 64-Bit Server VM (20.8-b03 mixed mode linux-amd64 compressed oops) # An error report file with more information is saved as: # /home/forax/java/workspace/java-blog/src/hs_err_pid26155.log 10 # # If you would like to submit a bug report, please visit: # http://java.sun.com/webapps/bugreport/crash.jsp # Aborted # # A fatal error has been detected by the Java Runtime Environment: # # Internal Error (type.cpp:2520), pid=26155, tid=139712055437056 # Error: ShouldNotReachHere() # # JRE version: 6.0_33-b04 # Java VM: Java HotSpot(TM) 64-Bit Server VM (20.8-b03 mixed mode linux-amd64 compressed oops) # If you would like to submit a bug report, please visit: # http://java.sun.com/webapps/bugreport/crash.jsp # --- T H R E A D --- Current thread (0x7f1160093800): JavaThread C2 CompilerThread0 daemon [_thread_in_native, id=26169, stack(0x7f113f5f6000,0x7f113f6f7000)] Stack: [0x7f113f5f6000,0x7f113f6f7000], sp=0x7f113f6f3860, free space=1014k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.so+0x85f725] VMError::report_and_die()+0x265 V [libjvm.so+0x3e4dd6] report_should_not_reach_here(char const*, int)+0x46 V [libjvm.so+0x830041] TypeOopPtr::make_from_constant(ciObject*, bool)+0x2a1 V [libjvm.so+0x73bcd5] Parse::push_constant(ciConstant, bool)+0x385 V [libjvm.so+0x73b628] Parse::do_get_xxx(TypePtr const*, Node*, ciField*, bool)+0x3e8 V [libjvm.so+0x73b152] Parse::do_field_access(bool, bool)+0x142 V [libjvm.so+0x73a926] Parse::do_one_bytecode()+0x68d6 V [libjvm.so+0x72b07a] Parse::do_one_block()+0x1aa V [libjvm.so+0x728ad0] Parse::do_all_blocks()+0x150 V [libjvm.so+0x72860b] Parse::Parse(JVMState*, ciMethod*, float)+0x59b V [libjvm.so+0x322064] ParseGenerator::generate(JVMState*)+0x84 V [libjvm.so+0x39e7e2] Compile::Compile(ciEnv*, C2Compiler*, ciMethod*, int, bool, bool)+0x762 V [libjvm.so+0x3217de] C2Compiler::compile_method(ciEnv*, ciMethod*, int)+0x9e V [libjvm.so+0x3a914a] CompileBroker::invoke_compiler_on_method(CompileTask*)+0x2ca V [libjvm.so+0x3a8a35] CompileBroker::compiler_thread_loop()+0x355 V [libjvm.so+0x8208c9] compiler_thread_entry(JavaThread*, Thread*)+0x9 V [libjvm.so+0x819dd1] JavaThread::run()+0x121 V [libjvm.so+0x7117af] java_start(Thread*)+0x13f Current CompileTask: C2: 57 1 sun.misc.UnsafeCrash.test()I (13 bytes) --- P R O C E S S --- Java Threads: ( = current thread ) 0x7f1160098800 JavaThread Low Memory Detector daemon [_thread_blocked, id=26171, stack(0x7f113f3f4000,0x7f113f4f5000)] 0x7f1160096000 JavaThread C2 CompilerThread1 daemon [_thread_in_native, id=26170, stack(0x7f113f4f5000,0x7f113f5f6000)] =0x7f1160093800 JavaThread C2 CompilerThread0 daemon [_thread_in_native, id=26169, stack(0x7f113f5f6000,0x7f113f6f7000)] 0x7f1160091000 JavaThread Signal Dispatcher daemon [_thread_blocked, id=26168, stack(0x7f113f6f7000,0x7f113f7f8000)] 0x7f1160074800 JavaThread Finalizer daemon [_thread_blocked, id=26167, stack(0x7f113f7f8000,0x7f113f8f9000)]
Re: review request (L): JDK changes for 7023639: JSR 292 method handle invocation needs a fast path for compiled code
On 07/14/2012 02:02 PM, David Schlosnagle wrote: On Fri, Jul 13, 2012 at 5:41 AM, John Rose john.r.r...@oracle.com wrote: On Jul 11, 2012, at 5:53 PM, John Rose wrote: As some of you have noticed, Chris Thalinger, Michael Haupt, and I have been working on the mlvm patches [1] for JEP-160 [2] for several months. These changes make method handles more optimizable. They refactor lots of magic out of the JVM and into more manageable Java code. … An associated webrev for hotspot-comp/jdk/ will be posted soon; it is already present on mlvm-dev for the curious to examine. (This change set also deletes a lot of old code.) Here is that webrev: http://cr.openjdk.java.net/~jrose/7023639/webrev.jdk.00/ These are the changes to JDK code that accompany the JVM changes already under review. There are 2900 LOC deleted, and 7000 LOC added. Key changes: - method handle behavior is fully represented by LambdaForm objects - chained method handles (including adapter method handles) are gone - an ASM-based bytecode spinner compiles LambdaForms when they warm up - bound method handles are compactly represented without boxing - the private symbol-resolution interface to the JVM (MemberName) is improved - unit tests have more systematic coverage - a number of minor bugs are fixed This is implementation work. No public Java APIs are changed, although the javadoc is slightly edited for clarity. Please have a look. — John John, Hi David, I hijack the thread ... I had a few quick comments on your webrev. Some of these might be considered premature or unnecessary optimization, so take them as you wish. src/share/classes/java/lang/invoke/BoundMethodHandle.java On lines 131-132: Just out of curiosity, why use pos+1 in one line versus 1+pos on the next? Would it be worthwhile to extract pos+1 to a local int end to save some bytecode? MethodHandle bindArgument(int pos, char basicType, Object value) { int end = pos + 1; MethodType type = type().dropParameterTypes(pos, end); LambdaForm form = internalForm().bind(basicType, end, tcount()); if (basicType == 'I' !(value instanceof Integer)) { // Cf. ValueConversions.unboxInteger value = ValueConversions.primitiveConversion(Wrapper.INT, value, true); } return cloneExtend(type, form, basicType, value); } Usually you should not care about this kind of change, micro-optimizing is useless. Or this code is not called enough and you don't care or this code is hot and the JIT will do what should be done and because the JIT may inline the code of dropParameterTypes and bind(), it will better optimize that you because it will be able to share more sub-expressions. Now just for fun, if you count the number of bytecodes, I think your proposed code is bigger because 'end' is the local variable 4 and the is no istore_4/iload_4 so the compiler will use the generic iload/istore which takes 2 bytes. so pos + 1 is iload_1, iconst_1, iadd, so the current version use 6 bytecodes, your version use iload_1, iconst_1, iadd, istore 4 + iload 4 * 2= 9 bytecodes On lines 202-212: Similarly would it be worthwhile to extract types() to a local String within internalValues()? final Object internalValues() { String types = types(); Object[] boundValues = new Object[types.length()]; for (int i = 0; i boundValues.length; ++i) { try { switch (types.charAt(i)) { case 'L': boundValues[i] = argL(i); break; case 'I': boundValues[i] = argI(i); break; case 'F': boundValues[i] = argF(i); break; case 'D': boundValues[i] = argD(i); break; case 'J': boundValues[i] = argJ(i); break; default : throw new InternalError(unexpected type: + types.charAt(i)); } } catch (Throwable t) { throw new InternalError(t); } } return Arrays.asList(boundValues); } default case should never be called, so don't 'optimize' for it. On line 464: Is there any reason CACHE should not be final? static final MapString, Data CACHE = new IdentityHashMap(); this one may be important, static final = const for the VM On lines 473-486: Do the accesses and mutations of CACHE need to be thread safe or are the uses assumed to be safe by other means? On lines 778-784: Should this use types.charAt(int) rather than toCharArray() to copy the array? private static String makeSignature(String types, boolean ctor) { StringBuilder buf = new StringBuilder(SIG_INCIPIT); for (int i = 0; i types.length(); ++i) { buf.append(typeSig(types.charAt(i))); } return buf.append(')').append(ctor ? V : BMH_SIG).toString();
Re: hg: mlvm/mlvm/hotspot: meth: fix to intrinsic compilation; assembler tweaks for debugging; backtrace filtering
On 07/12/2012 07:14 PM, Charles Oliver Nutter wrote: On Thu, Jul 12, 2012 at 2:11 AM, John Rose john.r.r...@oracle.com wrote: One problem with mixed mode systems is that a method like executeMyLanguagesInterpretedMethodUntilItGetsCompiled wants to display a method name based on some value inside the frame itself. Simply filtering or renaming JVM methods does not cover this case at all. This is indeed the problem we face in JRuby. The Hidden annotation might help us hide JRuby internals used for call-to-call plumbing, but there's no static way to solve the problem of generating a trace for interpreter frames. I agree, but it will greatly simplify the problem because stacktrace will be smaller so it will be faster to patched them. Here's part of a Java trace from JRuby, with both compiled and interpreted bits in it: Trace snippit: at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:135) at org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:63) at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104) at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:75) at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:139) at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:175) at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:566) at org.jruby.runtime.invokedynamic.InvocationLinker.invocationFallback(InvocationLinker.java:149) at blah.method__0$RUBY$foo(blah.rb:4) at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:566) at org.jruby.runtime.invokedynamic.InvocationLinker.invocationFallback(InvocationLinker.java:149) at blah.__file__(blah.rb:13) at blah.load(blah.rb) To generate a Ruby backtrace from this, we use the Java backtrace as a master and mine out the interesting pieces. Specifically: 1. Elements with RUBY and __file__ in them are included because they represent compiled Ruby code. 2. Any Java-based Ruby method in the trace is included, using a list we maintain in JRuby as those methods are bound. 3. Interpreter entry points are indicated by INTERPRET_METHOD, INTERPRET_EVAL, and so on. For each such element, we insert backtrace data from our interpreter's backtrace stack (on heap) This works well for Ruby code but does often omit frames users *want* to see, like reflection-called Java methods. Hiding would work for removing elements, but we'd have to tag a large amount of JRuby code (like the entire interpreter) to remove everything. Having an annotation to indicate the actual method name would be useful for (2) above, but it would not do anything to help us dynamically determine the frame for an interpreted section. While I'd love to see better programmatic access to backtrace generation, I can't really envision what it would look like. A special method handle acting as a decorator should be enough :) Let say we have a static method backtraceDecorator in MethodHandles that takes a target method handle and returns a new method handle. The new method handle will have a supplementary parameter of type String before all parameters, the content of this parameter will be used instead of the method name of the target method in the stack trace. So basically, the handle handle will change the method name of the method handle taken as argument, so the target method handle must be either a constant method handle or the result of a lookup.find* or lookup.unreflect*. Now in the runtime instead of calling INTERPRET_METHOD directly, the runtime should call the method handle created like this: MethodHandle interpreterEntryPoint = MethodHandles.backtraceDecorator( lookup.findVirtual(..., ''INTERPRET_METHOD)); and this method handle should be called with the method name as first parameter. On the plus side...the requirement that we always assemble our backtraces from bits and bobs means we can do fun things like alternative formatting and adding color :) http://min.us/mbIkONLut I dare say JRuby has the nicest stack traces of any language on the JVM. - Charlie cheers, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: hg: mlvm/mlvm/hotspot: meth: fix to intrinsic compilation; assembler tweaks for debugging; backtrace filtering
On 07/10/2012 12:10 PM, Guillaume Laforge wrote: It'd be useful for others as well :-) Yes, for JRuby which is mixed mode (AST walker + bytecode compiler), you still have to do some post processing but for Groovy which is compiled only, I think you can avoid any processing at runtime if you post process the groovy.jar at compile time (because Hidden is not a public annotation) :) Rémi On Tue, Jul 10, 2012 at 11:12 AM, Rémi Forax fo...@univ-mlv.fr mailto:fo...@univ-mlv.fr wrote: On 07/10/2012 10:42 AM, john.r.r...@oracle.com mailto:john.r.r...@oracle.com wrote: Changeset: 851ca5831589 Author:jrose Date: 2012-07-10 01:42 -0700 URL: http://hg.openjdk.java.net/mlvm/mlvm/hotspot/rev/851ca5831589 meth: fix to intrinsic compilation; assembler tweaks for debugging; backtrace filtering ! meth-lazy-7023639.patch I think backtrace filtering, should be enhanced by adding a new method attribute in the bytecode that indicates if a method should be visible or not when dumping a stacktrace and a display name that should be used instead of the bytecode method name. This may reduce the need of stacktrace post-processing and makes Charles happy. Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net mailto:mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev -- Guillaume Laforge Groovy Project Manager SpringSource, a division of VMware Blog: http://glaforge.appspot.com/ Social: @glaforge http://twitter.com/glaforge / Google+ https://plus.google.com/u/0/114130972232398734985/posts ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: hg: mlvm/mlvm/jdk: meth: add @Hidden annotations
On 07/11/2012 08:19 AM, john.r.r...@oracle.com wrote: Changeset: 77f38b00af98 Author:jrose Date: 2012-07-10 23:19 -0700 URL: http://hg.openjdk.java.net/mlvm/mlvm/jdk/rev/77f38b00af98 meth: add @Hidden annotations ! meth-lazy-7023639.patch ! meth-lazy-7023639.xbmh.patch just a question, even if it's not a big deal if one is able to generate bytecode, why Hidden is not a top level type of java.lang.invoke ? Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: hg: mlvm/mlvm/jdk: meth: add @Hidden annotations
On 07/11/2012 10:24 PM, John Rose wrote: On Jul 11, 2012, at 2:29 AM, Rémi Forax wrote: just a question, even if it's not a big deal if one is able to generate bytecode, why Hidden is not a top level type of java.lang.invoke ? Good question. The answer is that hiding of stack frames is limited to hiding method handle implementation details. Specifically, the frames that are to be hidden are those which are not specified by the JSR 292 modifications to the JVMS. For example, if you have z=filterReturnValue(::x,::y), and some method m does z.invoke(), the visible stack frames will be m calling x and m calling y. Internally there will be a continuation during the call to x which waits to apply y. This must be some sort of activation frame, but the JVMS requires it to be hidden. In JDK 7 it was a ricochet frame (hand assembly code) and in JDK 8 it will be a lambda form activation (bytecode). Both must be suppressed in backtraces. A general Lisp-machine-style convention for redacting back traces would be interesting, but it would also require a new JSR. On the other hand, Hidden could be top-level. It's a matter of coding taste. Binding it to LF makes it more clear that it has a specific use. Emulating a language call using bytecode is something that should ring a bell for a lot of people of this list :) And even for the JDK, it by example generates bytecode when optimizing a call to j.l.r.Method and this bytecode should not appear in the stacktrace. — John Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: hg: mlvm/mlvm/jdk: meth-lazy: merged BMH factory into BMH, added test for primitive parameter binding
On 07/12/2012 12:10 AM, Michael Haupt wrote: Hi Remy, Am 11.07.2012 um 22:48 schrieb Rémi Forax: Hi Mickael, some minor comments, makeSignature should use a StringBuilder and not a StringBuffer. good point, thanks! ans I don't think generateConcreteBMHClass needs to return a Class? extends BoundMethodHandle, a Class? seems enough so the cast near the end of the method can be removed. I'd like to keep that to make really clear what Class type instances are expected. (There are places where Class? would just feel very wrong.) but the cast seems wrong to me because this one is not enforced by the VM so if there is a bug, you may lure the reader of the code. I think a doc comment is better here. Or perhaps you can use asSubClass() ? Best, Michael regards, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: hg: mlvm/mlvm/hotspot: meth: fix to intrinsic compilation; assembler tweaks for debugging; backtrace filtering
On 07/10/2012 10:42 AM, john.r.r...@oracle.com wrote: Changeset: 851ca5831589 Author:jrose Date: 2012-07-10 01:42 -0700 URL: http://hg.openjdk.java.net/mlvm/mlvm/hotspot/rev/851ca5831589 meth: fix to intrinsic compilation; assembler tweaks for debugging; backtrace filtering ! meth-lazy-7023639.patch I think backtrace filtering, should be enhanced by adding a new method attribute in the bytecode that indicates if a method should be visible or not when dumping a stacktrace and a display name that should be used instead of the bytecode method name. This may reduce the need of stacktrace post-processing and makes Charles happy. Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: How to making Class.forName work in indy?
On 07/10/2012 10:07 PM, Jochen Theodorou wrote: Hi all, assuming you have to compile code with indy that realizes this: Class.forName(x) Meaning, we want to execute forName from Class using invokedynamic. If you then have frames in your trace looking like this: java.lang.Class.forName0(Native Method) java.lang.Class.forName(Class.java:186) java.lang.invoke.MethodHandleImpl$GuardWithCatch.invoke_L2(MethodHandleImpl.java:1130) org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:680) groovy.util.JavadocAssertionTestSuite.clinit(JavadocAssertionTestSuite.groovy:71) then it is clear to me, why it is failing. At least the Indyinterface part, if not even the GuradWithCatch part are disturbing the stack walking Class#forName is using. has anyone any idea on how to make this work across JVMs? One version, that sounds quite crazy to me, but may just work, is to compile a helper method on the class, that directly calls Class#forName and thus making it work, since there is no longer a wrong stack frame at the position Class#forName is looking for. But well... it may be a possibility bye Jochen Class.forName(name) is equivalent to Class.forName(name, true, lookup.lookupClass().getClassLoader()) As true and lookup.lookupClass().getClassLoader() are constant in the BSM, you can bound them using insertArguments. Rémi || ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: TaggedArrays (Proposal)
On 07/07/2012 04:54 AM, Mark Roos wrote: Hi Rémi, you mention And now the trick, there is a nice way (several in fact) to explain to the JIT that even if the bytecode contains tests, if the variable contains effectively an int, it's a good idea to remove them. Ok, in Smalltalk there are some methods which are usually integer ops so its easy to determine vars that are likely integer. I have slots in the stack reserved for just this idea. So if I have a pair of slots A and B where A would be the integer from and B the reference. Would I test B to be null and if so do an integer op on A? yes, that the basic idea. So plese point me at the bytecode tricks that make the test go away. It's not a bytecode trick, it's a JIT trick. The VM profiles jump instruction like 'if' to know which branch is taken, and doesn't generate code but a deoptimization code if a branch is never taken. So the generator of a backend will generate a code like this for a + 1: int r1 = ... Object o1 = ... if (o1 == null) { r2 = r1 + 1 o2 = null } else { r2 = 0 o2 = invokedynamic + o1, 1 } if (o2 == null) { // next instruction ... } else { ... ... and because o1 is never null, the JIT will generate jne a_deoptimization_code inc r2 also because the JIT propagates null value aggressively, the jump can also disappear because there is perhaps already another check before, by example, the check o2 == null will be removed here. So the idea is to consider that if a variable can store an int, you should use two variables in the bytecode, so result of an operation will be spilled into two variables instead of using the stack. I call this idea, split-path and this is really effective, the main drawback is that the generated bytecode size is big compared to the code of the generated assembler so the JIT may decide to not inline something that it should. You have also to figure out how to get two return values from a method call, but exceptions are your best friend here. regards mark rgds, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: TaggedArrays (Proposal)
On 07/07/2012 07:02 PM, Dain Sundstrom wrote: On Jul 7, 2012, at 1:56 AM, Rémi Forax wrote: You have also to figure out how to get two return values from a method call, but exceptions are your best friend here. Can you give an example of what you mean here? Also, from all the presentations I've seen on the JVM, exceptions are very expensive to throw and catch, so I would expect this to be way slower then say returning the multiple values using a generated results class. Exception are not expensive if you throw them and just catch them (and don't use them) in the same inlining horizon, so you can use them to implement non Java control flow without thinking too much. Anyway, the idea here is to use exception as an exceptional mechanism by example when something that should be an int is not. In that case, throwing an exception is not a big deal because the VM will have to deopt which usual have a bigger impact on the run time. -dain Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Improving the performance of stacktrace generation
On 07/08/2012 12:03 AM, Charles Oliver Nutter wrote: Today I have a new conundrum for you all: I need stack trace generation on Hotspot to be considerably faster than it is now. In order to simulate many Ruby features, JRuby (over)uses Java stack traces. We recently (JRuby 1.6, about a year ago) moved to using the Java stack trace as the source of our Ruby backtrace information, mining out compiled frames and using interpreter markers to peel off interpreter frames. The result is that a Ruby trace with mixed compiled and interpreted code like this (https://gist.github.com/3068210) turns into this (https://gist.github.com/3068213). I consider this a great deal better than the plain Java trace, and I know other language implementers have lamented the verbosity of stack traces coming out of their languages. The unfortunate thing is that stack trace generation is very expensive in the JVM, and in order to generate normal exceptions and emulate other features we sometimes generate a lot of them. I think there's value in exploring how we can make stack trace generation cheaper at the JVM level. Here's a few cases in Ruby where we need to use Java stack traces to provide the same features: * Exceptions as non-exceptional or moderately-exceptional method results In this case I'm specifically thinking about Ruby's tendency to propagate errno values as exceptions; EAGAIN/EWOULDBLOCK for example are thrown from nonblocking IO methods when there's no data available. You will probably say that's a horrible use for exceptions and I agree. But there are a couple reasons why it's nicer too: - using return value sigils requires you to propagate them back out through many levels of calls - exception-handling is cleaner in code than having all your errno handling logic spliced into regular program flow In any case, the cost of generating a stack trace for potentially every non-blocking IO call is obviously too high. In JRuby, we default to having EAGAIN/EWOULDBLOCK exceptions not generate a stack trace, and you must pass a flag for them to do so. The justification is that these exceptions are almost always used to branch back to the top of a nonblocking IO loop, so the backtrace is useless. I don't see how to do more. * Getting the current or previous method's name/file/line Ruby supports a number of features that allow you to get basic information about the method currently being executed or the method that called it. The most general of these features is the caller method, which provides an array of all method name + file + line that would appear in a stack trace at this point. This feature is often abused to get only the current or previous frame, and so in Ruby 1.9 they added __method__ to get the currently-executing method's name+file+line. In both cases, we must generate a full Java trace for these methods because the name of a method body is not necessarily statically known. We often want only the current frame or the current and previous frames, but we pay the cost of generating an entire Java stack trace to get them. You can use Throwable.getStackTraceElement() which is package visible and OpenJDK specific but at least it will be faster for all VMs that uses OpenJDK. * Warnings that actually report the line of code that triggered them In Ruby, it is possible to generate non-fatal warnings to stderr. In many cases, these warnings automatically include the file and line at which the triggering code lives. Because the warning logic is downstream from the Ruby code, we again must use a full Java stack trace to find the most recent (on stack) Ruby frame. This causes warnings to be as expensive as regular exceptions. Please never optimize warnings, they are here to bug users until they fix the thing. So they should be slow :) Because the use of frame introspection (in this case through stack traces) has largely been ignored on current JVMs, I suspect there's a lot of improvement possible. At a minimum, the ability to only grab the top N frames from the stack trace could be a great improvement (Hotspot even has flags to restrict how large a trace it will generate, presumably to avoid the cost of accounting for deep stacks and generating traces from them). Any thoughts on this? Does anyone else have need for lighter-weight name/file/line inspection of the call stack? - Charlie Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Improving the performance of stacktrace generation
On 07/08/2012 12:50 AM, Charles Oliver Nutter wrote: On Saturday, July 7, 2012, Rémi Forax wrote: You can use Throwable.getStackTraceElement() which is package visible and OpenJDK specific but at least it will be faster for all VMs that uses OpenJDK. I'll certainly explore that to see if it improves the situation. If it's faster, it might be a way out in some circumstances. It seems like an official public API is needed here... Please never optimize warnings, they are here to bug users until they fix the thing. So they should be slow :) Yes, except when *everyone else* has faster warnings than you. Then you're just giving people another reason to not use your stuff. Granted, JRuby is now becoming far and away the fastest Ruby implementation for exactly the reasons that stack traces are slow, but often making 90% of the code 5x faster but 10% of the code 100x slower means people give up on you without bothering. Remember, I agree with you...but I also feel like this attitude has allowed stack trace generation to avoid optimization effort for a long time, and there *are* useful things you can do with an introspectable call stack. Throwable.getStackTraceElement() let you walk the stack trace without generating the whole stack trace, in my opinion it's enough. BTW, I also think you are responsible too for the cost of generating stacktraces because the cost depends on the size of the stacktrace and an AST interpreter is a big consumer of stack frames. I am guilty too, PHP.reboot has the same issue and I try to mitigate that by avoiding to have an helper methods in the stacktrace of calls (not that easy without tailcall). - Charlie Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: TaggedArrays (Proposal)
On 07/06/2012 08:48 PM, Mark Roos wrote: From Rémi on static analysis for loops Not having such kind of analysis is almost a crime. For a language like Smalltalk I was thinking that having such an analysis would be the work of the gods. With user overridable methods, reflection and run time code creation I have only found a few places where I could do such an static analysis. And those are fragile. So do you have a good paper that would apply? I have looked at many from the 80s/90s and have yet to find a general approach for a Smalltalk style language. I am thinking of adding a mutable flag to integers so I can share a body but 'lazy' freeze to a real integer but it seems like I'll be adding lots of checks just to fix what could be a small problem over an entire application. yes, i don't think it's a good idea too because it's too late, you want to avoid boxing not try to share boxes. I also have a thought to allocate a few primitive slots in each stack frame to pass real primitives but again I have to test everywhere for which slot is valid. First the idea is to use an analysis but not to answer to the question, is this variable is an integer or not but to answer to the question is this variable may be an integer or not. Once the analysis done, you know how many supplementary slots you need. And now the trick, there is a nice way (several in fact) to explain to the JIT that even if the bytecode contains tests, if the variable contains effectively an int, it's a good idea to remove them. The best thought may be to add patterns to the language to avoid the use of visible integers as indexes ( e.g. instead of '1 to:3 do:[aBlock over a collection]' do 'aCollection from:1 to:3 do:[ aBlock on each indexed item]') Now I can do the static analysis. As they say 'If it hurts don't do it' regards mark cheers, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Working around NoClassDefFound
On 07/04/2012 03:01 AM, Mark Roos wrote: From Charlie It also doesn't appear to happen with JRuby on the boot classpath...it happens only when JRuby is loaded in a child classloader. That was my experience as well. The only way I found to avoid it was to put my code into the bootClassPath. I messed with other ideas ( but not Rémi's) but the error always eventually returned. regards mark There is only one workaround to avoid the NoClassDefFound error as far as I know, the signature of all invokedynamic calls must not contain any class which is not in the bootclasspath. So either you put the runtime classes in the bootclasspath or you erase all your signature to only use primitive types and Object. cheers, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: TaggedArrays (Proposal)
On 07/04/2012 03:18 AM, Jim Laskey wrote: Actually it's built on a modified 7. Getting it into a 7 release is a community issue. I think tagged values comes under JSR-292, so some of the process is covered. Not sure what the other steps involve. And of course the API/implementation needs to be picked on a bit too. Sent from my iPhone 4 Jim, is there a way to getting it as a patch in the mlvm workspace, so we can test a little bit the API/implementation. Rémi On 2012-07-03, at 10:01 PM, Mark Roos mr...@roos.com mailto:mr...@roos.com wrote: Hi Jim, You made a comment: implementation for platforms not supporting TaggedArrays (and JDK 1.7) Are you saying that a native version fo jdk1.7 is not possible, or just that you have not got around to it? regards mark ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net mailto:mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: two things invokedynamic canbot do
On 06/26/2012 12:37 PM, Jochen Theodorou wrote: Am 26.06.2012 11:59, schrieb Rémi Forax: On 06/26/2012 11:40 AM, Jochen Theodorou wrote: Hi all, just to be sure I understand correctly... afaik there are two things invokedynamic cannot do. (1) calling private methods actually I am not sure this is really true. There is a Lookup instance I can use to get handles to private methods (afaik), therefore it should be possible. Or is there an artificial restriction of some kind? No restriction, or you need a way to get the proper lookup or you use reflection to get a j.l.r.Method, call setAccessible(true) and unreflect it as a MethodHandle (see MethodHandles.unreflect*). ok, so I can remove those stupid helper methods ;) (2) calling super() afaik I cannot generate a call site that replaces the invokeSpecial call to a super class constructor. You can call super.foo() ah true... this is not reflection.. even if I get the handle from the super class it won't call the overriding method in the current class. I totally forgot... how do you handle this in your backport? I don't :( The current plan is to add an empty trampoline method in the code by default and to redefine the code of this method when you create a method handle that use invokespecial. But it only works in agent mode not in reflection only mode. but not super(). It's because the VM verifies that you should not have access to an uninitialized object, ie. access to an object before calling super(). So if you do a super() using a MethodHandle, because it's hidden by an invokedynamic (or a constant method handle) the VM as no way to know that this invokedynamic will call the constructor of the super class, so the verifier will not let you call invokedynamic on this in a constructor without calling explicitly super() before. I see... that is a bit unfortunate. I have to do quite some tricks to enable the super constructor call according to Groovy rules for method dispatch. And what I do I won't call fast. For Dart, I don't generate Dart constructor as Java constructor, I create public default Java constructor that just call super() and translate the Dart constructor to a Java method. Then, a call to new is translated to an invokedynamic that will, at runtime, fold the call to the Java constructor and the call to the method that correspond to the Dart constructor. This allow me to workaround the VM limitation but I don't allow any Dart to Java integration like being able in Java to inherits from a Dart class. bye blackdrag ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: megamorphic lambda prevention
On 06/21/2012 01:57 PM, Krystal Mok wrote: That's the inlining problem that Cliff Click was talking about [1], right? yes, I wonder how well the new interpreter design in Graal would handle this kind of case, since it's supposed to have picked the good parts from trace-based compilation, but without actually having to do tracing. Can't wait to see more details of it at this year's JVM Language Summit. I spend a couple of days in Linz last month with the Autrian part of the Graal team (Thomas, Lukas and Gilles) and we discuss about this issue (among other things). I think we should book a room during the Summit to try to see if something can be done in Hotspot. - Kris Rémi [1]: http://www.azulsystems.com/blog/cliff/2011-04-04-fixing-the-inlining-problem On Thu, Jun 21, 2012 at 6:37 PM, Jochen Theodorou blackd...@gmx.org mailto:blackd...@gmx.org wrote: Hi all, I was wondering... if I have code like this: list.each { x - foo(x) } list.each { x - bar(x) } list.each { x - something(x) } then isn't it the a case where within the each method we easily get something megamorphic, since there are too many different kinds of lambdas involved? Isn't that a general problem with internal iterators and is there any plan to enhance hotspot to counter that problem? bye Jochen -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net mailto:mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: megamorphic lambda prevention
On 06/21/2012 03:52 PM, Jochen Theodorou wrote: Am 21.06.2012 13:57, schrieb Krystal Mok: That's the inlining problem that Cliff Click was talking about [1], right? yes, the issue was actually mentioned more than once on this list already I wonder how well the new interpreter design in Graal would handle this kind of case, since it's supposed to have picked the good parts from trace-based compilation, but without actually having to do tracing. Can't wait to see more details of it at this year's JVM Language Summit. If I understood Thomas right, then Graal doesn't have that problem. Afaik hotspot doesn't have to have that problem too. It just was a design decision not to make call site caching per execution path. Afaik chaning that now would take quite some effort, but is not impossible. You can create the execution path backward, when it's needed. bye blackdrag Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: megamorphic lambda prevention
On 06/21/2012 03:55 PM, Jochen Theodorou wrote: Am 21.06.2012 13:21, schrieb MacGregor, Duncan (GE Energy): Yes, it is very easy for those sites to become megamorphic. We work round this by using exactInvokers on function invocation call sites, and caching on the method type of the functions rather than the types. So in my own words... you don't check on the type of the lambda, but on what it takes as input, thus you get no changes for differing lambdas. Is that about right? That's a nice idea, indeed bye blackdrag at least you will not change the inlining cache too often and you can adapt arguments for several method handles but the VM will not consider the method handle as constant so it will not be inlined with the code of the loop. Another way is to transform the high order function (each() in your example) as a method handle tree but the JSR 292 lacks a loop combiner. In Groovy, you can also duplicate the bytecode of the high order function when you are able to emit bytecode at runtime, see [1]. Said differently and with Hotspot in mind, an idea is that you can execute a code but with another profile metadata object that the usual one. Because Hotspot is already able to generate code with a profile object created by the interpreter (for tiered compilation), having a way to ask for a specialized version of a method with a fresh profile doesn't seem too hard but it's more a hack than a real answer. Rémi [1] http://weblogs.java.net/blog/forax/archive/2011/04/08/fixing-inlining-%E2%80%9Cproblem%E2%80%9D-prototype ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: megamorphic lambda prevention
On 06/21/2012 04:12 PM, Jochen Theodorou wrote: Am 21.06.2012 16:00, schrieb Rémi Forax: On 06/21/2012 03:52 PM, Jochen Theodorou wrote: Am 21.06.2012 13:57, schrieb Krystal Mok: [...] I wonder how well the new interpreter design in Graal would handle this kind of case, since it's supposed to have picked the good parts from trace-based compilation, but without actually having to do tracing. Can't wait to see more details of it at this year's JVM Language Summit. If I understood Thomas right, then Graal doesn't have that problem. Afaik hotspot doesn't have to have that problem too. It just was a design decision not to make call site caching per execution path. Afaik chaning that now would take quite some effort, but is not impossible. You can create the execution path backward, when it's needed. I was thinking that the VM can create the path backward. you can in PHP.reboot, but in Groovy I don't have access to the full execution path. Inside each I don't know what did call me and at the point of where I call each, I don't see what each will actually do. yes, but you can detect that 'each' is a special method because it takes a closure (Closure) as parameter and try to do some magic. bye blackdrag Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: megamorphic lambda prevention
On 06/21/2012 04:38 PM, Matt Fowles wrote: All~ Couldn't the VM detect hot mega-morphic methods and have them bestow some of their hotness upon the callers. Making their callers more likely to inline them. Then you have a different call site for the each which is likely to be monomorphic. Matt yes, that the idea when i said the the VM can go backward. Note that it works only for callsites that are megamorphic because of a value that comes from the parameters of the high order method. There is also a special case for OSR because in that case you may have the stack so it can be easier to go backward. Rémi On Thu, Jun 21, 2012 at 10:30 AM, Rémi Forax fo...@univ-mlv.fr mailto:fo...@univ-mlv.fr wrote: On 06/21/2012 04:12 PM, Jochen Theodorou wrote: Am 21.06.2012 16 tel:21.06.2012%2016:00, schrieb Rémi Forax: On 06/21/2012 03:52 PM, Jochen Theodorou wrote: Am 21.06.2012 13 tel:21.06.2012%2013:57, schrieb Krystal Mok: [...] I wonder how well the new interpreter design in Graal would handle this kind of case, since it's supposed to have picked the good parts from trace-based compilation, but without actually having to do tracing. Can't wait to see more details of it at this year's JVM Language Summit. If I understood Thomas right, then Graal doesn't have that problem. Afaik hotspot doesn't have to have that problem too. It just was a design decision not to make call site caching per execution path. Afaik chaning that now would take quite some effort, but is not impossible. You can create the execution path backward, when it's needed. I was thinking that the VM can create the path backward. you can in PHP.reboot, but in Groovy I don't have access to the full execution path. Inside each I don't know what did call me and at the point of where I call each, I don't see what each will actually do. yes, but you can detect that 'each' is a special method because it takes a closure (Closure) as parameter and try to do some magic. bye blackdrag Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net mailto:mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: megamorphic lambda prevention
On 06/21/2012 04:38 PM, Matt Fowles wrote: All~ Couldn't the VM detect hot mega-morphic methods and have them bestow some of their hotness upon the callers. Making their callers more likely to inline them. Then you have a different call site for the each which is likely to be monomorphic. Matt yes, that the idea when i said the the VM can go backward. Note that it works only for callsites that are megamorphic because of a value that comes from the parameters of the high order method. There is also a special case for OSR because in that case you may have the stack so it can be easier to go backward. Rémi On Thu, Jun 21, 2012 at 10:30 AM, Rémi Forax fo...@univ-mlv.fr mailto:fo...@univ-mlv.fr wrote: On 06/21/2012 04:12 PM, Jochen Theodorou wrote: Am 21.06.2012 16 tel:21.06.2012%2016:00, schrieb Rémi Forax: On 06/21/2012 03:52 PM, Jochen Theodorou wrote: Am 21.06.2012 13 tel:21.06.2012%2013:57, schrieb Krystal Mok: [...] I wonder how well the new interpreter design in Graal would handle this kind of case, since it's supposed to have picked the good parts from trace-based compilation, but without actually having to do tracing. Can't wait to see more details of it at this year's JVM Language Summit. If I understood Thomas right, then Graal doesn't have that problem. Afaik hotspot doesn't have to have that problem too. It just was a design decision not to make call site caching per execution path. Afaik chaning that now would take quite some effort, but is not impossible. You can create the execution path backward, when it's needed. I was thinking that the VM can create the path backward. you can in PHP.reboot, but in Groovy I don't have access to the full execution path. Inside each I don't know what did call me and at the point of where I call each, I don't see what each will actually do. yes, but you can detect that 'each' is a special method because it takes a closure (Closure) as parameter and try to do some magic. bye blackdrag Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net mailto:mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: megamorphic lambda prevention
On 06/21/2012 04:38 PM, Matt Fowles wrote: All~ Couldn't the VM detect hot mega-morphic methods and have them bestow some of their hotness upon the callers. Yes, that's basically the idea when I say the VM can create a path backward. Making their callers more likely to inline them. only if the receiver of the call depends on parameters. Also, the VM should not trust the profile but the propagated types from the parameters. Then you have a different call site for the each which is likely to be monomorphic. yes; Also in case of OSR, the VM has more info because it knows the call stack, so going backward can be easier. Matt Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: megamorphic lambda prevention
On 06/21/2012 09:17 PM, Jochen Theodorou wrote: Mark, can you explain what you mean with depth? if you mean the depth of a call path, then more than 20 is indeed more rare, but it depends on the circumstances. In Grails for example (web frame work in the Groovy world) a depth of 20 should be a much more commen case, simply because there is a framework around it, that already has the usual call path depth. As for seeing depth of the call path from the point the blokc is called to the point the iterator method is called, that is usually a quite short path of maybe depth 3. But maybe you something else by depth. No, I think here depth means depth of the guardWithTest, i.e. number of different receiver types. Anyway, the goal is inlining and inlining is something you get with a more or less constant call site. Invalidating it means to remove that attribute of being constant, the counter will start new. That's why you may be able to remove old cases from your call site structure that way, but for inlining it does not help at all. Well... I am sure Remi will correct me should I tell wrong things here ;) adding a new GWT to a tree of GWTs or trashing the tree of GWTs and use only one GWT is equivalent if the tree of GWTs is already JITed, because the generated code will be deoptimized anyway. A big tree of GWTs is really bad because it's a linear search so it's slow, moreover a code with to many nodes will not be inlined at all. Also trashing a tree of GWTs is not only a way remove path that has not used but also a way to re-order a tree of GWTs. bye Jochen cheers, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: performance degeneration from jdk7u2 to jdk7u6?
On 05/24/2012 10:38 AM, Jochen Theodorou wrote: Am 23.05.2012 23:33, schrieb Rémi Forax: On 05/23/2012 07:50 PM, Jochen Theodorou wrote: no one helping me on the assembly analysis? Wow, you have generated the biggest fib function I have ever seen. About the bytecode you generate, as you said you have to remove $getCallSiteArray() because it seems it does some side effects so the JIT is not able to remove it. the first time an array is generated, later nothing happens anymore... but yes, this has to go. [...] the method signature is (I)I, it is called with Object in the body, because I cannot ensure that x-1 and x-2 will return an int. Since I cannot know for sure that fib(I)I is called, the result of the recursive fib call is seen as Object. All I know for sure is that the result of fib(x-1)+fib(x-2) will be converted to an int later and will cause an exception if the conversion is not possible. But that is the result of the plus, thus you don't exactly need a compatible return type for fib. In for example: int fib(int x) { if (x2) return 1 this.metaClass.fib = {int i - i==1?Forax:Remi } String.metaClass.plus = {String b - delegate.length()+b.length()} return fib(x-1)+fib(x-2) } assert fib(3)== 10 I replace fib inside fib with, well it returns a String, but signature wise I replace it with a method returning Object. String is not compatible with int. And it does not lead to an exception because I also replace String#plus with a version that simply returns the added length of both Strings. So fib(3) will call fib(2) and fib(1), which has the results Remi and Forax. Then I call plus on those results, leading to String#plus, which returns the added lengths, which is 10 and compatible with int. yes, you're right, I've forgotten that you can change two methods at the same time. Also, you should never use methods like |DefaultTypeTransformation.intUnbox| because you know that the return type is an int, you should back-propagate it and the return type of plus should be (Object;Object)I in the original example that is true, yes. That may allow to skip at intUnbox call... but only if I later select a plus method that returns int or Integer. In a different thread I already asked for requirements in that direction and that I get very differing results depending on what signatures I use. No I have at lest the hint, that returning int or Integer might be a good idea. if invokedynamic knows more, you can provide a path with less boxing so it's usually better. Now, the generated code, because of getCallSiteArray(), your real code starts at line 168 and here you start to box the two ints to two Integers to be able to call NumberMath.subtract(Number,Number) which call IntegerMath.substractImpl that unbox them. The VM is not able to remove calls to box / unbox for j.l.Integer. I see... unfortunate. You should generate a must simpler path here. You should never call a class like g.r.typehandling.*Math because all of these methods takes Numbers as parameters. You should create one simple class, with methods like this: static int add(int left, int right) { return left + right; } because it doesn't force you to do the boxing. So you will do the boxing only if it's necessary, i.e. only when the parameter is Object. And to now which method you have to call, instead of relying on getMath() you should use guardWithTest and test only parameters that are Object. Object is pretty often the case. Well if I count boxing and unboxing for (a+b)+(c+d), then now I have to box a to d, unbox for the add, box the result, no boxing for the outer plus call, but two unboxing and a boxing for the result and... if the method returns int, a final unboxing. That makes 7 boxing and 7 unboxing. If I use your add(II)I and backpropagate the return type, then I have one boxing each for the inner pluses, two unboxing for the outer plus. Totals to 2 boxing, 2 unboxing. Sounds better... even without backpropagating the call. I will try that out. :) bye Jochen cheers, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: performance degeneration from jdk7u2 to jdk7u6?
On 05/23/2012 07:50 PM, Jochen Theodorou wrote: no one helping me on the assembly analysis? Wow, you have generated the biggest fib function I have ever seen. About the bytecode you generate, as you said you have to remove $getCallSiteArray() because it seems it does some side effects so the JIT is not able to remove it. I don't understand why when you call fib in the body of fib, you are not able to say that the signature is (Object)I. You detect that this is a recursive call (you don't use the same BSM) but it seems you think that because fib can be changed using the meta object protocol, you should type it (Object)Object. But because you are in already fib, you already suppose that at least the return type is int. Basically, you can change the method fib when being in the middle of the execution of fib, because at least one call of fib is on the stack, the new method must have a return type which is compatible with fib. Also, you should never use methods like |DefaultTypeTransformation.intUnbox| because you know that the return type is an int, you should back-propagate it and the return type of plus should be (Object;Object)I Now, the generated code, because of getCallSiteArray(), your real code starts at line 168 and here you start to box the two ints to two Integers to be able to call NumberMath.subtract(Number,Number) which call IntegerMath.substractImpl that unbox them. The VM is not able to remove calls to box / unbox for j.l.Integer. You should generate a must simpler path here. You should never call a class like g.r.typehandling.*Math because all of these methods takes Numbers as parameters. You should create one simple class, with methods like this: static int add(int left, int right) { return left + right; } because it doesn't force you to do the boxing. So you will do the boxing only if it's necessary, i.e. only when the parameter is Object. And to now which method you have to call, instead of relying on getMath() you should use guardWithTest and test only parameters that are Object. cheers, Rémi Am 19.05.2012 09:23, schrieb Jochen Theodorou: Hi all, I was about to get a brand new assembly to ask some questions on the list here when I installed the newest available jdk7 update 6. I ran my simple Fibonacci test program and noticed that the time it took was sudden 4.6s, where it was 3.5s before. This plus 1s doesn't look too god to me. Has there been anything special that causes this? I mean before the indy version was a little faster than our call site caching, now it is a little slower. General java performance seems not to be reduced, so I would assume it is special to indy. See http://rifers.org/paste/show/1702 ... so now... is there sombody who can tell me why my indy version is now even slower than my call site version? Sadly I am not really fit enough in assembler anymore, especially the one produced by hotspot to really read that output. So I have no idea what is wrong. Two things I should mention... the call to $getCallSiteArray is surplus, yes, but that doesn't take much time. The only thing I know that really takes a lot of time is the exception guard, but that I have to have until I find a way to avoid it and the call site version has both as well of course. They should not count as satisfying explanation. bye Jochen ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: using MethodHandle in place of virtual methods?
On 05/19/2012 08:04 AM, Per Bothner wrote: On 05/16/2012 05:52 PM, Charles Oliver Nutter wrote: Thanks for a detailed and helpful response! On Wed, May 16, 2012 at 12:55 PM, Per Bothnerp...@bothner.com wrote: The attachment ProcUsingMH.java is a sketch of an alternative implementation where each apply[01N] method has a matching MethodHandle field. The apply[01N] method is now final and just invokes the corresponding MethodHandle. I don't think this would optimize like you're hoping. I'm not expecting a big performance gain. I'm hoping for a slight gain getting rid of using a dispatch-switch, partly because it avoids an extra indirection (and virtual dispatch), and partly because (as you mentioned) HotSpot may have trouble optimizing switches, at least large ones. To summarize my questions, before I actually try to implement this: (1) A change to replace a virtual applyX method by a final method that calls a InvokeMethod in a final field: My guess is this would performancewise be more-or-less a wash, with neither major gain or less. Is that likely? (2) My guess is using the new MethodHandle scheme might be slighly more efficient than using a switch on a procedure-index. It be be significantly more efficient in cases of big switch. Is that likely? I think there may be additional benefits to getting rid of the dispatch-switch: More helpful exception stack traces; plus avoiding the need to generate the dispatch classes. Finally, it seems that if Kawa in the future makes use of invokedynamic, having the MethodHandles in the Procedure would be an advantage. We do keep a method handle in each method object, but only so we can more easily retrieve it and bind it to the invokedynamic call site. I don't think your'e going to see the performance gain you're hoping for with the Procedure object that aggregates handles. You really need the invokedynamic call site for the whole thing to optimize together well. It probably makes sense to not implement a switch to using MethodHandles until I have a design (roadmap) for using invokedynamic, since the benefit of the former change is at best modest and probably not worth the effort unless Kawa also makes use of invokedynamic. Does that match your advice? As Charles said, you will not see big gain to use method handle until you use invokedynamic, but I think it worth a test. Here a version of Procedure that doesn't use inheritance at all and that you be more or less backward compatible with code that are already compiled with Procedure. import java.lang.invoke.MethodHandle; import java.lang.invoke.MethodHandles; import java.lang.invoke.MethodHandles.Lookup; import java.lang.invoke.MethodType; class Procedure { public final MethodHandle mh; // Used for classical call. private MethodHandle varargsMH; // Used for varargs call, allocated lazily Procedure(MethodHandle mh) { this.mh = mh; } /** backward compatibility. */ public final Object apply0() throws Throwable { if (mh.type().parameterCount() != 0) throw new RuntimeException(); return mh.invokeExact(); } /** backward compatibility. */ public final Object apply1(Object arg1) throws Throwable { if (mh.type().parameterCount() != 1) throw new RuntimeException(); return mh.invokeExact(arg1); } /** backward compatibility. */ public final Object applyN(Object[] args) throws Throwable { MethodHandle varargsMH = this.varargsMH; if (varargsMH == null) { varargsMH = mh.asSpreader(Object[].class, mh.type().parameterCount()); } return varargsMH.invokeExact(args); } } class Builtins { public static final Object hash(Object arg1) { return arg1.hashCode(); } private static Procedure createProcedure(Lookup lookup, String name, int parameterCount) { MethodHandle mh; try { mh = lookup.findStatic(Builtins.class, name, MethodType.genericMethodType(parameterCount)); } catch (NoSuchMethodException | IllegalAccessException e) { throw (LinkageError)new LinkageError().initCause(e); } return new Procedure(mh); } static final Procedure hash; static { Lookup lookup = MethodHandles.lookup(); hash = createProcedure(lookup, hash, 1); } } class Main { public static void main(String[] args) throws Throwable { // new way, get the method handle and call System.out.println(Builtins.hash.mh.invokeExact((Object)foo)); // old way - apply1 System.out.println(Builtins.hash.apply1(foo)); // old way - applyN System.out.println(Builtins.hash.applyN(new Object[]{foo})); } } cheers, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Boxing, still a limit of invokedynamic?
On 05/13/2012 07:21 PM, Charles Oliver Nutter wrote: On Sun, May 13, 2012 at 11:04 AM, Jochen Theodoroublackd...@gmx.org wrote: I wanted to ask you of your opinion. If I am going to compile something like a+b-c and a,b,c are all primtives, but I won't know that the results will be really the primtives too, then this means I will most probably compile it like this: invokedynamic(minus, invokedynamic(plus,a,b), c) meaning the result of a+b will be an Object (since I won't know it is a primitive) and then there will be one boxing for that, just to unbox again for the minus and then box again for the result of the minus. If now the result is not supposed to be a primitive, then there won't be another unbox, till the next operation done with that value. You could also encode a+b-c as a single invokedynamic operation, but I guess you're looking for a general solution... Now, even if the JIT is able to see through the one boxing and unboxing fro the result of plus, what will stay is the boxing for the result of the minus plus the many unboxing actions used when this result is used. Isn't that a conceptual problem? And how do others deal with that? First of all...how are you expecting that JIT will see through the first boxing? If the return result is going to be Object, it's going to go into an Integer. Perhaps you are hoping for escape analysis to get rid of it? If that's the case, why wouldn't the same expectation apply to the second call? If (a+b) returns an Integer that's immediately passed into (tmp-c) and both calls inline, in theory EA should have enough to eliminate the intermediate. If the result of (tmp-c) is never used as an object and never escapes, then EA should be able to get rid of that too. Of course this is all assuming that EA will be working across indy boundaries in the near future. Currently, it does not. In JRuby, where we have no static typing or type hints, we always do these invocations as all reference types. We're banking on JVM helping us out in the future, so my goal is to just use indy as efficiently as possible and keep call protocols simple. A confusing point for me: in your case, where you know they're all ints, how do you not know that + and - also return int? Can't you determine statically that this whole expression will return a primitive int? I think currently Groovy allows to replace + by a method that will return everything you want. But here, I think the spec of Groovy (if it means something) should be changed to say that when your replace a method by another, the return type must be a subtype of the existing method. I am asking because I was experimenting with method signatures and for such plus and minus methods and got mixed results. I was expecting the primtive versions would achieve the best results, but that was not the case. a plus(int, int) had worse performance than a plus(int,Integer) or plus(Integer,int) in some cases and sometimes it looked like plus(Integer,Integer) is worse, in other cases not. Well, this is causing me some problems. Why do I get such strange results? I would assume it depends on the JIT and the boxing logic it is able to recognize and not. What does the assembly look like? In my case, passing int instead of Fixnum where possible (usually only when a literal integer appears in the argument list) definitely helps; I don't have to construct a Fixnum or go to a cache to get it, and on the other side there's no type-checking required to make sure I really have a Fixnum. The int paths should be faster than the Integer paths. And again remember...I don't think the JIT in u4- does anything with the boxing coming out of these calls. It might do something on the other side, but not across the invokedynamic call. One more thing I noticed is, that if I have a = b+c, with all of them being int and b+c returning object, then letting the MethodHandle do the conversion from Object to int is actually much worse performance wise, than a cast to integer and calling valueOf. Shouldn't that be at least equal, if not fast considering that the result of b+c was first boxed and then is unboxed? Perhaps doing it in the handles makes the code more opaque? Do the non-handle way and the handle way have exactly the same logic? Object - int is not equivalent to Object - Integer - int, it can be Object - Byte - int by example. You have to chain several calls to asType() see slide 20 of my jvm summit talk last year, http://wiki.jvmlangsummit.com/images/9/93/2011_Forax.pdf Bottom line here is that if you need a reference type on LHS, you'll have to create a reference type, and we need the JVM to figure out that it can brush that part away. - Charlie Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: selective inlining of MH.invokeExact() callsites
On 05/09/2012 08:02 PM, Christian Thalinger wrote: On May 8, 2012, at 2:11 AM, Garcia Gutierrez Miguel Alfredo wrote: What's the behavior of @ForceInlining , in particular for MethodHandle.invokeExact() ? We introduced that annotation as an experiment for inlining exact invokers for LambdaForms (note: LambdaForm is not directly related to Project Lambda). The generated bytecode versions of these LFs are usually just argument shuffling or binding and calling the target. We know that this code compiles down to almost nothing in native machine code and that's why we force them to inline. The ForceInline annotation is a powerful tool which may produce bad results in the hands of costumers. Currently we don't hide it completely from users but that might happen in the future. and Christian correctly if I'm wrong but it only forces inlining when the JIT is triggered. So using the annotation on something which is not called enough will do nothing. As I said in an earlier mail, the golden hammer is the annotation PleaseJITThisCodeWithAPrivateMetaDataObjectNowAndReJITLaterWhenTheMethodOrTheLoopIsHot -- Chris Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: hg: mlvm/mlvm/hotspot: meth-lazy: renamed ConstantPoolCacheEntry vfinalMethod to f2Oop; use from_interpreted_offset instead of interpreter_entry_offset in TemplateTable::invokedynamic
On 04/26/2012 07:32 PM, christian.thalin...@oracle.com wrote: Changeset: 81bd2928d85d Author:twisti Date: 2012-04-26 10:32 -0700 URL: http://hg.openjdk.java.net/mlvm/mlvm/hotspot/rev/81bd2928d85d meth-lazy: renamed ConstantPoolCacheEntry vfinalMethod to f2Oop; use from_interpreted_offset instead of interpreter_entry_offset in TemplateTable::invokedynamic ! meth-lazy-7023639.patch Hi Christian, You forget to rename the name 'vfinal' in the comment + // Preserve the value of the vfinal flag on invokevirtual bytecode + // which may be shared with this constant pool cache entry. cheers, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: hg: mlvm/mlvm/hotspot: annot: brought up-to-date
On 04/27/2012 12:54 AM, christian.thalin...@oracle.com wrote: Changeset: 4ad86bc6f440 Author:twisti Date: 2012-04-26 15:53 -0700 URL: http://hg.openjdk.java.net/mlvm/mlvm/hotspot/rev/4ad86bc6f440 annot: brought up-to-date annot-inline: new patch for supporting @Inline annotation meth-lazy: refreshed Contributed-by: Haupt Michaelmichael.ha...@oracle.com + annot-inline.patch ! annot.patch ! meth-lazy-7023639.jit.patch ! meth-lazy-7023639.patch ! series Hi Christian, @Inline is really cool. adds a new Annotation AlwaysCompile and I will be happy forever :) Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Boxed types and constat propagation
On 04/25/2012 06:38 PM, Christian Thalinger wrote: On Apr 23, 2012, at 4:29 PM, Rémi Forax wrote: On 04/24/2012 12:57 AM, Kohsuke Kawaguchi wrote: On 04/23/2012 11:18 AM, Rémi Forax wrote: So again, I don't see how shifting from method handle tree to byte code generation would somehow make this non-issue. In fact, it's easy to know that a.foo() is a boolean because can be only applied on booleans so the invokedynamic call corresponding to a.foo() should return a boolean. From the compiler perspective, you have to back-propagate the excepted type from the root to the leaf. Charles was alluding to the same thing, so I was thinking about this, and I think I understand it now. That's a lot of optimization that I otherwise wouldn't have to do, if only boxed types and constant propagation worked well, but yeah, I see how it can work now. Thanks for all the insights. BTW, I've patched the JIT to consider all final fields of a class of java/lang as truly final but that not enough for having the escape analysis to work with integers. It seems that because of the cache used in Integer.valueOf(), the JIT considers that the Integers are not escapable. Correct. That's the main problem. Unfortunately the caching is now part of the spec. Although something that John and I talked about yesterday could help here: immutable (or stable) arrays. We might add some optimization for this soon since we'd need it for bound arguments in our new JSR 292 implementation. I've done something similar in the backport, it takes an array (in fact a map object to integer index but that an implementation detail) and generates a class with one static final field by array item and I substitute the bound argument array access by the static field when generating the bytecode. My main issue is that when the MethoHandle is GCed, the bound arguments are not GCed. -- Chris Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Boxed types and constat propagation
On 04/23/2012 06:37 PM, Kohsuke Kawaguchi wrote: 2012/4/22 Rémi Foraxfo...@univ-mlv.fr: So as you can see, 30 and 12 are not recognized as constants. You're right, 30 and 12 are not recognized as int constant but they are recognized as java/lang/Integer constant. 0x7f785ca29706: mov$0x7d66619d8,%r10 ; {oop(a 'java/ang/nteger' = 30)} so escape analysis works but Hotspot doesn't trust final field thus doesn't consider the value in the Integer as a constant. I see why HotSpot can't in general trust the 'final' keyword, but it sure would be nice if it can at least for these boxed types. After all, if Integer.value is modified via reflection, all kinds of weird things start to happen... I fully agree :) Also it's good to know that Graal, the JIT written in Java on top of Hotspot, trusts final by default and doesn't seem to have real trouble to run usual benchmarks. It's a prototype so I suppose that at some point they will have to change that but anyway it is possible to consider some final fields as constants without breaking all programs. After a few more experiments, I realize the root cause of this isn't so much as JSR-292 but more in HotSpot. For example, the following method produces the following assembly code, and as you can see it's failing to optimize body() into just return false. In fact, Hotspot doesn't fail. But Hotspot also obfuscates the code :( So my question is: - Am I missing something? yes :) in that case, the body of 'body' is (there is a ret at 0x7fadf244130c) 0x7fadf24412e6: mov$0x7d6602fe0,%r10 ; {oop(a 'java/lang/Class' = 'java/lang/Boolean')} 0x7fadf24412f0: mov0x74(%r10),%r8d;*getstatic FALSE This is doing Boolean %r8 = Boolean.FALSE java.lang.Boolean::valueOf@10 (line 149) ; - BoxedBooleanInlineTest::bool1@1 (line 29) ; - BoxedBooleanInlineTest::body@1 (line 21) 0x7fadf24412f4: movzbl 0xc(%r12,%r8,8),%r11d This is doing %r11 = %r8.value with proper sign bit expansion 0x7fadf24412fa: test %r11d,%r11d and this is testing if %r11 is 0. I have forgotten to say that I barely able to read the assembler code. So here is my mistake, I was thinking this code was useless. test is always something mysterious to me. 0x7fadf24412fd: jne0x7fadf244130d ;*iconst_0 ; - BoxedBooleanInlineTest::body@24 (line 21) 0x7fadf24412ff: xor%eax,%eax ;*ireturn and yes, this is return 0 The code is fully optimized xor %eax, %eax put 0 in eax, which is by convention the register containing the return value and the rest of the code is not used. The question here is why Hostspot generates the unnecessary codes above the xor instruction. The code tests that zero is equals to zero before returning zero :) I don't see that as a question. As I said and as you observed earlier, this code is *not* fully optimized, in that it doesn't treat Boolean.FALSE.value as a constant. And under that assumption, every instruction in the above code does make sense. You're right. - How do other language implementers cope with this? You are the first as far as I know to use only a tree of method handles to implement expressions. The rest of us generates bytecodes and have a compiler that does constant propagation. OK, that's good to know, but I'm not seeing how that changes the picture. The issue is that, as things stand now, the language runtime would have to do the type inference to use primitive types instead of Object, or else it won't benefit from constant propagation (which triggers other optimizations like dead code elimination and further escape analysis.) Say I compile a.foo() a.bar() in JEXL to a byte code (or a.foo a.bar in JRuby if that helps people understand the example better.) Unless I can somehow prove that a.foo() returns a boolean, I'd have to treat it like they return Object. And so even if at the runtime HotSpot discovers that a is mostly bound to a specific type that implements foo() as return true, it ends up in the assembly code like my example (with the guard if statement on top to check if the type of 'a' is of this common type), not the equivalent of return false --- suboptimal result. Yet as I see it, such type inference is hard/impossible for most dynamic languages. I certainly don't see how JRuby can do that. And I thought the whole point of this new addition in Java7 was to let HotSpot do these optimizations, as opposed to have each language runtime do it. So again, I don't see how shifting from method handle tree to byte code generation would somehow make this non-issue. In fact, it's easy to know that a.foo() is a boolean because can be only applied on booleans so the invokedynamic call corresponding to a.foo() should return a boolean. From the compiler
Re: Boxed types and constat propagation
On 04/24/2012 12:57 AM, Kohsuke Kawaguchi wrote: On 04/23/2012 11:18 AM, Rémi Forax wrote: So again, I don't see how shifting from method handle tree to byte code generation would somehow make this non-issue. In fact, it's easy to know that a.foo() is a boolean because can be only applied on booleans so the invokedynamic call corresponding to a.foo() should return a boolean. From the compiler perspective, you have to back-propagate the excepted type from the root to the leaf. Charles was alluding to the same thing, so I was thinking about this, and I think I understand it now. That's a lot of optimization that I otherwise wouldn't have to do, if only boxed types and constant propagation worked well, but yeah, I see how it can work now. Thanks for all the insights. BTW, I've patched the JIT to consider all final fields of a class of java/lang as truly final but that not enough for having the escape analysis to work with integers. It seems that because of the cache used in Integer.valueOf(), the JIT considers that the Integers are not escapable. Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Boxed types and constat propagation
On 04/24/2012 01:43 AM, Kohsuke Kawaguchi wrote: Any chance you can share that patch with us? It'd give me some pointers into poking around JDK code, now that I learned how to build it... Sure, it's easy, in ciField.cpp, there is a method named trust_final_non_static_fields(), you can add if (holder-is_in_package(java/lang)) return true; or change the flag TrustFinalNonStaticFields to true if you have build the fastdebug VM. cheers, Rémi On 04/23/2012 04:29 PM, Rémi Forax wrote: On 04/24/2012 12:57 AM, Kohsuke Kawaguchi wrote: On 04/23/2012 11:18 AM, Rémi Forax wrote: So again, I don't see how shifting from method handle tree to byte code generation would somehow make this non-issue. In fact, it's easy to know that a.foo() is a boolean because can be only applied on booleans so the invokedynamic call corresponding to a.foo() should return a boolean. From the compiler perspective, you have to back-propagate the excepted type from the root to the leaf. Charles was alluding to the same thing, so I was thinking about this, and I think I understand it now. That's a lot of optimization that I otherwise wouldn't have to do, if only boxed types and constant propagation worked well, but yeah, I see how it can work now. Thanks for all the insights. BTW, I've patched the JIT to consider all final fields of a class of java/lang as truly final but that not enough for having the escape analysis to work with integers. It seems that because of the cache used in Integer.valueOf(), the JIT considers that the Integers are not escapable. Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Boxed types and constat propagation
On 04/23/2012 12:34 AM, Charles Oliver Nutter wrote: On Sun, Apr 22, 2012 at 2:12 AM, Rémi Foraxfo...@univ-mlv.fr wrote: You are the first as far as I know to use only a tree of method handles to implement expressions. The rest of us generates bytecodes and have a compiler that does constant propagation. JRuby doesn't do any constant propagation in our existing compiler, but we will be able to do some in the newer compiler. Ruby's constants are lazily discovered at runtime, so we can't propagate their values...but Hotspot should be able to in many cases. Obviously, doing constant propagation on something that can be monkey patched is harder. Do you have a kind of closed world assumption ? i.e. the compiler knows that there is no eval and will never load new code ? - Charlie Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Boxed types and constat propagation
See inlined comments :) On 04/22/2012 12:02 AM, Kohsuke Kawaguchi wrote: Hi, I was inspired by the talk by Charles in JAX 2012 and was playing with invokedynamic a bit. I'm observing what seems like a constant propagation failure, which I'd imagine would affect some important use cases, so I wanted to check if I'm not doing something stupid. I've used Apache JEXL [1] as my toy dynamic language. My basic strategy was to convert an expression into a graph of MethodHandles. This is a fairly straight-forward process, where each node in the JEXL AST is converted into function of the type (JexlContext)-Object. JexlContext represents the context object for an evaluation. I then compiled the expression 30+12 to see how well it'd optimize, which essentially does the following: import static java.lang.invoke.MethodHandles.*; public void test1() throws Throwable { // builds 30+12 as tree MethodHandle a = constant(Object.class, 30); MethodHandle b = constant(Object.class, 12); MethodHandle h = lookup().unreflect(getClass().getMethod(add,int.class,int.class)); MethodHandle r = foldArguments(foldArguments(h,asReturnType(int.class,a)),asReturnType(int.class,b)); r = Sandbox.wrap(r); assertEquals(42, r.invokeWithArguments()); } public static int add(int a, int b) { return a+b; } public static MethodHandle asReturnType(Class type, MethodHandle h) { return h.asType(MethodType.methodType(type,h.type())); } I was hoping that this would optimize to return 42, but on my JDK7u3 on linux-amd64, it only gets optimized to the followig: # {method} 'invokedynamic' '()I' in 'Gen0' # [sp+0x20] (sp of caller) 0x7f785ca29700: push %rbp 0x7f785ca29701: sub$0x10,%rsp 0x7f785ca29705: nop ;*synchronization entry ; - Gen0::invokedynamic@-1 0x7f785ca29706: mov$0x7d66619d8,%r10 ; {oop(a 'java/ang/nteger' = 30)} 0x7f785ca29710: mov0xc(%r10),%eax 0x7f785ca29714: mov$0x7d66618b8,%r10 ; {oop(a 'java/ang/nteger' = 12)} 0x7f785ca2971e: add0xc(%r10),%eax ;*iadd ; - GuardedIntAddTest::add@2 (line 27) ; - java.lang.invoke.MethodHandle::invokeExact@14 ; - Gen0::invokedynamic@0 0x7f785ca29722: add$0x10,%rsp 0x7f785ca29726: pop%rbp 0x7f785ca29727: test %eax,0x5b2c8d3(%rip)# 0x7f7862556000 ; {poll_return} 0x7f785ca2972d: retq So as you can see, 30 and 12 are not recognized as constants. You're right, 30 and 12 are not recognized as int constant but they are recognized as java/lang/Integer constant. 0x7f785ca29706: mov$0x7d66619d8,%r10 ; {oop(a 'java/ang/nteger' = 30)} so escape analysis works but Hotspot doesn't trust final field thus doesn't consider the value in the Integer as a constant. I think this would affect dynamic languages that treat primitives and reference types interchangeably, which is the majority. If I understand correctly, those languages need to compose method handlers of the type (...)-Object, like I did, and rely on the inlining to discover unnecessary boxing/unboxing. p.18 in JSR 292 cookbook [2] is affected by this, too, since it uses a similar MethodHandle types. After a few more experiments, I realize the root cause of this isn't so much as JSR-292 but more in HotSpot. For example, the following method produces the following assembly code, and as you can see it's failing to optimize body() into just return false. In fact, Hotspot doesn't fail. But Hotspot also obfuscates the code :( So my question is: - Am I missing something? yes :) in that case, the body of 'body' is (there is a ret at 0x7fadf244130c) 0x7fadf24412e6: mov$0x7d6602fe0,%r10 ; {oop(a 'java/lang/Class' = 'java/lang/Boolean')} 0x7fadf24412f0: mov0x74(%r10),%r8d;*getstatic FALSE ; - java.lang.Boolean::valueOf@10 (line 149) ; - BoxedBooleanInlineTest::bool1@1 (line 29) ; - BoxedBooleanInlineTest::body@1 (line 21) 0x7fadf24412f4: movzbl 0xc(%r12,%r8,8),%r11d 0x7fadf24412fa: test %r11d,%r11d 0x7fadf24412fd: jne0x7fadf244130d ;*iconst_0 ; - BoxedBooleanInlineTest::body@24 (line 21) 0x7fadf24412ff: xor%eax,%eax ;*ireturn The code is fully optimized xor %eax, %eax put 0 in eax, which is by
Re: 7127687: MethodType leaks memory due to interning
John, I think you can remove all public methods of class WeakInternSet but ,add() i.e. tailor the implementation only for the need, at least until Doug Lea provides a weak/concurent/hashmap in java.util.concurent. Rémi On 03/29/2012 11:45 AM, Jim Laskey wrote: What we are trying to do is intern the MethodType. Maps are designed to provide the key - value relationship (one way.) If we only used the key and a dummy value in the Map, it would be equivalent to a write only collection, since there is no method to return a found key. We would have to set the value to sometime meaningful like the existing MethodType. Doing so creates the hard reference to the MethodType and transitively to the unloadable Classes. We encountered this problem in Nashorn where we generate object classes at runtime. MethodTypes that referred to these classes hung on to them indefinitely. When running JavaScript test suites, we soon exhausted the perm gen. I tried using WeakHashMap initially but then realized the implications. WeakHashSet is useless, since it uses WeakHashMap, making it not weak at all. This fix has been well tested, as it is integrated in our test system (4 times a day on several different platforms.) Sent from my iPhone On 2012-03-28, at 11:51 PM, Vitaly Davidovich vita...@gmail.com mailto:vita...@gmail.com wrote: We only care about keys here anyway, right? WeakHashMap with a dummy value object (some static reference) should achieve the same thing since its Entry will have the key referenced weakly and the value strongly (but we don't care since the value is a dummy static ref). Maybe I misunderstood the intent though ... Sent from my phone On Mar 28, 2012 9:02 PM, Jim Laskey jlas...@me.com mailto:jlas...@me.com wrote: The WeakHashMap leads to a non-weak reference to the class, since only the key is weak. Same is true for public versions of WeakHashSet. The collection used here is truly weak. Sent from my iPhone 4 On 2012-03-28, at 9:42 PM, Vitaly Davidovich vita...@gmail.com mailto:vita...@gmail.com wrote: Hi John, I think you can use diamond generic inference when declaring the weak intern set. Also any reason you didn't use WeakHashMap directly with dummy value to simulate the set? Or wrap the WeakHashMap and synchronize the accessors to it? Sent from my phone On Mar 28, 2012 7:52 PM, John Rose john.r.r...@oracle.com mailto:john.r.r...@oracle.com wrote: http://cr.openjdk.java.net/~jrose/7127687/webrev.00/ http://cr.openjdk.java.net/%7Ejrose/7127687/webrev.00/ 7127687: MethodType leaks memory due to interning Summary: Replace internTable with a weak-reference version. This is a point fix for JDK 8, and will (pending approval) also be back-ported to JDK 7u. — John Notes on process: This code is part of JSR 292. Therefore the review comments will be collected in mlvm-dev, and changes will be integrated via hsx/hotspot-comp. At least one reviewer must be an official Reviewer the JDK 8 Project [1], but other reviewers are most welcome. [1] http://openjdk.java.net/census#jdk8 ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net mailto:mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net mailto:mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net mailto:mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net mailto:mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: review request (S): 7129034: VM crash with a field setter method with a filterArguments
On 03/27/2012 10:04 AM, John Rose wrote: http://cr.openjdk.java.net/~jrose/7129034/webrev.00/ 7129034: VM crash with a field setter method with a filterArguments Summary: add null checks before unsafe calls that take a variable base reference; update unit tests This is a point fix for JDK 8, and will also be back-ported to JDK 7u. — John Note: This class is part of JSR 292. Therefore the review comments will be collected in mlvm-dev. looks good. Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Bug in ClassValue
Last week, Fredrik found that I've forgotten to implement ClassValue in the backport of jsr292. While struggling to implement it, I've found a bug in the implementation of the JDK. This snippet creates an infinite loop :( java.lang.ClassValueInteger cv = new java.lang.ClassValueInteger() { @Override protected Integer computeValue(Class? type) { remove(int.class); return 1; } }; cv.get(int.class); while this code is naughty, I think it's a legal one. cheers, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: This was very cool, adding a profiler using MH
On 02/17/2012 06:29 AM, John Rose wrote: On Feb 16, 2012, at 8:43 PM, Mark Roos wrote: The key code is below and it shows the power of methodHandles is ways one may not expect. And it added almost nothing to the run time ( maybe a few %). It inserts itself into each callsite target and collect entry and exit data. Nice! As a bonus, your profile points are given a chance to modify the arguments and return value. If you don't want to pass the arguments and return value, you can use a void-returning combiner with foldArguments (instead of filterArguments). The combiner will see the arguments and then return nothing, allowing the invoker to continue as before. Yes, in PHP.reboot I track the hotness of a method by calling a void returning method that will increment a counter that is bound to the method. My first version was using a code very similar to the code of Mark. I have then switch to use a fold + a void-returning combiner when we decide to support that idiom. A different use of foldArguments would also let you pass both the original arguments and the return value to the profileExit routine, so it could see the whole call. Something like this: invoker = {spreader which takes standard RtObject[] array}; profileExit = {method of type (RtCallSite, RtObject, RtObject[]) = RtObject}; profileExit = insertArguments(profileExit, 0, thisCallSite); // (RtObject, RtObject[]) = RtObject invoker = foldArguments(profileExit, invoker); // note that invoker serves as *combiner* // add entry profiling last, so that profileExit will see (possibly) transformed arguments: profileEntry = {method of type (RtCallSite, RtObject[]) = RtObject[]}; profileEntry = insertArguments(0, thisCallSite); // (RtObject[]) = RtObject[] invoker = filterArguments(invoker, profileEntry); You could also interpose on abnormal termination (exceptions), using guardWithCatch. The try-finallly method handle combiner is one example of the cookbook: http://code.google.com/p/jsr292-cookbook/source/browse/trunk/interceptors/src/jsr292/cookbook/interceptors/Interceptors.java#68 A relatively simple way to do take full control of traced calls is to delegate the actual calling to the profile routine, but this might (presently) hurt performance, since it will make all calls go through one hot point, profileWrapper: invoker = {spreader which takes standard RtObject[] array}; profileWrapper = {method of type (RTCallSite, MethodHandle, RtObject[]) = RtObject}; // wrapped call will execute this: profileWrapper(thisCallSite, unwrappedInvoker, arg...) invoker = insertArguments(profileWrapper, 0, thisCallSite, invoker); I'm glad you are enjoying the toolkit! -- John cheers, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Smalltalk and tiered compile data
On 02/08/2012 10:47 PM, Mark Roos wrote: Hi Rémi Hi Mark, I think I am starting to get it. My normal fallback creates a GWT and does a getTarget + setTarget on the root callsite to put this GWT at the head of the chain. I assume that this use of setTarget is as expected and causes no problems. I think I found this technique in some of your early code. yes ! For the switchPoint approach I would set the first (root) target to a switchPoint which invokes a new callsite which then takes the normal fallback. This keeps the switchPoint at the head with the GWT chain now growing from the second callsite. The invalidate path has a method handle which will create a new switchPoint with fallbacks and place it in the root callsite replacing the now dead switchPoint. This setTarget will occur on the next invocation of the root callSite after the invalidation. This state is similar to the first after a bootstrap. yes ! So I am going to do a lot of setTargets. One for each method cache miss and again for every call site which is invalidated plus any lookups it needs to do. Given that every setTarget involves a safe point this sounds like it could get expensive. Should I be concerned? No :) setTarget() doesn't implies automatically a safepoint. There is a safepoint if you do a setTarget() on a Callsites which have been used to generate a compiled code. thanks mark cheers, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Smalltalk and tiered compile data
On 02/08/2012 02:55 AM, Mark Roos wrote: Hi Rémi Just a clarification on the switchpoint usage to invalidate sites. The switch path would still do a setTarget correct? No, SwitchPoint.invalidateAll() doesn't call setTarget(). So I am still sending lots of setTargets just based on switch point state? No. The idea is that you introduce a guard created from the SwitchPoint at the beginning of your target. You create the guard with a target (your old target) and a fallback which in this case is the fallback that you install when you call setTarget. When you call invalidateAll with all your SwitchPoints (you can create one by signature if you want) then the VM ask all other threads to go to a safe point. For all callsite targets that have been JITed, the code is marked as dead and the next call to invokedynamic on that callsite will go through the target method handle tree instead of calling the JITed code. When the VM will call the guard of the invalidated SwitchPoint instead of calling the target, it will calll the fallback method of the guard. All will be done at one safepoint. It also seems that if the switch point is in series with the target that the callsite will not invalidate until its next use. technically, it's the guard of the switch point which is in serie but yes, JITed code willl be dropped later (when no callsite will be able to call it) and if the code is not JITed, the SwitchPoint is just turn off so nothing will be done until the callsite is called again. Seems like this could spread the invalidations out over time giving hotspot even more to do. No, because the VM will be able to do the cleanup without using a safepoint and the cleanup phase is not something heavy. Or am I not clear on how to do this? thanks mark cheers, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Smalltalk and tiered compile data
On 02/07/2012 08:28 PM, Mark Roos wrote: Christian asked: What exactly do you mean by invalidate call sites before the benchmark? Are you using MutableCallSites and call setTarget on them? Exactly. I am using setTarget to set each call site to its initial fallback method. This should drop the GWT lookup chain forcing each call site to rebuild the chain. The method handles for the code still exist but the GWTs that reference them are gone. This is my current way to force the call sites to get new versions of the method code. thanks Mark, you should use a SwitchPoint instead of calling a lot of setTarget() because each setTarget() may require all threads of the VM to go to a safepoint so you may create a 'safepoint storm' (as coined by Dan Heidinga). mark Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Benchmarking Smalltalk on JVM
On 02/02/2012 04:45 AM, Mark Roos wrote: From Rémi Without the descriptors of invokedynamic and the code of the BSM, it's hard to tell. Yes but they have no invoke dynamics and I was just wondering if my indy part was causing the issue. Your answer told me that I should be OK so that was helpful. This same code was much faster on jdk8-b20 for some reason. It can be an escape analysis change. As far as I know, escape analysis don't work through indy call but if Charles see same performance as Java, escape analysis has to work ?? I will play around and see where the time is going . Would be nice to have an way to get the 8086 object code. https://wikis.oracle.com/display/HotSpotInternals/PrintAssembly thanks mark Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Benchmarking Smalltalk on JVM
On 02/02/2012 04:45 AM, Mark Roos wrote: from Rémi if you know it will never escape,you should use an int directly. Well I am trying to build a Smalltalk system which has no static types so I have to box the ints. Since the code I showed was programmer entered I need to stay with the boxes. There are cases where the compiler generates the index code and there I do use static ints if I can be sure they are not passed. or you can box only just before it's passed. The MutableInteger trick only works because the VM does the escape analysis for you but the escape analysis done by the VM is more britlle than the one you can write, by example, you know that increment() is a pure function, the VM has to inline it to know. So if one call is not inlined in the middle of the body of the loop, then the VM will not remove your MutableInteger. It does cause some issues when I open a debugger on the stack so I may want to keep then boxed anyway and thus the MutableInteger ??, yes your debugger has to support it, but if you want a typed smalltalk you will need that anyway. mark Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Benchmarking Smalltalk on JVM
On 02/01/2012 01:52 AM, Mark Roos wrote: I just loaded about 250K lines of Smalltalk code into my jvm implementation so now I can start some real benchmarks using our application. All of this was done on a Mac. My first try was a object load which takes about 20 files and creates a pretty complex object set. This takes 100 seconds in ST and using the initial jdk7 release I also get 100 seconds. Not bad. But I see that one of the major slowdowns is in my use of boxed integers vs STs use of Fixnums. So I did some more detailed experiments. Using this code snippet which creates and drops about 2 million Integers which ST does in about 10ms. | bytes pos sum | bytes := ByteArray new:100. sum := 0. pos := 1. [pos = 100]whileTrue:[ sum := bytes at:pos. pos := pos + 1]. ^sum For the initial JDK7 I get 400ms, moving to jdk8 b20 it drops to 117ms ( very nice). I then converted some constructor lookups to statics to get to 66ms. Then the obvious move to make an integer cache for which I used the jTalk range of -2000 to 4000 gave 30ms And finally ( to handle the index integer) I created a MutableInteger which dropped me to 5ms. So 2X better than the ST I started with. But then I upgraded to jsk8b23 and now the best I see is 16ms. It also seems like the jit sometimes compiles and sometimes not even using the same startup sequence. Bleeding edge I would guess. But for the final test I used jdk7u4 and my load is 73 seconds. Not as good as the best jdk8b20 ( 60 seconds) but faster than native Smalltalk Hi Mark, I believe tiered compilation was enable by default between jdk8b20 and jdk8b23. I have seen some weird compilation pattern too but no time to really investigate. looking good mark Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Benchmarking Smalltalk on JVM
On 02/01/2012 10:44 PM, Mark Roos wrote: Hi Charles Its pretty simple. All of my integers are boxed and are by definition immutable. However I noticed that many uses of integer were for loop counters and indexes where the integer never escapes from the method. So I added two primitives, one to copy a integer into a new box and the other to increment the java primitive held inside the box. In all other ways it inherits from my Integer class. The value is in reducing Integer creation for big loop/index ints. Usage looks like position := 1 newMutable. gets a mutable integer with an initial value of 1 position increment:1.increments the internal primitive position = 10 normal integer compare method I'll probably add a mutable bit to the header to protect the unwary in case it escapes but for now its a power tool. regards mark if you know it will never escape,you should use an int directly. Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Benchmarking Smalltalk on JVM
On 02/01/2012 10:44 PM, Mark Roos wrote: This may be a little much to ask but... These bytecodes take about 20ns per cycle to run on my 2.8 GHz mac using jdk8-B23 without TieredCompile. Does this seem reasonable given the number of indy calls? The GWT depth on the method sends is 1 thanks mark LABEL 56 LABEL 1 56 aload 4 58 aload 3 59 astore 1 60 aload 1 INDY (asm) 61 [at:] RtCallSite, (6) {RtTestCases class benchmarkLoop, 19} 66 astore 1 67 aload 1 68 astore 5 INDY (asm) 70 [41] ConstantCallSite, (6) {dummy} 75 aload 4 77 astore 1 78 aload 1 INDY (asm) 79 [increment:] RtCallSite, (6) {RtTestCases class benchmarkLoop, 23} 84 astore 1 LABEL 85 LABEL 0 85 aload 4 87 astore 1 INDY (asm) 88 [4100] ConstantCallSite, (6) {dummy} 93 aload 1 INDY (asm) 94 [=] RtCallSite, (6) {RtTestCases class benchmarkLoop, 24} 99 astore 1 100 aload 1 101 getstatic ri/core/rtalk/RtObject _true Lri/core/rtalk/RtObject; JUMP 104 if_acmpeq LABEL 1 Without the descriptors of invokedynamic and the code of the BSM, it's hard to tell. Anyway, you can optimize the last instructions, = should return a boolean so the sequence should be: ldc 4100 aload 1 indy = (ILObject;)Z if_eq LABEL 1 for that you have to propagate types, from root to leafs to type the return type of invokedynamic with the expected type (the condition of an if is a boolean) an from leafs to root (the first argument of = is an int). cheers, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: No way to create a no-op MethodHandle that returns void?
On 01/24/2012 09:40 AM, John Rose wrote: On Jan 23, 2012, at 11:26 PM, Charles Oliver Nutter wrote: Oh, that does seem to work...what an ugly hack. And actually, you can just use Object.class but cast the resulting return to void, and it works. So basically, I do this: handler = MethodHandles.constant(Object.class, null) handler = MethodHandles.asType(void.class) The EG considered things like having MethodHandles.constant(void.class, null) or identity(void.class). Even identity(methodType(void.class, Throwable.class, String.class)). There is a certain logic to such things but it didn't feel regular enough, and users can get the same effect by asType (as you realized). Noctarius is right in referring to Void. The MH API treats Void fairly regularly as the wrapper type for void, with a unique (unit) wrapped value of null. handler = MethodHandles.dropArguments(0, Throwable.class, String.class) And then use that as the exception handler. It's not exactly the no-op I wanted, since it has the cast logic in there, but it's close enough. FWIW, I'm using this in the tests for invokebinder, for testing the tryFinally operation: That binder stuff is great. Next, we want a builder syntax in Java (for more than just List+Map constants). Plus a little type-system bridge from Foo::bar and foo-bar to MethodHandle. Then we can compose our method handles quite swimmingly. Lambda should provide bridge from Foo::bar to MethodHandle :) -- John Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: hg: mlvm/mlvm/jdk: LF'ified makeAllocator; 18 MethodHandlesTests working
On 01/04/2012 05:28 PM, Christian Thalinger wrote: On Dec 30, 2011, at 4:25 PM, Christian Thalinger wrote: On Dec 30, 2011, at 3:45 PM, Rémi Forax wrote: On 12/30/2011 03:26 PM, christian.thalin...@oracle.com wrote: Changeset: 700540b4b52b Author:twisti Date: 2011-12-30 15:25 +0100 URL: http://hg.openjdk.java.net/mlvm/mlvm/jdk/rev/700540b4b52b LF'ified makeAllocator; 18 MethodHandlesTests working ! meth-lazy-7023639.bcg.patch Hi Christian, I believe you should create the UNSAFE_ALLOCATEINSTANCE with lookup.bind() instead of doing a findVirtual + a bindReceiver to avoid to bind the receiver for each call of findConstructor. Recent changes allows this now and I just pushed a changeset that does exactly that. Thanks again! -- Chris :) Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: hg: mlvm/mlvm/jdk: LF'ified makeAllocator; 18 MethodHandlesTests working
On 12/30/2011 03:26 PM, christian.thalin...@oracle.com wrote: Changeset: 700540b4b52b Author:twisti Date: 2011-12-30 15:25 +0100 URL: http://hg.openjdk.java.net/mlvm/mlvm/jdk/rev/700540b4b52b LF'ified makeAllocator; 18 MethodHandlesTests working ! meth-lazy-7023639.bcg.patch Hi Christian, I believe you should create the UNSAFE_ALLOCATEINSTANCE with lookup.bind() instead of doing a findVirtual + a bindReceiver to avoid to bind the receiver for each call of findConstructor. BTW, nice to see that collect is not used anymore during the allocation. cheers, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: The NoClassDefFoundError bug is back :(
On 12/08/2011 06:58 PM, Charles Oliver Nutter wrote: I just had a report of the same error in 1.7.0GA, and I saw it on the CI server last night after I set up a 1.7.0_01 build. Did it linger into GA? http://ci.jruby.org/job/jruby-test-matrix/jdk=sun-java-7,label=master/406/console On Tue, Dec 6, 2011 at 3:52 AM, Rémi Foraxfo...@univ-mlv.fr wrote: I've forget to say that a workaround is to erase the type used the callsite before calling dynamicInvoker(). Can you elaborate on this? I'm not using dynamicInvoker...I bind the site and use invokeWithArguments. Are you saying I could insert an asType() that clears everything to Object? invokeWithArguments uses foldArgument and a generic invoker that maybe why you see the same error. about asType(), the idea is to erase the type before calling invokeWithArguments instead of mh.invokeWithArgument(args); you can use mh.asType(mh.type().erase()).invokeWithArguments(args); and I have no idea if it works and about the perf cost if there is a cost. If so, would that introduce overhead? - Charlie Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: The NoClassDefFoundError bug is back :(
On 12/05/2011 04:28 PM, Rémi Forax wrote: John, Christian, I was updating the multi-dispatch sample from the cookbook when the NoClassDefFoundError bug re-appear. I've reduced the code to this simple sample: public class NoClassDefFoundSample { public void foo() { // do nothing } public static void main(String[] args) { NoClassDefFoundSample classDefFoundSample = new NoClassDefFoundSample(); for(int i=0; i 10; i++) { classDefFoundSample.foo(); // invokedynamic } } } public class NoClassDefFoundBug { // bootstrap method public static CallSite invokevirtual(Lookup lookup, String name, MethodType type) throws Throwable { MethodHandle mh = lookup.findVirtual(type.parameterType(0), name, type.dropParameterTypes(0, 1)); CallSite invokerCallSite = new MutableCallSite(mh); MethodHandle dynamicInvoker = invokerCallSite.dynamicInvoker(); CallSite callSite = new MutableCallSite(type); callSite.setTarget(dynamicInvoker); return callSite; } } The sample triggers the JIT and the JITed code of the dynamicInvoker() throws the NoClassDefFoundError. Code is available here: https://code.google.com/p/jsr292-cookbook/source/browse/#svn/trunk/multi-dispatch/src/bug https://code.google.com/p/jsr292-cookbook/source/browse/trunk/multi-dispatch/NoClassDefFoundSample.java cheers, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
The NoClassDefFoundError bug is back :(
John, Christian, I was updating the multi-dispatch sample from the cookbook when the NoClassDefFoundError bug re-appear. [forax@localhost multi-dispatch]$ java -ea -cp classes:. Main2 skip java.lang.String.contentEquals/2 skip java.lang.String.replace/3 Exception in thread main java.lang.NoClassDefFoundError: Main2 at java.lang.invoke.MethodHandle.invokeExact(MethodHandle.java) at java.lang.invoke.MethodHandle.invokeExact(MethodHandle.java) at Main2.test2(Main2.java:27) at Main2.main(Main2.java:42) [forax@localhost src]$ java -version java version 1.8.0-ea Java(TM) SE Runtime Environment (build 1.8.0-ea-b07) Java HotSpot(TM) 64-Bit Server VM (build 22.0-b03, mixed mode) The sample is here: http://code.google.com/p/jsr292-cookbook/source/browse/trunk/multi-dispatch/src/jsr292/cookbook/mdispatch/ You can compile it from src, javac -d ../classes jsr292/cookbook/mdispatch/*.java The example that doesn't work is Main2, which is created by compiling Main2.java and then by rewriting all invokevirtual/invokestatic to invokedynamic using this line java -cp classes:../lib/asm-debug-all-4.0_RC1.jar jsr292.cookbook.mdispatch.Rewriter Main2.class Note that Main2.class is already rewritten so you don't have to do it if you just want to reproduce the bug without modifying Main2.java I think the error is thrown when calling a method handle created by this line https://code.google.com/p/jsr292-cookbook/source/browse/trunk/multi-dispatch/src/jsr292/cookbook/mdispatch/SmallSelectorMetadata.java?spec=svn13r=13#121 at this callsite http://code.google.com/p/jsr292-cookbook/source/browse/trunk/multi-dispatch/Main2.java#27 How the code works: it installs a bimorphic cache on the receiver and an inlining cache on the parameters. After 3 cache misses, the implementation switch to use a table based resolution that lookup for most specific method handle and use an exactInvoker to call it. The bug appear when the exactInvoker is called. regards, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: review request (L): 7030453: JSR 292 ClassValue.get method is too slow
On 12/04/2011 08:38 PM, Joe Darcy wrote: Hi John, Are there alternatives to adding two new fields to java.lang.Class? I assume most Class'es won't have ClassValue information associated with them. -Joe If you use Groovy, JRuby or Nashorn in your code, all visible classes will use this two fields. Any alternative will slow down the access to the class value. BTW, if we had to remove some fields, I vote for moving all fields related to the reflection in a delegate class. Looking up for members is slow so most of the code that call reflection methods use their own cache, so the performance impact will be small in my opinion (I have no data to prove that :) Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: review request (L): 7030453: JSR 292 ClassValue.get method is too slow
On 12/05/2011 01:27 AM, Joe Darcy wrote: On 12/4/2011 2:13 PM, Rémi Forax wrote: On 12/04/2011 08:38 PM, Joe Darcy wrote: Hi John, Are there alternatives to adding two new fields to java.lang.Class? I assume most Class'es won't have ClassValue information associated with them. -Joe If you use Groovy, JRuby or Nashorn in your code, all visible classes will use this two fields. Any alternative will slow down the access to the class value. In the mean time, all the non-Groovy, non-JRuby, non-Nashorn, etc. uses of class Class and all the classes not visible in those environments when they are being used will be larger. Adding the fields may be the right time/space trade-off, but I think the point merits some discussion given how many Class objects get created and the relative proportion of Java executions where ClassValue is currently used. The more reasonable time/space trade-off can change over time of course. -Joe I agree but as I said, in that case, I think it's better to take a look to the big picture and see if not only class values fields but also annotations related fields or reflection related fields can be moved. Also, if we don't provide a fast ClassValue, people will create their own concurrent weak hash map using class as key that will be worst because it seems that only few people knows how to do that right. Personally, I don't. Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Process-level fork on OpenJDK...is it madness?
On 11/30/2011 04:28 PM, Jochen Theodorou wrote: Am 30.11.2011 14:02, schrieb Rémi Forax: [...] What kind of initialization work is this? Could the result of that work be cached? we have to setup the initial meta class system, which requires to use reflection to inspect some classes and other work. Yes, this could be cached, if we would know how. It worth to give a try to java.lang.ClassValue, here. You you be able to create your metaclass only when needed. Also note that you can also lazyly initialize the list of methods, fields etc. because even if two threads ask the same list at the same time, the result will be the same, so there is no need to use synchronized here. (this is exactly what java.lang.Class code does) : it is all lazy, but what gives it if you need it for even the most simple script? For println 1+1 you will need the a meta class for the current class, you will need the int meta class, you will need to load the default methods too... and one second is burnt. The only way I see to avoid that is to not load the meta-class until someone reference them so you can compile this example to System.out.println(2) and if there is a ref to a meta-class somewhere, discard the code and recompile it with meta-class check. I do something like that in PHP.reboot but my unit of compilation is the method and not the class, which also avoid to compile a code you never use. bye Jochen Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Implementing a Smalltalk debugger with JSR292
On 11/27/2011 07:16 AM, Mark Roos wrote: One of the key parts of Smalltalk is the 'live' debugger. Unlike the general dynamic language features which are well supported by the additions from JSR292 the debugger requires support which may not have been considered as necessary to support dynamic languages. So we were not sure we would be able to provide that portion of the Smalltalk experience on the jvm. The good news is that we were able to implement almost all of the Smalltalk debugger features using only the services provided in the released jdk7. I thought I would take a moment to describe how we did it to both demonstrate the approach and to solicit suggestions for improvements. The Smalltalk debugger is 'live' in that it exists as a separate thread within the same process/memory space as the thread being debugged. This allows one to manipulate and inspect all objects from the same viewpoint as the debugged thread. Smalltalk offers the ability to inspect all instances of a class(type), all references to a specific object, the variables on all levels of the stack, senders and implementers of methods, and the ability to single step through method sends. There is also the ability to restart a thread from any level of the stack but we opted to wait on the coro patch before implementing this ( I also don't use it as it can have quite a few side effects ). The approach we took has two facets, we ( mainly oscar ) coded a C++ jvmti agent with a JNI interface which allowed us to call some JVMTI apis from within the jvm being debugged and we added some logic to the callsite to handle the stepping. Implementers and senders of methods is handled via reflection on the classes and methods present so that was easy. To support all instances and all references requires heap inspection which we get from using the jvmti heap functions. This had some issues with some of the support classes for invoke dynamic but we were able to use a two pass tagging approach to make sure we found all of the references to our objects. This has to find objects both in arrays and in instance vars. Inspecting the stack was straight forward once we filtered the stack trace to only have our method sends present. As an option one can also inspect the full jvm trace. Using the jvmti variable access api allows the locating the variable which is then placed into a static field of out debugger support class. This field is then access by Smallltalk via a primitive (in Smalltalk a primitive is the way we share with the underlying environment). Once we have this object we can manipulate its instance vars from the Smalltalk side as well. When an error is thrown the thread is suspended ( we added some jvmti thread management apis just to get away from the deprecated methods) and a new thread is launched with an instance of the debugger and a pointer to the thread to debug. At this point one can only inspect the thread locals and anything else in the object memory. The thread is not restartable so we kill it ( by sending ThreadDeath ). But it the error is a halt or breakpoint we can then step the thread along. We tried this with jvmti but is was broken and seems to add quite a bit of delay to everything. Plus its a callback approach which looked like a lot of work. So instead we tweaked the call site logic to add a debug check. I liked the way this worked a lot. For a dynamic look up we already have a callsite with a target of one or more GWTs to select the implementation which matches the receiver class. What we have to do to implement a debugger is to place before the first GWT a test which determines if this is the time to suspend. Unfortunately GWTs are added to the end so we need a way to keep the test at the beginning (thanks to John and Réme suggestion) we can simply have a callsite have a target which is another callsite. The first callsite points at the test logic and the second gets the GWT chain. One nice thing is that we can revert to the single site version using a debug flag. The code to get the initial target for the bootstrap callsite looks like: *private**void*setBootstrapTarget(MethodHandle mh){ // get the appropriate initial call site sequence *if*( RtDebugger./_debugEnable/){ // for debugging we need to have a sequence of methodHandles that is always the first // code executed when a _callsite_ is invoked. This checks to see if we should // hold here for a debug step or continue. _realSite= *new*MutableCallSite(mh); // this is the extra call site to hold the gwts MethodHandle invoker = _realSite.dynamicInvoker(); MethodHandles.Lookup lookup=MethodHandles./lookup/(); MethodHandle debugEntry= *null*; MethodType mt=MethodType./methodType/(*void*.*class*, RtObject.*class*); *try*{ debugEntry = lookup.findStatic(RtDebugger.*class*, debugEntry, mt); } *catch*(Throwable e) { e.printStackTrace(); }
Re: OpenJDK 8 + lambdas build for OS X
Hi Behrang, the lambda workspace is not yet included in the master workspace of openjdk 8 because a lot of APIs/implementations are too young/too experimental. Rémi On 11/17/2011 01:31 PM, Behrang Saeedzadeh wrote: Hi all, I just grabbed the latest version of the OpenJDK 8 build from http://openjdk-osx-build.googlecode.com/files/OpenJDK-1.8-x86_64-b11-2012-release.dmg but it doesn't seem to contain the recent drop of lambdas, as I was unable to compile this simple Java class: public class Main { public static void main(String[] args) { runMe(() - {}); } public static void runMe(Runnable r) { r.run(); } } Or am I using an incorrect syntax? Or is it that the lambdas aren't still included in the builds provided by that Google Code project? Cheers, Behrang Saeedzadeh http://www.behrang.org ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: OpenJDK 8 + lambdas build for OS X
On 11/17/2011 02:50 PM, Henri Gomez wrote: So Lambda ( Jigsaw) are in their own Projects with their own hg forests. Note that Mike Maurizio synced up the lambda forest with JDK 8 recently: http://mail.openjdk.java.net/pipermail/lambda-dev/2011-November/004128.html so creating such builds should be easier then before, but given that neither Lambda nor the Mac Port are integrated into JDK 8, you'd have three fast moving parts to deal with, so it wouldn't necessarily be trivial: Just potentially a bit easier then a few weeks ago. ;) Any interest in having such 'labs/forest' available as package for OS/X users ? If so, advices more than welcomed Hi Henry, I think having lambda + openjdk8 for Mac users will be cool, like you have it on windows and linux. http://jdk8.java.net/lambda/ As Dalibor said, you need to merge lambda workspace and MacPort workspace. Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: MutableCallSite + constant handle slower than field accesses?
On 10/17/2011 10:30 PM, Charles Oliver Nutter wrote: It seems like the natural solution! :) Invokedynamic is nothing without the handles wiring it up...so they should always live happily together in the land of fairies and unicorns. no pony :( - Charlie Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Mac+newest JDK changes?
On 10/14/2011 09:43 AM, Ola Bini wrote: Hi, I'm finally about to get back to hacking on Seph, specifically looking at performance - I would really like to be able to run the latest changes (things for u2 related to invoke dynamic) in MacOS X - what is the current status of this, and is it even possible at the moment? I will answer with another question :) What is the hotspot version of the JDK MacOS preview ? http://jdk7.java.net/macportpreview/ Cheers cheers, Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: dyn.js - invokedynamic-based js implementation
On 10/13/2011 04:24 PM, Douglas Campos wrote: Remi, By example, for javascript Number, it should be a j.l.Double but with it's own vtable when you talk about having a vtable, you mean the verified-entrypoint recipe from jsr292 cookbook? (or something into these lines) So I would be providing more methods to a j.l.Double, linking with static methods via methodhandles? yes, but the vtable of the VEP tailor its method handles in a very special way, a vtable storing simple method handles is enough. cheers -- qmx Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Google Dart
On 10/11/2011 09:08 AM, Krystal Mok wrote: Variance in Dart somehow bears a problem similar to the one in Java's array: http://try-dart-lang.appspot.com/s/qAoU The dartboard don't do any type checking at all. But yes, reading the doc it seems that generics are covariant and contravariant. Rémi On Mon, Oct 10, 2011 at 4:02 PM, Rémi Forax fo...@univ-mlv.fr mailto:fo...@univ-mlv.fr wrote: On 10/10/2011 09:51 AM, Marcus Lagergren wrote: FYI, Google Dart now has a twitter account : https://twitter.com/#!/dart_lang https://twitter.com/#%21/dart_lang https://twitter.com/#%21/dart_lang and Website is up http://www.dartlang.org/ /M OMG, Dart is the next Java not the next Javascript ! Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net mailto:mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Google Dart
On 10/10/2011 08:58 PM, Charles Oliver Nutter wrote: I agree it would be an interesting language on the JVM. It may be the dynamic Java I've wanted to make for a long time, with the added bonus of optional static types. This could almost be a weekend project atop invokedynamic. The type system is not that simple if you want to avoid to do one cast for each assignation (and for each method call arguments). I think you need one supplementary day :) - Charlie Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Google Dart
On 10/10/2011 09:51 AM, Marcus Lagergren wrote: FYI, Google Dart now has a twitter account : https://twitter.com/#!/dart_lang https://twitter.com/#%21/dart_lang and Website is up http://www.dartlang.org/ /M OMG, Dart is the next Java not the next Javascript ! Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Google Dart
In fact, there is already a frontend in Java: http://code.google.com/p/dart/source/browse/#svn%2Fbranches%2Fbleeding_edge%2Fdart%2Fcompiler%2Fjava%2Fcom%2Fgoogle%2Fdart%2Fcompiler It's used to compile to javascript but it can be adapted to generate bytecode at runtime. So yes, it seems simple (or not that hard) to write a runtime using invokedynamic. BTW, you can currently use Rhino for that. Rémi On 10/10/2011 10:14 AM, Alexander Turner wrote: It is a very interesting idea. It take Java to js trechnology google created and uses it for a more flexible language. The mix of static and dynamic typing (much like vb does/did) has a proven track record of being much liked by day job programmers. I suspect, much more than Go, this language could shake things up. I also anoreason itshould not run very well on the jvm under invoke dynamic semantics. For the first time in a long time - a new language to get exciterd about... On Oct 10, 2011 8:02 AM, Rémi Forax fo...@univ-mlv.fr mailto:fo...@univ-mlv.fr wrote: On 10/10/2011 09:51 AM, Marcus Lagergren wrote: FYI, Google Dart now has a twitter account : https://twitter.com/#!/dart_lang https://twitter.com/#%21/dart_lang https://twitter.com/#%21/dart_lang and Website is up http://www.dartlang.org/ /M OMG, Dart is the next Java not the next Javascript ! Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net mailto:mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Help with JIT talk for tomorrow
On 10/06/2011 03:27 PM, Krystal Mok wrote: I tried to document PrintCompilation a while ago, here: https://gist.github.com/1165804#file_notes.md Hope it helps. Sorry for the late reply, we're in holiday this week here in China. Regards, Kris Mok Wow ! Impressive ! You should update the Hotspot internal wiki http://wikis.sun.com/display/HotSpotInternals/Home John, is it possible ? Rémi On 2011-10-5, at 7:47, Charles Oliver Nutterhead...@headius.com wrote: Hey all...I'm updating and expanding my talk on C2 and will be posting some questions. Hopefully there's someone out there who can answer them :) First up... given this output from PrintCompilation, what do the ! and n and % mean? Also, in Java 7 there's now two numbers instead of one at the begining of the line...what are they? 1401 70 java.util.concurrent.ConcurrentHashMap::hash (49 bytes) 1412 71 java.lang.String::indexOf (7 bytes) 1420 72 ! java.io.BufferedReader::readLine (304 bytes) 1420 73 sun.nio.cs.UTF_8$Decoder::decodeArrayLoop (543 bytes) 1422 42 java.util.zip.ZipCoder::getBytes (192 bytes) made not entrant 1435 74 n java.lang.Object::hashCode (0 bytes) 1443 29 ! sun.misc.URLClassPath$JarLoader::getResource (91 bytes) made zombie 1443 25 sun.misc.URLClassPath::getResource (74 bytes) made zombie 1443 36 sun.misc.URLClassPath::getResource (74 bytes) made not entrant 1443 43 java.util.zip.ZipCoder::encoder (35 bytes) made not entrant 1449 75 java.lang.String::endsWith (15 bytes) 16311 % sun.misc.URLClassPath::getResource @ 39 (74 bytes) 1665 76 java.lang.ClassLoader::checkName (43 bytes) - Charlie ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: hg: mlvm/mlvm/hotspot: 2 new changesets
On 10/02/2011 03:03 AM, Mark Roos wrote: This is a pretty cool use of coroutines. I am trying a similar approach for a web browser base UI and was looking for a pattern do handle the time out case. So I send something to the UI, say open a dialog box, and suspend the coroutine. but the UI never responds so I have to have a loop checking time outs. Let me explain how AsyncSwing works. I try use the same coroutine (named uiCoroutine in the code) for all UI events. The main coroutine, the one which is implicit do the event pumping, when an event is received, I yield to the uiCoroutine with the UI event as argument, the uiCoroutine do the dispatch to the component and switch back to the main coroutine. If a call to a blocking IO is done in an event callback, it's done in the uiCoroutine so I yield to the main coroutine with a command object (representing the work to complete before yielding back), create a new coroutine and save it as the new uiCoroutine for the next events and starts the command as an async IO and go back in the event loop. I would like something on the coroutine like suspend but throw an exception in x ms. I could do it with threads ( suspend a cancelling thread for x ms ) but I was looking for a light weight approach. Any thoughts on how to place a time out on a starved coroutine? You have two ways to implement a timer. Or you post a special event containing the deadline and re-post it if the current time doesn't match the deadline (this is how javax.swing.Timer works), or you use a ScheduledThreadPool with only one thread, submit a runnable when you start a timeout and cancel the corresponding Future if the operation doesn't timeout or if the runnable is scheduled, you just have to end the coroutine. regards mark Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: hg: mlvm/mlvm/hotspot: 2 new changesets
On 09/27/2011 11:26 PM, Rémi Forax wrote: On 09/27/2011 11:12 PM, Lukas Stadler wrote: Yes :-) and there's more: Pre-built binaries for linux x64 and more documentation at: http://ssw.jku.at/General/Staff/LS/coro/ The page is work in progress, though. I'll have to finish it by next week, in order to direct people to it at my JavaOne talk :-) - Lukas I was talking with Gilles Duboscq yesterday and I realized that I could write an example mixing UI code with blocking IO that will not freeze the event loop because you can use coroutine to fake blocking IO use async IO under the carpet. Rémi Finally find the time to implement it :) http://weblogs.java.net/blog/forax/archive/2011/10/01/asyncswing Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: hg: mlvm/mlvm/hotspot: 2 new changesets
On 09/27/2011 11:12 PM, Lukas Stadler wrote: Yes :-) and there's more: Pre-built binaries for linux x64 and more documentation at: http://ssw.jku.at/General/Staff/LS/coro/ The page is work in progress, though. I'll have to finish it by next week, in order to direct people to it at my JavaOne talk :-) - Lukas I was talking with Gilles Duboscq yesterday and I realized that I could write an example mixing UI code with blocking IO that will not freeze the event loop because you can use coroutine to fake blocking IO use async IO under the carpet. Rémi On Sep 27, 2011, at 21:43 , Remi Forax wrote: Cool ! Remi lukas.stad...@jku.at mailto:lukas.stad...@jku.at wrote: Changeset: 9bb80f812fd7 Author:Lukas Stadler lukas.stad...@jku.at mailto:lukas.stad...@jku.at Date: 2011-09-26 15:11 +0200 URL: http://hg.openjdk.java.net/mlvm/mlvm/hotspot/rev/9bb80f812fd7 rebase coro to current hsx/hotspot-comp ! coro.patch ! series Changeset: 1bdc6f420130 Author:Lukas Stadler lukas.stad...@jku.at mailto:lukas.stad...@jku.at Date: 2011-09-27 18:34 +0200 URL: http://hg.openjdk.java.net/mlvm/mlvm/hotspot/rev/1bdc6f420130 coro: fixes to stack walking, support for ricochet frames ! coro.patch ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net mailto:mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net mailto:mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: hg: mlvm/mlvm/hotspot: 2 new changesets
On 09/27/2011 11:12 PM, Lukas Stadler wrote: Yes :-) and there's more: Pre-built binaries for linux x64 and more documentation at: http://ssw.jku.at/General/Staff/LS/coro/ The page is work in progress, though. I'll have to finish it by next week, in order to direct people to it at my JavaOne talk :-) - Lukas very cool, with the binary (product VM), ping pong works ! new Coroutine(new Runnable() { @Override public void run() { for(int i = 0; i 5; i++) { System.out.println(pong + i); Coroutine.yield(); } } }); for (int i = 0; i 5; i++) { System.out.println(ping + i); Coroutine.yield(); } output: ping 0 pong 0 ping 1 pong 1 ping 2 pong 2 ping 3 pong 3 ping 4 pong 4 Rémi On Sep 27, 2011, at 21:43 , Remi Forax wrote: Cool ! Remi lukas.stad...@jku.at mailto:lukas.stad...@jku.at wrote: Changeset: 9bb80f812fd7 Author:Lukas Stadler lukas.stad...@jku.at mailto:lukas.stad...@jku.at Date: 2011-09-26 15:11 +0200 URL: http://hg.openjdk.java.net/mlvm/mlvm/hotspot/rev/9bb80f812fd7 rebase coro to current hsx/hotspot-comp ! coro.patch ! series Changeset: 1bdc6f420130 Author:Lukas Stadler lukas.stad...@jku.at mailto:lukas.stad...@jku.at Date: 2011-09-27 18:34 +0200 URL: http://hg.openjdk.java.net/mlvm/mlvm/hotspot/rev/1bdc6f420130 coro: fixes to stack walking, support for ricochet frames ! coro.patch ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net mailto:mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net mailto:mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: hg: mlvm/mlvm/hotspot: 2 new changesets
On 09/27/2011 11:12 PM, Lukas Stadler wrote: Yes :-) and there's more: Pre-built binaries for linux x64 and more documentation at: http://ssw.jku.at/General/Staff/LS/coro/ The page is work in progress, though. I'll have to finish it by next week, in order to direct people to it at my JavaOne talk :-) - Lukas and you can reify the stack, see below. Lukas, I think you should store a special singleton object instead of null in the object array (objectValues) because currently there is no way to know if null is null or a scalar. Also I think we should have a way to ask to an overload of AsymRunnable.call() to throw an exception that will be thrown by ret() in the coroutine and vice versa. Rémi Coroutine coroutine = new Coroutine() { @Override protected void run() { int i = 3; int j = 4; Object o = null; int k = 2 + myYield(); } private int myYield() { yield(); return 0; } }; Coroutine.yield(); CoroutineFrame[] frames = coroutine.serialize(); for(int i=0; iframes.length; i++) { CoroutineFrame frame = frames[i]; frame.dump(); System.out.println(object values +Arrays.toString(frame.objectValues)); System.out.println(scalar values +Arrays.toString(frame.scalarValues)); } prints: frame: CoroutineBase.startInternal@49 (3 locals, 0 expressions, 0 monitors) object values [fr.umlv.async.io.SerializeMain$1@69adff28, null, null] scalar values [0, 0, 0] frame: .run@8 (5 locals, 1 expressions, 0 monitors) object values [fr.umlv.async.io.SerializeMain$1@69adff28, null, null, null, null, null] scalar values [0, 3, 4, 0, 0, 2] frame: .myYield@0 (1 locals, 0 expressions, 0 monitors) object values [fr.umlv.async.io.SerializeMain$1@69adff28] scalar values [0] frame: Coroutine.yield@6 (0 locals, 0 expressions, 0 monitors) object values [] scalar values [] On Sep 27, 2011, at 21:43 , Remi Forax wrote: Cool ! Remi lukas.stad...@jku.at mailto:lukas.stad...@jku.at wrote: Changeset: 9bb80f812fd7 Author:Lukas Stadler lukas.stad...@jku.at mailto:lukas.stad...@jku.at Date: 2011-09-26 15:11 +0200 URL: http://hg.openjdk.java.net/mlvm/mlvm/hotspot/rev/9bb80f812fd7 rebase coro to current hsx/hotspot-comp ! coro.patch ! series Changeset: 1bdc6f420130 Author:Lukas Stadler lukas.stad...@jku.at mailto:lukas.stad...@jku.at Date: 2011-09-27 18:34 +0200 URL: http://hg.openjdk.java.net/mlvm/mlvm/hotspot/rev/1bdc6f420130 coro: fixes to stack walking, support for ricochet frames ! coro.patch ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net mailto:mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net mailto:mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Request for review (L): 7087838: JSR 292: add throttling logic for optimistic call site optimizations
Hi Mark, the throttle occurs if you deoptimize an already JITed callsite 10 times. If the callsite is not JITed, you can call setTarget() without triggering the throttle logic. So the main problem you will have is if your program always calls very often the same hot code and is really big. Because your code is big, you will have to flush the callsites, so you will flush the hot code which will be recompiled just after (because it's a hot code). After 10 flushes, you will see a performance degradation because your hot code will be no more JITed. Rémi On 09/18/2011 05:31 PM, Mark Roos wrote: From Christian The optimistic optimization for MutableCallSite and VolatileCallSite invalidate compiled methods on every setTarget. This possibly results in a recompile. For ever-changing call sites this is a performance hit. What is the use model of callsites which drives this? In my case I am using callsites as inline caches so I would expect a flurry of set targets as the initial classes pass though and then occasional sets as new classes show up. The depth is usually about 1 or 2 but sometimes a lot more ( megamorphic ) The other problem is that the sites become stale after awhile so what I do is flush them all ( it would have been nice if that was easy, implied switch point) and let them rebuild as the new classes go by. It seems like it would be hard to differentiate callsites which are always changing from ones which are simply megamorphic or have just been around so long that they have seen lots of classes. By the way how many changes until you throttle? As in my case the GWT is a simple extract field, compare and jump. It would be interesting if the optimization realized that and created a different optimization when the site has lots of targets in its chain. For instance switching from a series of tests to a vTable like lookup. Or maybe just let me specify this some way. Perhaps this argues for finer grain control on the callsites themselves. Providing hints to the optimizer rather than depending on heuristics seems safer at the point in their evolution. At least a way to reset them to some initial state would be helpful as we are creating dynamic languages which are expected to run for long periods. Since this does seem to impact my means of dealing with stale sites do you have any thoughts on an alternative approach? regards mark ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Request for review (L): 7087838: JSR 292: add throttling logic for optimistic call site optimizations
On 09/18/2011 07:16 PM, Mark Roos wrote: In looking at my code. In general 98% of the callsites are 3 targets. Those that are larger I can catch and use a different lookup. I also believe that Charles Nutter limits his depth to 5. So the general case I see will not exceed the 10. But I do have some cases where 10 is not the right number Part of my app is a compiler so some of its callsites see 20 or 30 classes ( AST node types). And part is a data flow evaluator where the data has about 30 types For these callsites you should use a dispatch table (a vtable dedicated to a callsite) instead of a sequence of 20 guards, because your code is equivalent to looking up a class in a linkedlist, so it's awfully slow. Moreover, are you sure the code that contains these callsites is JITed, it's pretty easy to hit other thresholds like by example the max number of internal nodes (ideal nodes). And as I watch the jit work it attacks these sites pretty fast so I could image exceeding the 10. I would like both or these to jit as they are used a lot And my final use case which is dynamic method replacement. Here I have to reset the callsites so they get the correct code. In our system this happens doing feature loading ( small dynamic patches) and for changing math operations ( dataflow) and finally during edit. Both easily exceed the 10 during the execution life of the app (hopefully months). I would be fine if there was a way to tell the jitter to reset its counts on a callsite as all of my use cases can absorb time at that instance. I really would have issues with losing the jitting when the app is expected to be running at full speed and would have to find some clever hack around it. Another point is that the classes callsites see during startup can be quite different than during normal operation. This is another reason we do the bulk invalidation mark Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Request for review (L): 7087838: JSR 292: add throttling logic for optimistic call site optimizations
Hi Christian, hi all, I understand that you need such kind of logic but I think it's not compatible with the approach taken by Mark Roos i.e flush all callsites if more than a predefined number of callsites have installed an inlining cache. A possible solution is to add a way in the API to know if a callsite will trigger a deoptimization if the target changes. Rémi On 09/16/2011 02:02 PM, Christian Thalinger wrote: [This change will be pushed after 7087357. So ignore the code removal in src/share/vm/classfile/javaClasses.cpp.] http://cr.openjdk.java.net/~twisti/7087838/ 7087838: JSR 292: add throttling logic for optimistic call site optimizations Reviewed-by: The optimistic optimization for MutableCallSite and VolatileCallSite invalidate compiled methods on every setTarget. This possibly results in a recompile. For ever-changing call sites this is a performance hit. The fix is to add some throttling logic that prevents the optimistic optimization after a specified amount of invalidations per CallSite object. This change also moves the flush_dependents_on methods from Universe to CodeCache. src/share/vm/c1/c1_GraphBuilder.cpp src/share/vm/ci/ciCallSite.cpp src/share/vm/ci/ciCallSite.hpp src/share/vm/classfile/javaClasses.cpp src/share/vm/classfile/javaClasses.hpp src/share/vm/classfile/systemDictionary.cpp src/share/vm/classfile/vmSymbols.hpp src/share/vm/code/codeCache.cpp src/share/vm/code/codeCache.hpp src/share/vm/code/dependencies.cpp src/share/vm/code/dependencies.hpp src/share/vm/memory/universe.cpp src/share/vm/memory/universe.hpp src/share/vm/oops/methodOop.cpp src/share/vm/opto/callGenerator.cpp src/share/vm/opto/memnode.cpp src/share/vm/prims/jvmtiRedefineClasses.cpp src/share/vm/prims/methodHandles.cpp src/share/vm/runtime/globals.hpp ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Hotspot loves PHP.reboot
I've taken the time to write a small prototype (by hand :( ) see http://www-igm.univ-mlv.fr/~forax/tmp/jsr292-deopt.zip using Fibonacci as John suggest. The idea is to transfer the control from fibo written using ints to fibo written using Object (BigInteger at runtime). If an operation overflow, an exception is thrown and the stack is reconstructed to restart the operation on objects. Moreover, a call can now fail because the result value is not an int anymore, in that case, the new return value is thrown inside an exception (one by thread), again the exception is catched and the control flow is transfered to the version that use objects just after the call. To summarize, you can transfer the control flow, either because an operation fail or because the return value of a call is not an int anymore. public static int fibo(int n) { if (n 2) return 1; return fibo(n - 1) + fibo(n - 2); } In fibo, we have 5 deoptimization pointcuts, n -1 can overflow, n - 2 can overflow, + can overflow, the first call to fibo can return a big integer, the second call to fibo can return a big integer. Each pointcut is encapsulated in a try catch that jump to a specific exception handler. All exception handlers first push a constant (corresponding to the pointcut number) and then jump to the same code that load all variables that potentially store the stack and do an invokedynamic invokedynamic calls a specific function (I have called it an exit function) which constructed from the fibonacci function but using objects. Depending on the pointcut number, a preamble code jump before an operation to restart it or after a function call to replace the return value stored in the exception. Because fibo is recursive, it can also call a plain old fibonacci function using objects without preamble jump code. You can note that the exit function (and the plain function if the function is recursive) can be generated lazily, only if needed i.e. if an overflow occurs. Now the bench on my laptop for fibo(45), i.e when there is no overflow: $ time java -cp .:classes JavaFibo // classical Java, fibo(45) using ints real0m7.393s user0m7.238s sys0m0.014s $ time java -cp .:classes JavaLongFibo // classical Java, fibo (45) using longs real0m6.021s user0m6.000s sys0m0.017s $ time java -cp .:classes Fibo// ints + overflow detecttion real0m8.263s user0m8.112s sys0m0.015s Not that bad but it's only with 5 pointcuts. For the record, here is the time for fibo(47): $ time java -cp .:classes Fibo real3m21.727s user3m22.882s sys0m0.513s I know that the deoptimization bootstrap method can install a transfer method that is a little faster but I think that either BigInteger are slow or I have made a mistake in the deoptimization code. Anyway, the result value is Ok. Rémi On 09/08/2011 09:47 PM, John Rose wrote: On Sep 8, 2011, at 4:57 AM, Thomas Wuerthinger wrote: Why not the following code pattern? Does it generate too many bytecodes? That's a reasonable alternative. It generates data movement bytecodes O(L * M), where L is the average number of live values at deopt points and M is the number of deopt points. The quadratic exponent on bytecode size bothers me, at least a little. The other pattern pins the live values into a common set of locals, reducing data movement bytecodes (and probably compiled code data movement). Using Remi's trick of an invokedynamic instead of a varargs array, the number of data movement bytecodes can be cut down about 3x (no iconst/aastore). But it's still quadratic. -- John ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Hotspot loves PHP.reboot
On 09/08/2011 01:12 AM, John Rose wrote: On Sep 7, 2011, at 3:28 PM, Rémi Forax wrote: On 09/07/2011 10:38 PM, John Rose wrote: Object l0, l1, l2, ...; l0 = l1 = l2 = ... null; // these are required only for definite assignment in the catch body try { ...do fast path... if (...constraint violated...) throw new DeoptExc(...); return ...fast result... } catch (DeoptExc ex) { Object[] display = { l0, l1, l2, ... }; return backupInterpreter(ex, display); // N.B. could throw DeoptExc to caller also } This problem is that the eager initializations of the various locals slow down the fast path. (Did I get it right Remi?) The register allocator should push them down into the catch body, and maybe even into the static debug info of the JVM. Locals is not the only problem, you have to track stack values too. To clarify, I meant only JVM locals, which serve as virtual registers for the application language locals, stacks, whatever. JVM stack elements are guaranteed to be thrown away when a JVM exception is thrown, so they function like virtual registers that are blown by deopt. events. It's the combination of locals initialization, local variable creation for storing stack values Yes, if there are logically stacked values that need tracking into slow paths, they need to be spilled to JVM locals. The cost of this will be reduced by good register allocations in the JIT. Ensuring this is part of the tuning job. and supplementary exception handlers (that's the main problem, or you have multiple handlers, or you have one handler and a constant) that slow down the fast path. I think a single handler is going to be simpler all the way around. Thus, the deopt. source location and value display map have to be stored somewhere so they can be made a parameter to the handler. I think the exception object is the most likely place to store it; TLS is also a candidate. currently I'm thinking to use multiple exception handler entry points but only one common code. handler1: iconst_0 goto common_handler handler2: iconst_1 goto common_handler ... common_handler: swap pop // remove exception object aload spill1 aload spill2 ... invokedynamic foo (ILObject;LObject ...) To extend on your cute idea below, have the source location and value display map be static parameters to the invokedynamic instruction which initiates transfer to the slow path. That way everything is in the class file, but lazily unpacked only if needed. There's little or no need to use executable bytecode instructions (other than an indy at the throw point and an indy in the local handler) to manage the deopt. transition. The actual bytecodes can be devoted to executing the fast path code, and testing for type-state faults. for PHP.reboot I need the AST node corresponding to the operation that overflow, so I need a live value. But I can use one parameter of the method to transfer all AST nodes wrapped in one object bound to the current method. BTW, although one might think that the repertoire for static BSM arguments is limited (string, class, int, long, MH, MT), note that the MH gives a hook to open-ended constant formation. Just treat the MH as a thunk to be executed to materialize a desired constant. but you can get live value unless you allow to insert live values to the constant pool when linking the class (another old dream). Also, in the catch block you can use invokedynamic to avoid the object array creation at call site. Yes, cute idea. Simplifying code on the slow path should make for faster loading and startup. Or (putting on my Pack200 hat), invokedynamic is the ultimate code-stream compressor. -- John Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Hotspot loves PHP.reboot
On 09/07/2011 09:08 AM, John Rose wrote: On Sep 7, 2011, at 12:00 AM, Per Bothner wrote: I assume this is one reason why Kawa's IntNum is (mostly) faster than BigInteger. Yes, that's probably true. Here's a dirty secret: As you can see from the OpenJDK sources, BigDecimal, but not BigInteger, has this optimization (see private field BigDecimal.intCompact). Why BigDecimal but not BigInteger? Because specjbb2000 uses BigDecimal. (OTOH, an imperfect metric like specjbb2000 is far better than no metric at all, for driving competition.) This remember me that we don't have any benchmarks using dynamic languages which is, as you explain, not good on the long term. What about having 10 to 12 benchs, one by language, provided by each language runtime developer as a good bench for their runtime ? -- John Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Hotspot loves PHP.reboot
On 09/07/2011 01:00 PM, MacGregor, Duncan (GE Energy) wrote: Could we do pass a method handle into this hypothetical to this hypothetical addDetectingOverflow and allow thus allow the caller to specify what should happen in the overflow case? Or does that still leave too much of a problem regarding actually returning the values? The return value is one problem because what you need to provide is not the return value of addDetectingOverflow but to the method that inlines (perhaps not directly) addDetectingOverflow. The other problem is that the VM will have to gather all locals to pass them as argument of the method handle. It's simpler to jump to a code that will call the interpreter (or another compiled code as PyPy does) hence the use of an exception. Rémi *From:*mlvm-dev-boun...@openjdk.java.net [mailto:mlvm-dev-boun...@openjdk.java.net] *On Behalf Of *John Rose *Sent:* 06 September 2011 21:05 *To:* Da Vinci Machine Project *Subject:* Re: Hotspot loves PHP.reboot On Sep 6, 2011, at 8:51 AM, Charles Oliver Nutter wrote: Did we ever figure out if it's possible to trick Hotspot into doing a JO instead of the raw bit-level operations? John/Christian/Tom: what would it take to get HS to know that we're doing an integer overflow-after-maths check and do the (faster?) JO? (1) Write a compelling API for something like Integer.addDetectingOverflow. (2) Roll it into JDK 8+epsilon. (3) Do the JIT work. People have thought on and off about (1) for many years, but with no clear winner. Exceptions or boxed objects have unpleasant interactions and are hard to use, while smuggling out the 33rd bit some other way (TLS, a long or double, a return-by-reference, a sentinel value) is painful. (This is a case where tuples would make things simple, but it is not enough to motivate introducing tuples.) -- John ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Hotspot loves PHP.reboot
On 09/07/2011 01:22 PM, Christian Thalinger wrote: On Sep 7, 2011, at 11:15 AM, Rémi Forax wrote: On 09/07/2011 09:08 AM, John Rose wrote: On Sep 7, 2011, at 12:00 AM, Per Bothner wrote: I assume this is one reason why Kawa's IntNum is (mostly) faster than BigInteger. Yes, that's probably true. Here's a dirty secret: As you can see from the OpenJDK sources, BigDecimal, but not BigInteger, has this optimization (see private field BigDecimal.intCompact). Why BigDecimal but not BigInteger? Because specjbb2000 uses BigDecimal. (OTOH, an imperfect metric like specjbb2000 is far better than no metric at all, for driving competition.) This remember me that we don't have any benchmarks using dynamic languages which is, as you explain, not good on the long term. What about having 10 to 12 benchs, one by language, provided by each language runtime developer as a good bench for their runtime ? That is a VERY good idea! How about if PHP.reboot contributes the first one? ;-) I've to think a bit about what is the best program for a bench of PHP.reboot. I also think that we should provide only one jar with a special entrypoint. What is the best location for that benchmark ? -- Christian Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Hotspot loves PHP.reboot
On 09/07/2011 05:07 AM, Charles Oliver Nutter wrote: On Tue, Sep 6, 2011 at 6:10 PM, John Rosejohn.r.r...@oracle.com wrote: Yes. Your request for JO makes me think some users would be happy with a boolean test, a la addWouldOverflow. It's what happens after the test that differs widely among applications, so why not just standardize the test. if (addWouldOverflow(p, q)) { throw or return BigInt or ... } return new Integer(p + q); The p+q, if it occurs within addWouldOverflow(p, q), will value-number to the explicit p+q, allowing the expected assembly code which computes p+q and then checks the overflow bit. (Actually, it's likely that the addl p',q instruction would occur twice, because hotspot not very good at tracking condition codes.) That was my immediate concern. JO will act based on the last operation, so we wouldn't duplicate any work. Of course, at the level of multiple addl's it may be a small price to pay for a less code-order-sensitive option like addWouldOverflow. Thinking about how you'd JIT with such intrinsics made me realize the best case is still the full-on addDetectingOverflow since it could emit the add and jo operations all together in the proper order. Anything that depends on the bytecode ordering (iadd followed by this intrinsic call) would be tweaky, and then there's the simple fact that in the*absence* of JIT there's no real way to do didAddOverflow without passing everything in again like we do in JRuby now. Perhaps no gain in that case. Only the full addDetectingOverflow could reliable do the add and jo in precisely the correct order, figuring in any other register effects. That's true, except that exceptions tend to be imprecise: It's hard to tell which sub-expression cause the exception, out of a complex statement. Addressing both the precision and pre-allocation problems, you could ask the application to create the exception: public staticX extends Throwable int addDetectingOverflow(int x, int y, X onOverflow) throws X This is pretty good, though it's another unusual precedent for JDK (or at least I know of no APIs that have this form). Still, it might be the lightest-weight option, since it allows you to opt completely out of all allocation. The other solution is to do the strict opposite, to use an exception that have a private constructor so it can't be created by a user code and have no stacktrace, etc (see http://download.oracle.com/javase/7/docs/api/java/lang/Throwable.html#Throwable%28java.lang.String,%20java.lang.Throwable,%20boolean,%20boolean%29) so the VM knows that only methods *DetectingOverflow are able to throw that specific exception. int addDetectingOverflow(int x, int y) throws IntegerOverflowException This also have the advantage that the inlining heuristic can be tweaked to not count exception handlers that receive that specific exception. Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Hotspot loves PHP.reboot
On 09/07/2011 09:32 PM, Per Bothner wrote: On 09/07/2011 02:15 AM, Rémi Forax wrote: This remember me that we don't have any benchmarks using dynamic languages which is, as you explain, not good on the long term. What about having 10 to 12 benchs, one by language, provided by each language runtime developer as a good bench for their runtime ? Well, there are the Computer Language Benchmark Game problems at http://shootout.alioth.debian.org/ . Isaac Gouy doesn't want to support more languages officially, but we can certainly use these as a starting point on some alternative server. (There are fast Kawa versions of all the current benchmarks: http://per.bothner.com/blog/2010/Kawa-in-shootout/ ) The shootout benchmark compares languages/runtimes, I want to compare JVMs or versions of JVMs running idiomatic code of each dynamic languages. Rémi ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev