Dear Peter:
thank you very much for your thorough and extensive analysis taking the
evolvement of Java into account!
You use the Java compiler and its behaviour (which adds syntax sugar to the
Java language and is
known to stick strictly to the Java language specification) for explanations
and as reference, which
is also what I have been doing in the past to counter-check the implemented
behaviour of the bridge.
The Java compiler has a very different outset: *at compilation time* it knows
everything it needs to
know for compiling. If something is missing or wrong at compilation time it is
able to stop
compilation raising appropriate errors, thereby inhibiting the successful
creation of Java classes
as long as the Java programmer does not correct erroneous code or supplies
missing vital
information. Java being a strictly typed language allows one to cast arguments
to force the compiler
to pick the method the Java programmer had in mind in the case that the
compiler could choose one
from a set of a signatures.
By contrast a bridge for an interpreted, dynamically typed language (like
ooRexx) needs to reflect
and infer at *runtime* in order to be able to choose the Java methods with the
appropriate
signature. In this particular case all primitive datatypes are (ooRexx/C)
strings in the dynamically
typed language such that at runtime each argument needs to be checked whether
they contain valid
values for conversion to primitive datatypes in the signature of a Java method
candidate.
In the dynamically typed language there is no means available for casting that
would be needed to
help solve the presented problem (there is optionally a box()-routine in the
bridge that allows the
ooRexx programmer to explicitly determine which primitive type is to be used
for picking an
appropriate Java method).
The new code in the bridge (had to rewrite that part from scratch because of
Java 9 and later) uses
java.lang.reflect to pick an appropriate Constructor, Method or Field object.
For Java 1.6 and 1.7
(although java.lang.invoke/MethodHandle got introduced with 1.7 I found cases
where inaccessible
types did not have accessible ancestors in the JRE) java.lang.reflect
invocation will be carried
out, for Java 1.8 and later java.lang.invoke/MethodHandle invocations gets
employed.
Comparing the speed between invocations using java.lang.reflect and
java.lang.invoke,
java.lang.invoke wins, but by a quite small margin.
Upon further testunit tests the rewritten bridge using java.lang.invoke would
execute all ooRexx
programs like java.lang.reflect since Java 1.1 with one single exception so far
which I reported
here, where invocation via java.lang.reflect behaves differently to
java.lang.invoke.
Because of this discrepancy the question would be: is this considered a bug
that will be corrected?
Would there be otherwise a solution possible in the bridge that could take care
of situations like
this, where the bridge has no context-information other than referring to
Constructor, Method or
Field objects with the supplied arguments?
A few coarse ideas to address this:
* if the unreflect() method gets used then MH.invoke() should behave like in
the java.lang.reflect
case; this means that the same assumptions should govern MH.invoke():
reason being that if a
java.lang.reflect object gets unreflected, also the established
java.lang.reflect rules should
keep working in this case for the MH.invoke* in order to remain fully
compatible with each other,
o if a MH gets created without unreflect() then this would not be
necessary
* the bridge could restrain itself to only use java.lang.reflect for
invocations on Java 8 and
higher instead of java.lang.invoke
o clearly, the current Java development efforts in this corner are
concentrated on
java.lang.invoke, so it would be desirable to switch from
java.lang.reflect to
java.lang.invoke which is not possible as long as this problem
(different behaviour in j.l.r
vs. j.l.i) persists
* do not fix this, but document this as an incompatibility with the
ramification that existing
programs in dynamically typed languages need to be rewritten (e.g. in this
case creating an
Array object from the List and supply that instead)
* ... ?
---rony
P.S.: It would be great if Java implemented reflective invocations for
dynamically typed languages
as this would solve the problem for all such languages once and forever in a
standard manner, rather
than have every bridge implementor create his own implementation, which is
challenging, effortful
and possibly error prone. In my case the assumptions that governed the original
implementation for
reflective Constructor, Method and Field access for over fifteen years got
broken by Java 9, because
j.l.r (notable 'setAccessible') changed in a fundamental aspect. If Java could
be emplyoed instead,
the Java supplied implementation would be able to adopt changes in the inner
workings accordingly
thereby insulating bridges from such details and problems.
On 08.06.2018 18:23, Peter Levart wrote:
> I think what you found is a discrepancy between handling of varargs methods
> using reflection vs.
> method handle's .invokeWithArguments().
>
> Reflection basically ignores the fact that some methods are varargs methods.
> It treats them
> exactly the same as if they had an array parameter (not a varargs parameter).
>
> For example, Arrays.asList() is a varargs method:
>
> public static <T> List<T> asList(T ... elements)
>
> But reflection treats is exactly the same as the following method:
>
> public static <T> List<T> asList(T[] elements)
>
> That's because varargs were introduced to Java in Java 5 as a kind of
> compilation sugar,
> implemented as an array parameter, while reflection is an older beast and
> wasn't updated to behave
> differently with varargs methods.
>
> MethodHandle(s) came later, in Java 7, and took into consideration varargs
> methods.
>
> So where's the problem? When you say in Java:
>
> Method asListMethod = Arrays.class.getMethod("asList",
> Object[].class);
> String[] elements = {"a", "b", "c"};
>
> System.out.println(
> asListMethod.invoke(null, elements)
> );
>
> You get: IllegalArgumentException: wrong number of arguments
>
> Because Method.invoke() is also a varargs method:
>
> public Object invoke(Object obj, Object... args)
>
> ... so java compiler (in order to be backwards source compatible with
> pre-varargs methods) treats
> this invocation in a way that just passes the String[] elements to the invoke
> method via the args
> parameter without any conversion (because String[] is a subtype of Object[]).
> Mathod.invoke then
> treats elements of the args[] array as individual verbatim parameters to be
> passed to the method
> when invoking it. As reflection treats Arrays.asList() as a normal no-varargs
> method which accepts
> a single Objet[] parameter and args[] contains 3 elements to be passed to a
> method with 3
> parameters, exception occurs.
>
>
> But when you use method handles and .invokeWithArguments():
>
> Method asListMethod = Arrays.class.getMethod("asList",
> Object[].class);
> MethodHandle asListMH =
> MethodHandles.lookup().unreflect(asListMethod);
>
> String[] elements = {"a", "b", "c"};
>
> System.out.println(
> asListMH.invokeWithArguments(elements)
> );
>
> ... the following happens: MethodHandle.invokeWithArguments() is also a
> varargs method:
>
> public Object invokeWithArguments(Object... arguments)
>
> ... so java compiler (in order to be backwards source compatible with
> pre-varargs methods) treats
> this invocation in a way that just passes the String[] elements to the
> invokeWithArguments method
> via the single 'arguments' parameter without any conversion (because String[]
> is a subtype of
> Object[]). MethodHandle.invokeWithArguments therefore takes 3 array elements
> and tries to invoke
> the underlying asList method with them. It observes that Arrays.asList is a
> varargs method, so it
> wraps the 3 "wannabe parameters" with the Object[] and passes the array to
> the asList method as
> single parameter. The result of above code is therefore:
>
> [a, b, c]
>
> If you want MethodHandle.invokeWithArguments() to treat "wannabe parameters"
> passed to it as
> verbatim parameters regardless of the variable/fixed arity of the method to
> be invoked, then you
> can transform the variable arity MethodHandle to a fixed arity MH:
>
> Method asListMethod = Arrays.class.getMethod("asList",
> Object[].class);
>
> String[] elements = {"a", "b", "c"};
>
> System.out.println(
> asListMethod.invoke(null, (Object) elements)
> );
>
> MethodHandle asListMH =
> MethodHandles.lookup().unreflect(asListMethod);
> asListMH = asListMH.asFixedArity();
>
> System.out.println(
> asListMH.invokeWithArguments((Object) elements)
> );
>
> Here you have it. Both of the above invocations produce equal output:
>
> [a, b, c]
> [a, b, c]
>
>
> What happens here is the following (will only describe the MH case - Method
> case is similar):
>
> MethodHandle.invokeWithArguments() is a varargs method:
>
> public Object invokeWithArguments(Object... arguments)
>
> ... so java compiler this time wraps the '(Object) elements' value with an
> Object[] because the
> value is of Object type (which is not a subtype of Object[]) - no backwards
> compatibility needed
> here, varargs conversion kicks in in the javac.
> MethodHandle.invokeWithArguments therefore takes
> an array with 1 element (the element being 'elements' array) and tries to
> invoke the underlying
> asList method with it. This time it observes that the method is not a varargs
> method, (because
> MethodHandle was transformed to fixed arity method handle) so it passes the
> single 'elements'
> wannabee parameter to the Arrays.asList single Object[] parameter - this
> invocation works because
> fixed arity asList() takes Object[] and 'elements' is a String[]...
>
> Hope this helps you understand what's going on.
>
> Regards, Peter
>
> On 06/08/18 12:07, Rony G. Flatscher wrote:
>> On 11.03.2018 20:22, Rony G. Flatscher wrote:
>>> Well, still trying to find out what the reason is, that core reflection's
>>> invoke behaves
>>> differently to MethodHandle's invokeWithArguments in one single case so far
>>> (using the method
>>> java.utli.Arrays.asList(...)).
>>>
>>> Here is a little Java program that excercises reflective access to
>>> "java.util.Arrays.asList(T... a)" using core reflection, i.e.
>>> "Method.invoke(Object obj,
>>> Object... args)" and "MethodHandle.invokeWithArguments(Object...
>>> arguments)":
>>>
>>> import java.util.*;
>>>
>>> import java.lang.reflect.*;
>>> import java.lang.invoke.*;
>>>
>>> class DemoAsListProblem
>>> {
>>> public static void main (String args[])
>>> {
>>> String arrNames[]=new String[] { "anne", "bert", "celine"};
>>> System.out.println("[1] (main) arrNames=\""+arrNames+"\",
>>> .toString()=\""+arrNames.toString()+"\"");
>>> List listNames=Arrays.asList(arrNames);
>>> System.out.println("[2] (main) after invoking
>>> Arrays.asList(arrNames), listNames=\""+listNames+"\"");
>>> System.out.println("\ninvoking testReflective() ...\n");
>>>
>>> testReflective();
>>> }
>>>
>>> public static void testReflective()
>>> {
>>> String arrNames[]=new String[] { "anne", "bert", "celine"};
>>> System.out.println("[3] (testReflective)
>>> arrNames=\""+arrNames+"\", .toString()=\""+arrNames.toString()+"\"");
>>>
>>> Method methAsList=null;
>>> List listNames=null;
>>> try {
>>> Class paramTypes[]=new Class[] { (new Object[0]).getClass()
>>> };
>>> methAsList=Arrays.class.getDeclaredMethod("asList",
>>> paramTypes);
>>> System.out.println("--- (core reflection) Method object
>>> asList: "+methAsList);
>>> System.out.println("\n--- (core reflection) now invoking
>>> Arrays.asList() via Method.invoke(...):");
>>>
>>> listNames=(List) methAsList.invoke( null, (Object[]) new
>>> Object[]{arrNames} ); // static method
>>> System.out.println("[4a] --- (CR) methAsList.invoke( null,
>>> (Object[]) new Object[]{arrNames} ) -> listNames=\""+listNames+"\"");
>>>
>>> listNames=(List) methAsList.invoke( null, (Object) new
>>> Object[]{arrNames} ); // static method
>>> System.out.println("[4b] --- (CR) methAsList.invoke( null,
>>> (Object) new Object[]{arrNames} ) -> listNames=\""+listNames+"\"");
>>>
>>> // "java.lang.IllegalArgumentException: wrong number of arguments":
>>> // listNames=(List) methAsList.invoke( null, (Object[])
>>> arrNames ); // static method
>>> // System.out.println("[5a] --- (CR) methAsList.invoke(
>>> null, (Object[]) arrNames ) -> listNames=\""+listNames+"\"");
>>>
>>> listNames=(List) methAsList.invoke( null, (Object)
>>> arrNames ); // static method
>>> System.out.println("[5b] --- (CR) methAsList.invoke( null,
>>> (Object) arrNames ) -> listNames=\""+listNames+"\"");
>>> }
>>> catch (Throwable t)
>>> {
>>> System.err.println("oops #1: "+t);
>>> t.printStackTrace();
>>> System.exit(-1);
>>> }
>>>
>>> System.out.println("\n--- (MH) now invoking Arrays.asList() via
>>> MethodHandle.invokeWithArguments(...):");
>>> MethodHandles.Lookup lookup=MethodHandles.lookup();
>>> MethodHandle mh=null;
>>> try {
>>> mh=lookup.unreflect(methAsList);
>>> System.out.println("--- (MH) unreflected MethodHandle mh:
>>> "+mh);
>>>
>>> listNames=(List) mh.invokeWithArguments( (Object[]) new
>>> Object[]{arrNames} );
>>> System.out.println("[6a] --- (MH) mh.invokeWithArguments(
>>> (Object[]) new Object[]{arrNames} ) -> listNames=\""+listNames+"\"");
>>>
>>> listNames=(List) mh.invokeWithArguments( (Object) new
>>> Object[]{arrNames} );
>>> System.out.println("[6b] --- (MH) mh.invokeWithArguments(
>>> (Object) new Object[]{arrNames} ) -> listNames=\""+listNames+"\"");
>>>
>>> listNames=(List) mh.invokeWithArguments( (Object[])
>>> arrNames );
>>> System.out.println("[7a] --- (MH) mh.invokeWithArguments(
>>> (Object[]) arrNames ) -> listNames=\""+listNames+"\"");
>>>
>>> listNames=(List) mh.invokeWithArguments( (Object) arrNames
>>> );
>>> System.out.println("[7b] --- (MH) mh.invokeWithArguments(
>>> (Object) arrNames ) -> listNames=\""+listNames+"\"");
>>>
>>> }
>>> catch (Throwable t)
>>> {
>>> System.err.println("oops #2: "+t);
>>> t.printStackTrace();
>>> System.exit(-2);
>>> }
>>> }
>>> }
>>>
>>> Compiling and running it under 9.0.4 yields the following output:
>>>
>>> [1] (main) arrNames="[Ljava.lang.String;@27ddd392",
>>> .toString()="[Ljava.lang.String;@27ddd392"
>>> [2] (main) after invoking Arrays.asList(arrNames), listNames="[anne,
>>> bert, celine]"
>>>
>>> invoking testReflective() ...
>>>
>>> [3] (testReflective) arrNames="[Ljava.lang.String;@2a18f23c",
>>> .toString()="[Ljava.lang.String;@2a18f23c"
>>> --- (core reflection) Method object asList: public static
>>> java.util.List java.util.Arrays.asList(java.lang.Object[])
>>>
>>> --- (core reflection) now invoking Arrays.asList() via
>>> Method.invoke(...):
>>> [4a] --- (CR) methAsList.invoke( null, (Object[]) new
>>> Object[]{arrNames} ) -> listNames="[anne, bert, celine]"
>>> [4b] --- (CR) methAsList.invoke( null, (Object) new
>>> Object[]{arrNames} ) -> listNames="[[Ljava.lang.String;@2a18f23c]"
>>> [5b] --- (CR) methAsList.invoke( null, (Object) arrNames )
>>> -> listNames="[anne, bert, celine]"
>>>
>>> --- (MH) now invoking Arrays.asList() via
>>> MethodHandle.invokeWithArguments(...):
>>> --- (MH) unreflected MethodHandle mh: MethodHandle(Object[])List
>>> [6a] --- (MH) mh.invokeWithArguments( (Object[]) new
>>> Object[]{arrNames} ) -> listNames="[[Ljava.lang.String;@2a18f23c]"
>>> [6b] --- (MH) mh.invokeWithArguments( (Object) new
>>> Object[]{arrNames} ) -> listNames="[[Ljava.lang.Object;@13a57a3b]"
>>> [7a] --- (MH) mh.invokeWithArguments( (Object[]) arrNames )
>>> -> listNames="[anne, bert, celine]"
>>> [7b] --- (MH) mh.invokeWithArguments( (Object) arrNames )
>>> -> listNames="[[Ljava.lang.String;@2a18f23c]"
>>>
>>> So a String array is turned into a List using Arrays.asList(strArray).
>>> Doing it with core
>>> reflection yields different results to doing it with invokeWithArguments().
>>>
>>> I would have expected that [4a] and [6a] would behave the same.
>>>
>>> Using reflective invoke() and invokeWithArguments() has been working for my
>>> bridge for all the
>>> test units (using literally the same arguments in both cases)
>>> interchangeably, the one exception
>>> is Arrays.asList().
>>>
>>> Maybe I am not seeing the obvious (having done too much "close-up tests"
>>> for quite some time
>>> now). So any hint, insight, help would be really appreciated!
>>>
>>> ---rony
>> As a few months have gone by without any follow-ups, I have been wondering
>> whether there is a
>> solution at all, if this is a problem rooted in the current implementation
>> of MethodHandle
>> methods not being 100% compatible with reflective invoke (which may imply
>> that one needs to stick
>> to use reflective invoke and forgo MethodHandle invokes for good).
>>
>> ---rony
_______________________________________________
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
