Re: Invoking default methods from a Proxy's InvocationHandler in JDK9

[Mandy Chung]

> On Jan 2, 2017, at 11:20 PM, Peter Levart  wrote:
> 
> Hi Matthew,
> 
> On 01/03/2017 04:28 AM, Matthew Hall wrote:
>> I'm a member of the JDBI [1] project, an open source SQL access library
>> atop JDBC.
>> 
>> A major part of our API provides implementations of declarative interfaces
>> defined by users (similar to MyBatis). Interface methods may be default (in
>> which case the default method implementation is used) or abstract (in which
>> case JDBI provides the implementation).
>> 
>> We're using JDK 8, and we use java.lang.reflect.Proxy and InvocationHandler
>> to provide declarative interface implementations for our users.
>> 
>> Prior to release of JDK 8, it appears that the subject of enhancing Proxies
>> for default methods was discussed [2], but ultimately did not make it into
>> the release.
>> 
>> The root of the problem is that the generated Proxy class overrides all
>> methods in the proxy interfaces. This means that the interface default
>> methods are now superclass methods to the proxy class, which makes them
>> un-invokable from outside code--including the InvocationHandler.
>> 
>> As far as we can tell, the _only_ way to invoke a default method--on a
>> proxy, from an InvocationHandler--is to resort to some horrible
>> setAccessible shenanigans [3].
>> 
>> Specifically, we have to:
>> 1. Call Constructor.setAccessible(true) to make the private constructor
>> MethodHandles.Lookup(Class lookupClass, int allowedModes) accessible,
>> 2. Invoke that private constructor to instantiate a MethodHandles.Lookup
>> with private (!) access to all the things,
>> 3. Call Lookup.unreflectSpecial to get the MethodHandle for the default
>> method, and
>> 4. Invoke the method through the MethodHandle.
>> 
>> This is ugly, but it works--for now. If JDK 9 takes away access to
>> setAccessible, it may cease to work.
>> 
>> I found some discussion about this last summer [4], but it's hard to tell
>> if anything came out of the discussion.
>> 
>> Is there anything on the roadmap for JDK9 to allow a proxy's
>> InvocationHandler to invoke default methods on proxies? Are there any
>> existing proposals I should be aware of?
> 
> I have created a prototype last year:
> 
> http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-June/041629.html
> 
> But I think the JDK 9 train has already left the station. So perhaps in JDK 
> 10...


https://bugs.openjdk.java.net/browse/JDK-8159746 is the JBS issue for this.

One other possibility is to fix proxy generated class not to override default 
methods but there would require more investigation to tease out before we can 
be certain if this can make JDK 9.

Mandy

Re: Invoking default methods from a Proxy's InvocationHandler in JDK9

[John Rose]

On Jan 2, 2017, at 11:33 PM, Alan Bateman  wrote:
> 
> On 03/01/2017 07:17, Remi Forax wrote:
> 
>> I do not think that the workaround to create a Lookup object by reflection 
>> works with 9 given that java.lang.invoke is not declared as an open package.
>> 
>> John Rose has proposed to add a method to get a private Lookup object 
>> through sun.misc.Unsafe but as far as i know, this proposal goes nowhere.
>> 
> I think we got a good place with MethodHandles.privateLookupIn. It's not 
> going to allow you get a Lookup with private access to types in core modules 
> such as java.base but it may be useful for some cases.

+1

As of JDK-8130227, it is possible to use Lookup.findSpecial to gain access to 
interface default methods.

See below for an example.

— John

/*
  $ javac InvokeDefault.java && java -ea InvokeDefault
  override foo on an OverridesDefault
  default foo on an OverridesDefault
  default foo on an OverridesDefault
  $ java version "9-ea"
  Java(TM) SE Runtime Environment (build 9-ea+131)
  Java HotSpot(TM) 64-Bit Server VM (build 9-ea+131, mixed mode)
*/

import java.lang.invoke.*;
import static java.lang.invoke.MethodHandles.*;
import static java.lang.invoke.MethodType.*;

interface HasDefault {
default String foo() { return "default foo on "+this; }
}
public class InvokeDefault {
private static class OverridesDefault implements HasDefault {
public String foo() { return "override foo on "+this; }
public String toString() { return "an OverridesDefault"; }
}
public static void main(String... av) throws Throwable {
HasDefault obj = new OverridesDefault();
System.out.println(obj.foo());
// ==> override foo on an OverridesDefault
System.out.println(HasDefaultShim.MH_foo().invoke(obj));
// ==> default foo on an OverridesDefault
System.out.println(InvokeDefault.MH_foo().invoke(obj));
// ==> default foo on an OverridesDefault
}
static MethodHandle MH_foo() {
try {
Class it = HasDefault.class;
MethodHandle mh = MethodHandles.lookup().findSpecial(it, "foo", 
methodType(String.class), it);
return mh;
} catch (ReflectiveOperationException ex) {
throw new AssertionError(ex);
}
}
}
interface HasDefaultShim extends HasDefault {
static MethodHandle MH_foo() {
try {
Class it = HasDefault.class;
Class me = HasDefaultShim.class;
MethodHandle mh = MethodHandles.lookup().findSpecial(me, "foo", 
methodType(String.class), me);
assert(mh.type().parameterType(0) == me);
// force the MH to accept all implementations of the interface
mh = explicitCastArguments(mh, methodType(String.class, it));
assert(mh.type().parameterType(0) == it);
return mh;
} catch (ReflectiveOperationException ex) {
throw new AssertionError(ex);
}
}
}

Re: Invoking default methods from a Proxy's InvocationHandler in JDK9

[Alan Bateman]

On 03/01/2017 07:17, Remi Forax wrote:


I do not think that the workaround to create a Lookup object by reflection 
works with 9 given that java.lang.invoke is not declared as an open package.

John Rose has proposed to add a method to get a private Lookup object through 
sun.misc.Unsafe but as far as i know, this proposal goes nowhere.

I think we got a good place with MethodHandles.privateLookupIn. It's not 
going to allow you get a Lookup with private access to types in core 
modules such as java.base but it may be useful for some cases.


-Alan.

Re: Invoking default methods from a Proxy's InvocationHandler in JDK9

[Peter Levart]

Hi Matthew,

On 01/03/2017 04:28 AM, Matthew Hall wrote:

I'm a member of the JDBI [1] project, an open source SQL access library
atop JDBC.

A major part of our API provides implementations of declarative interfaces
defined by users (similar to MyBatis). Interface methods may be default (in
which case the default method implementation is used) or abstract (in which
case JDBI provides the implementation).

We're using JDK 8, and we use java.lang.reflect.Proxy and InvocationHandler
to provide declarative interface implementations for our users.

Prior to release of JDK 8, it appears that the subject of enhancing Proxies
for default methods was discussed [2], but ultimately did not make it into
the release.

The root of the problem is that the generated Proxy class overrides all
methods in the proxy interfaces. This means that the interface default
methods are now superclass methods to the proxy class, which makes them
un-invokable from outside code--including the InvocationHandler.

As far as we can tell, the _only_ way to invoke a default method--on a
proxy, from an InvocationHandler--is to resort to some horrible
setAccessible shenanigans [3].

Specifically, we have to:
1. Call Constructor.setAccessible(true) to make the private constructor
MethodHandles.Lookup(Class lookupClass, int allowedModes) accessible,
2. Invoke that private constructor to instantiate a MethodHandles.Lookup
with private (!) access to all the things,
3. Call Lookup.unreflectSpecial to get the MethodHandle for the default
method, and
4. Invoke the method through the MethodHandle.

This is ugly, but it works--for now. If JDK 9 takes away access to
setAccessible, it may cease to work.

I found some discussion about this last summer [4], but it's hard to tell
if anything came out of the discussion.

Is there anything on the roadmap for JDK9 to allow a proxy's
InvocationHandler to invoke default methods on proxies? Are there any
existing proposals I should be aware of?


I have created a prototype last year:

http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-June/041629.html

But I think the JDK 9 train has already left the station. So perhaps in 
JDK 10...


What you can do until then is to use other libraries (cglib, etc.). They 
can use sun.misc.Unsafe to "inject" proxy implementation classes into 
class loaders / modules / packages of interfaces they are implementing 
(similarly to what java.lang.reflect.Proxy is doing) and use 
MethodHandle(s) obtained from normal Lookup(s) in those classes to 
invoke the default implementations (similarly to what I have done in the 
prototype). I haven't checked, but maybe some other library already 
supports that?


Regards, Peter



Regards,

Matthew Hall

[1] https://github.com/jdbi/jdbi/
[2]
http://mail.openjdk.java.net/pipermail/lambda-dev/2012-August/005675.html
[3]
https://github.com/jdbi/jdbi/blob/jdbi3/sqlobject/src/main/java/org/jdbi/v3/sqlobject/DefaultMethodHandler.java
[4]
http://jigsaw-dev.1059479.n5.nabble.com/creating-proxies-for-interfaces-with-default-methods-td5711955.html

Re: Invoking default methods from a Proxy's InvocationHandler in JDK9

[Remi Forax]

I do not think that the workaround to create a Lookup object by reflection 
works with 9 given that java.lang.invoke is not declared as an open package.

John Rose has proposed to add a method to get a private Lookup object through 
sun.misc.Unsafe but as far as i know, this proposal goes nowhere.

You can still access to the private constructor of Lookup using 
defineAnonymousClass.

As you may already know, i've developed the Proxy2 library [1], exactly for that
(and to be far more efficient than a java.lang.reflect.Proxy).

regards,
Rémi

[1] https://github.com/forax/proxy2

- Mail original -
> De: "Matthew Hall" <qualidaf...@gmail.com>
> À: jigsaw-dev@openjdk.java.net
> Envoyé: Mardi 3 Janvier 2017 04:28:09
> Objet: Invoking default methods from a Proxy's InvocationHandler in JDK9

> I'm a member of the JDBI [1] project, an open source SQL access library
> atop JDBC.
> 
> A major part of our API provides implementations of declarative interfaces
> defined by users (similar to MyBatis). Interface methods may be default (in
> which case the default method implementation is used) or abstract (in which
> case JDBI provides the implementation).
> 
> We're using JDK 8, and we use java.lang.reflect.Proxy and InvocationHandler
> to provide declarative interface implementations for our users.
> 
> Prior to release of JDK 8, it appears that the subject of enhancing Proxies
> for default methods was discussed [2], but ultimately did not make it into
> the release.
> 
> The root of the problem is that the generated Proxy class overrides all
> methods in the proxy interfaces. This means that the interface default
> methods are now superclass methods to the proxy class, which makes them
> un-invokable from outside code--including the InvocationHandler.
> 
> As far as we can tell, the _only_ way to invoke a default method--on a
> proxy, from an InvocationHandler--is to resort to some horrible
> setAccessible shenanigans [3].
> 
> Specifically, we have to:
> 1. Call Constructor.setAccessible(true) to make the private constructor
> MethodHandles.Lookup(Class lookupClass, int allowedModes) accessible,
> 2. Invoke that private constructor to instantiate a MethodHandles.Lookup
> with private (!) access to all the things,
> 3. Call Lookup.unreflectSpecial to get the MethodHandle for the default
> method, and
> 4. Invoke the method through the MethodHandle.
> 
> This is ugly, but it works--for now. If JDK 9 takes away access to
> setAccessible, it may cease to work.
> 
> I found some discussion about this last summer [4], but it's hard to tell
> if anything came out of the discussion.
> 
> Is there anything on the roadmap for JDK9 to allow a proxy's
> InvocationHandler to invoke default methods on proxies? Are there any
> existing proposals I should be aware of?
> 
> Regards,
> 
> Matthew Hall
> 
> [1] https://github.com/jdbi/jdbi/
> [2]
> http://mail.openjdk.java.net/pipermail/lambda-dev/2012-August/005675.html
> [3]
> https://github.com/jdbi/jdbi/blob/jdbi3/sqlobject/src/main/java/org/jdbi/v3/sqlobject/DefaultMethodHandler.java
> [4]
> http://jigsaw-dev.1059479.n5.nabble.com/creating-proxies-for-interfaces-with-default-methods-td5711955.html

Invoking default methods from a Proxy's InvocationHandler in JDK9

[Matthew Hall]

I'm a member of the JDBI [1] project, an open source SQL access library
atop JDBC.

A major part of our API provides implementations of declarative interfaces
defined by users (similar to MyBatis). Interface methods may be default (in
which case the default method implementation is used) or abstract (in which
case JDBI provides the implementation).

We're using JDK 8, and we use java.lang.reflect.Proxy and InvocationHandler
to provide declarative interface implementations for our users.

Prior to release of JDK 8, it appears that the subject of enhancing Proxies
for default methods was discussed [2], but ultimately did not make it into
the release.

The root of the problem is that the generated Proxy class overrides all
methods in the proxy interfaces. This means that the interface default
methods are now superclass methods to the proxy class, which makes them
un-invokable from outside code--including the InvocationHandler.

As far as we can tell, the _only_ way to invoke a default method--on a
proxy, from an InvocationHandler--is to resort to some horrible
setAccessible shenanigans [3].

Specifically, we have to:
1. Call Constructor.setAccessible(true) to make the private constructor
MethodHandles.Lookup(Class lookupClass, int allowedModes) accessible,
2. Invoke that private constructor to instantiate a MethodHandles.Lookup
with private (!) access to all the things,
3. Call Lookup.unreflectSpecial to get the MethodHandle for the default
method, and
4. Invoke the method through the MethodHandle.

This is ugly, but it works--for now. If JDK 9 takes away access to
setAccessible, it may cease to work.

I found some discussion about this last summer [4], but it's hard to tell
if anything came out of the discussion.

Is there anything on the roadmap for JDK9 to allow a proxy's
InvocationHandler to invoke default methods on proxies? Are there any
existing proposals I should be aware of?

Regards,

Matthew Hall

[1] https://github.com/jdbi/jdbi/
[2]
http://mail.openjdk.java.net/pipermail/lambda-dev/2012-August/005675.html
[3]
https://github.com/jdbi/jdbi/blob/jdbi3/sqlobject/src/main/java/org/jdbi/v3/sqlobject/DefaultMethodHandler.java
[4]
http://jigsaw-dev.1059479.n5.nabble.com/creating-proxies-for-interfaces-with-default-methods-td5711955.html