Hi core-libs-dev,
I am maintaining a module for the popular Jackson JSON library that attempts to
simplify code-generation code without losing performance.
Long ago, it was a huge win to code-generate custom getter / setter / field
accessors rather than use core reflection. Now, the gap is closing a lot with
MethodHandles, but there still seems to be some benefit.
The previous approach used for code generation relied on the CGLib + ASM
libraries, which as I am sure you know leads to horrible-to-maintain code since
you essentially write bytecode directly.
Feature development basically stopped because writing out long chains of
`visitVarInsn(ASTORE, 3)` and the like scares off most contributors, myself
included.
As an experiment, I started to port the custom class generation logic to use
LambdaMetafactory. The idea is to use the factory to generate `Function<Bean,
T>` getter and `BiConsumer<Bean, T>` setter implementations.
Then, use those during (de)serialization to access or set data. Eventually
hopefully the JVM will inline, removing all (?) reflection overhead.
The invocation looks like:
var lookup = MethodHandles.privateLookupIn(targetClass,
MethodHandles.lookup()); // allow non-public access
var getter = lookup.unreflect(someGetterMethod);
LambdaMetafactory.metafactory(
lookup,
"apply",
methodType(Function.class),
methodType(Object.class, Object.class),
getter,
getter.type())
This works well for classes from the same classloader. However, once you try to
generate lambdas with implementations loaded from a different classloader, you
run into a check in the AbstractValidatingLambdaMetafactory constructor:
if (!caller.hasFullPrivilegeAccess()) {
throw new LambdaConversionException(String.format(
"Invalid caller: %s",
caller.lookupClass().getName()));
}
The `privateLookupIn` call seems to drop MODULE privilege access when looking
across ClassLoaders. This appears to be because the "unnamed module" differs
between a ClassLoader and its child.
This happens without the use of modulepath at all, only classpath, where I
would not expect module restrictions to be in play.
Through some experimentation, I discovered that while I cannot call the
LambdaMetafactory with this less-privileged lookup, I am still allowed to call
defineClass.
So, I compile a simple class:
package <targetclasspackage>;
class AccessCracker { static final Lookup LOOKUP = MethodHandles.lookup(); }
and inject it into the target class's existing package:
lookup = lookup.defineClass(compiledBytes).getField("LOOKUP").get(null);
and now I have a full privileged lookup into the target classloader, and the
Metafactory then seems to generate lambdas without complaint.
This workaround seems to work well, although it's a bummer to have to generate
and inject these dynamic accessor classes.
It feels inconsistent that on one hand my Lookup is not powerful enough to
generate a simple call-site with the Metafactory,
but at the same time it is so powerful that I can load arbitrary bytecode into
the target classloader, and thus indirectly do what I wanted to do in the first
place (with a fair bit more work)
There's a bit of additional context here:
https://github.com/FasterXML/jackson-modules-base/issues/138
https://github.com/FasterXML/jackson-modules-base/pull/162/files
Any chance the Metafactory might become powerful enough to generate call sites
even across such unnamed Modules in a future release? Or even more generally
across arbitrary Modules, if relevant access checks pass?
I'm also curious for any feedback on the overall approach of using the
Metafactory, perhaps I am way off in the weeds, and should just trust
MethodHandles to perform well if you use invokeExact :) JMH does seem to show
some benefit though especially with Graal compiler.
Thanks a bunch for any thoughts,
Steven Schlansker
