On 7/15/2016 6:21 AM, Jochen Theodorou wrote:
I will give you an analysis of my situation so far:
assuming the whole Groovy runtime is a module, and assuming I have other
modules as well, and then I want to use a Groovy script at runtime, to
access those modules. And let us assume those modules do not know of
Groovy. Let us further assume the Groovy runtime will be in a loader
that delegates to a loader knowing the classes of those modules, or is
in the same loader.
So first problem is ... can I use the unnamed module for the scripts?
They read every (named) module, thus they can access the types.
Next are 4 types of method invocations to consider for the case of an
invocation from script in the unnamed module to named module:
reflective, indy, direct, generated bytecode for callsites (I name this
one csgen in short). The method itself is looked up by reflection
* for indy it is only a question of the lookup object, which will be
provided by the script and can do the call
* direct calls will work
* reflection would be done from the groovy runtime, thus the runtime
need to have a read on the named module. That´s the first time I will
need Java9 special code in this scenario
* csgen is more unclear. Let us assume it is in the unnamed package as
well, in a loader that delegates to the script loader. Basically it
becomes a direct call and works.
Next step is for my Groovy runtime having to call a method from the
script. Since everything is exported there is no problem on this side.
* indy will work after adding the read
* reflection - same
* direct call - same
* cs gen... the call to the csgen method will be done by an
interface/class of the Groovy runtime, and since the csgen class is in
the unnamed package, it has no problem calling the script methods.
With one exception, everything above is correct. (Exception: reflection
from the Groovy module to a named module, performed on behalf of the
script, does NOT need Java 9 code, because reflection gets readability
for free. Also, you mean that csgen code is in the unnamed MODULE; its
package is not so important.)
Key point: Just because a framework is delivered as a module, does not
mean the framework has to change its class loading behavior or require
that its plugins/scripts/introspectees be delivered as modules. Groovy
can continue to load scripts (or classes derived therefrom) into
user-defined loaders. The scripts can continue to access each other
(assuming suitable loader delegation) and Java platform APIs and Groovy
runtime APIs (assuming the Groovy module exports some packages).
Let us look at the same scenario with a named module for scripts. I will
use that as a step in between to think about actual compile time modules
written in Groovy. So for simplicity I assume I can create such a module
at runtime and it will export everything. How to do that? I have not the
slightest idea. Anyway, that means now I have calls from named script
module to named module:
* indy will work after the script module added a read to the target
module. This will require in the script, with a direct call to addReads,
because of caller sensitivity. Which means instead of just having JDK9
specific code in my runtime, I will now need JDK9 specific code to my
bytecode as well. And this needs to happen before any normal method
invocation can take place, thus static initializer, first thing... which
means completely new code for the compiler, that will work only on JDK9.
This means the compiler will then have to at least know if he compiles
for JDK9 or not...
* direct call - same
* reflection - the call is effectively done from the Groovy runtime, so
the Groovy runtime will have to add the read.
* csgen in the unnamed module means.. since the call is gated by an
interface/class from the runtime I assume I do not need a read edge from
the script module to the unnamed module csgen is in. The csgen class
then will be able to read the exported class of the target module, thus
has no problem.
Calls from the groovy runtime to the script are not different to before.
For creating named modules at run time, see the javadoc for
java.lang.reflect.Layer and all the linked javadoc down to
java.lang.module.ModuleDescriptor.Builder.
You're right about indy calls and direct calls following readability, so
you can either have the script's module require another named module in
the Configuration that you pass to the Layer, or you can have the
script's own code add readability as you describe.
(For reflection, note again that the Groovy runtime does NOT have to add
a read from its module to another named module.)
Now for a pivot. All the issues you raise in the rest of this email,
about how named modules containing bytecode-derived-from-Groovy-source
interact with named modules of the Groovy runtime or otherwise, are
familiar. Why? Because Nashorn faced exactly the same issues. There will
be a presentation at the JVM Language Summit (jvmlangsummit.com) on how
Nashorn uses modules to encapsulate its generated bytecode. The video
will be online at YouTube the next day. It'll make discussing the issues
below much easier.
Alex
Next step is then precompiled Groovy code in a named module. The
difference to before is that we now have to look at calls from hidden
API to hidden API of the same module and we have compile time defined
requires and exports.
Since the Groovy runtime needs to be able to call some methods by
reflection all packages, including the hidden API, will have to be
exported to the groovy runtime and of course there will be a require for
the Groovy runtime.
Let us briefly look at a call from the script module to an exported
class in another named module again... the script module will now
require the other named module, which means we have a reads here already.
* direct call works
* reflection - runtime has to add read
* indy works
* csgen in the unnamed module... the call is again done using
interfaces/classes from the runtime, meaning it should work.
Then let us look at a call from hidden/public API to hidden API of the
same module of our precompiled script.
* direct call works
* indy works
* reflection works, since everything is exported to the groovy runtime..
* csgen in the unnamed module... while the call to csgen will work
(gated by classes/interfaces of the runtime), the call from csgen to
hidden API will fail. Adding a read won´t help, end of line. A named
module would have the same problem. Solution unclear.
As for calls from the groovy runtime to the hidden API of the script
module... Some csgen problem, everything else is fine.
Finally calls from the script module to the hidden API of another
module... I can make this sometimes work with indy and with reflection.
csgen will have the known problem, direct calls will not work. Of course
there is now the question of if I need such a call... assume you did
write a program like this:
Module A, export exported
package exported
class MyExportedList extends ArrayList {
def sum(init){
def ret = init
for (it in this) ret +=it
return ret
}
}
Module B, hidden API, require A
class HiddenX {
int value
def plus(HiddenX other) { return value+other.value }
}
def list = [new HiddenX(value:1), new HiddenX(value:2), new
HiddenX(value:3)] as MyExportedList
assert list.sum(new HiddenX(value:0)) == 6
This will require the sum method in Module A to call plus on HiddenX for
the "+=". There is no interface or class for this, just the public
method. Since the class is in the hidden API of B, A will not be able to
access it.
* A direct method call is out of question for this of course, so no
static case at all.
* reflection through the groovy runtime
* indy... before I would have used the lookup object of the caller to
realize the call. Since A cannot read HiddenX, this won´t work anymore.
I would have to resort to a private "everything is allowed" constructor
in Lookup. Basically a hack, of which I won´t know if it will work in
the next JDK version. Maybe I could "steal" a lookup object from HiddenX
to avoid this, but it is very allowed to write HiddenX in another
language like Java, and then I will not be able to produce a helper
method for this. Meaning I will have to stay with the hackish
maintenance nightmare of using a private constructor.
* csgen... the sad story continues.
And btw, how am I supposed to decide if this call is allowed or not?
To sum things up:
* module runtime creation and handling beyond adding reads is unclear
* setAccessible failing on public classes from hidden APIs is
inconvenient, but at least we can find code for this, which is
compatible for multiple JDK versions
* csgen will probably have to fall back to reflection for most cases,
which means it will be a lot slower. A bigger change would be to give it
an indy backend. We would probably still suffer a performance penalty,
since we then have an additional layer with wrapping in between, that
cannot be removed by the JIT, as long as the JIT is unable to look
beyond the indy callsite itself.
* the indy part will have to use "forbidden" API now.
* runtime scripts will go into the unnamed module
* hidden APIs and non-public modifiers will be ignored by Groovy for as
long as the "forbidden" API works, which keeps current semantics
* most of the groovy runtime makes still heavy use of reflection and
will probably have to be reworked to use indy everywhere, if on JDK9
(probably JDK8+).
And I am not happy with this outcome so far.
bye Jochen
