Re: replacements for Unsafe

[Jochen Theodorou]

On 29.04.20 13:37, Remi Forax wrote:

Hi Jochen,
Per the Java spec, calling an empty static method of the inner class works.


but it has to work for any arbitrary inner class.

The Class.forName variant Kasper mentioned and I thought I tired already
(but did not) works for me.

Thanks

bye Jochen

Re: replacements for Unsafe

[Jochen Theodorou]

On 29.04.20 13:39, Kasper Nielsen wrote:

Hi Jochen,

Is there anything stopping you for doing something like:

try {
 Class.forName(innerClassName, true, initalizeClass.getClassLoader());
} catch (ClassNotFoundException e) {
 throw new ExceptionInInitializerError(e); // Should never happen
}


possibly the security manager, but it should be no real problem... and I
thought I
I tried that... looks like I did not.

Thanks, works for me!

bye Jochen

replacements for Unsafe

[Jochen Theodorou]

Hi,

when jigsaw started there was a lot of talk about providing alternatives
to what Unsafe offers. Today I am facing a problem I cannot solve
anymore it seems with this.

My problem is that I need to ensure the static block of an inner class
has been executed before I do something, because the static block
contains setup data the class will need and influence what I do with the
class. Of course this might be not the best choice, I am just trying to
keep certain functionality.

In the past I would have used Unsafe#ensureClassInitialized, but this
method has not found a replacement yet if I read JDK-8235521 :
Replacement API for Unsafe::ensureClassInitialized correctly.

Also because of Unsafe#getUnsafe, checking for the platform loader for
the caller (which is not the case for me) I cannot get the unsafe object
itself "properly".

Now I am wondering what alternatives exist. Are there any?

bye Jochen

Re: excluding transitive module

[Jochen Theodorou]

On 14.04.20 19:38, Alex Buckley wrote:
[...]

It's fine for
Library1 to require SharedApi, and for Library2 to require
SharedApiImpl, but if the exported packages overlap, then it's not fine
for Project to require, indirectly, both SharedApi (via Library1) and
SharedApiImpl (via Library2). That is, SharedApi and SharedApiImpl are
not composable.


And what do you do when you are in that situation?


If you had put them both on the classpath in JDK 8, then
you would have had a broken system. If you want to compose an
application that includes both, then one of them has to change.
There is no need to speak of "transitive" modules because `requires
transitive` has not entered the picture.


Without the module system I can just exclude one and be fine. With
module system I get into a split package problem adn have not found a
real solution yet

bye Jochen

Re: excluding transitive module

[Jochen Theodorou]

On 14.04.20 20:20, Remi Forax wrote:

Hi Jochen,
JPMS has no notion of of API and implementation of the same jar. It's a concept 
of your build tool and not something JPMS knows.

The notion of compilation dependencies and runtime dependencies is not a 
concept of JPMS but a concept of your build tools.

In term of JPMS, if you want to be able to use a jar at compile time and 
another one at runtime,
both should have the same module name, in fact they should have exactly the 
same module descriptor (the same module-info).

So in term of dependency,
- for,Maven/Gradle, the compile dependency will use SharedAPI and the runtime 
dependency SharedApiImpl
- for JPMS, both have the same module-info and if you want to depend on that 
module, just require it.

 From the Maven Central POV, obviously, you have two jars so they can not have 
the same arfifact ID (coordinate), but they contains the same module-info.class.


If the case is api and implementation and the same module name, then
using the build tool to exclude the api dependency works.

If the module name is not the same I have not found a solution. It is a
problem here because of shared packages.

I found this being a problem btw with a lot of jee libraries.

Of course I can make my project require whatever, but if it is libraries
I do not have under control things get a bit difficult without changing
them. But changing them is not an action really well supported by any
build tool I know.

bye Jochen

Re: excluding transitive module

[Jochen Theodorou]

On 14.04.20 11:09, Alan Bateman wrote:

On 14/04/2020 09:24, Jochen Theodorou wrote:

Hi all,

I am wondering if there is a solution purely in the module-info for this:

* Project requires Library1 and Library2
* SomeLibrary requires SharedApi
* OtherLibrary requires SharedApiImpl

The problem is, that it will not compile because SharedApi and
SharedApiImpl implement the same packages but not the same logic. One is
just an API, while the other is an implementation of the API.

This seems futile without first cleaning up the architectural issues.

How does SharedApi locate the implementation?


It does not, there is no SPI. It just implements the interfaces


Is this configurable or
does it always assume it's in the same run-time package as itself?


actually it does not even assume somebody else may have it. So yes.


If
the API and implementation (or default implementation) have to be in the
same run-time package then it leads to SharedApi and SharedApiImpl
needing to be combined into one module, not two.


so the best way would be to change SomeLibrary to require
SharedApiImpl... which means changing the library after the fact...
which is a tad difficult


Why is OtherLibrary requiring SharedApiImpl? This suggests that
SharedApiImpl is more than an implementation, does it have an additional
implementation specific API that OtherLibrary make use of?


There can be different implementations (it is just no an SPI
architecture). There is one that then builds on top of SharedAPI, but it
is not the implementation I need.


If you decide to combine the API and default implementation into the one
module then it will look something like this:

module api {
     exports api;
     uses spi;
}

The default implementation will be non-public classes in the same
package as the public API classes. The `uses spi` is stand in for
whatever the service type that some other implementation can provide
(assume it is indeed pluggable, why else would they be separated in the
first place?).

If you don't want to combine the API and implementation into the same
module then you'll have to move the implementation to another package so
you have something like the following fully encapsulated implementation:

module impl {
     requires api;
     provides spi with impl;
}

Different run-time packages means they can't use package-privates. If
there is an implementation-specific/extension API then impl will need to
export that package. There will be presumably be api types in the impl
specific API so the requires would change to `requires transitive api`.
OtherImpl will `requires impl`.


I see.

bye Jochen

excluding transitive module

[Jochen Theodorou]

Hi all,

I am wondering if there is a solution purely in the module-info for this:

* Project requires Library1 and Library2
* SomeLibrary requires SharedApi
* OtherLibrary requires SharedApiImpl

The problem is, that it will not compile because SharedApi and
SharedApiImpl implement the same packages but not the same logic. One is
just an API, while the other is an implementation of the API.

Scenario 1:

What would seem to be logic to me is to exclude SharedApi, but does
SomeLibrary then need its requires to be changed to SharedApiImpl? How
would I do that if I cannot change SomeLibrary?

Scenario 2:

Let us assume I could change the module-info of SomeLibrary. How would I
express that Shared API is compilation only? Is that what "requires
SomeLibrary for compilation" would be for?

bye Jochen

Re: From Gradle 6.0-rc-2 Onward, gradle-api-{$version}.jar Is Now A Working Automatic Module

[Jochen Theodorou]

I am happy you managed to do it.

bye Jochen
On 31.10.19 12:53, Plugins wrote:

Heads-up! The fixes that make Gradle's generated API JAR a
correctly-functioning automatic module have been released with Gradle
6.0-rc-2! [1]

The so-called „shaded“ JAR file is not distributed like other Gradle
artifacts. Instead, it is generated lazily whenever you're developing a
Gradle plugin. It lives at
{$GRADLE_USER_HOME}/caches/6.0-rc-2/generated-gradle-jars/gradle-api-6.0-rc-2.jar.

Sincere thanks to Cédric Champeau (melix), Sterling Greene (big-guy),
Alan Bateman, Jochen Theodorou (blackdrag) and Alex Buckley!


---


[1] http://bit.ly/GrdlisAmodule

Re: Unable to derive module descriptor for gradle-api-5.6.2.jar — Provider class moduleName=model-core not in module

[Jochen Theodorou]

On 12.10.19 00:41, Plugins wrote:

While trying to find a solution to what I lovingly call Gradle's
„Anti-JPMS“ issue [1], I discovered that even if that one particular
issue is worked around, Gradle then, frustratingly, presents yet another
hurdle to developing with the JPMS! Specifically, it co-opts the
META-INF/services mechanism to bundle a non-JAR file-spec-compliant
services configuration file.


That technically-inappropriate
META-INF/services/org.codehaus.groovy.runtime.ExtensionModule entry in
Gradle's gradle-api-{version}.jar file is used by Gradle to extend the
Groovy language; which Gradle relies on. Apparently, that service entry
extends Groovy with Java methods [2].


No idea what gradle uses here, but the standard mechanism uses now
META-INF/groovy/org.codehaus.groovy.runtime.ExtensionModule
(https://issues.apache.org/jira/browse/GROOVY-8480). So I assume Gradle
is there still on Groovy 2.4

[...]

@AlanBateman? Please can you suggest what can be done to work around
Gradle's moduleName=model-core „anti-JPMS“ blocker?


you can try upgrading to Groovy 2.5. Never tried that, no idea what
problems appear as part of that. This change in the extension module
handling is not the only one, hence the new version. Or you remove the
file and see what breaks. I think patching is out of question here...

bye Jochen

Re: Missing a wildcard open statement

[Jochen Theodorou]

On 14.01.19 17:18, Kasper Nielsen wrote:
[...]

So you need some kind of bootstrap class in every module that registers
their Lookup object somehow. And, you also have to make sure that the
bootstrap class does not make the Lookup object available to someone who
should not have access to it. This is solvable, for example, by using a
ServiceLoader.

But now, you also have X number of Lookup objects you need to keep track
of. And then someone gets the idea that you need an extension
hierarchy that crosses module boundaries. So you end up with needing one
Lookup object for one method on an object and another Lookup object on
another method on the object. Because the first method is on an abstract
class located in another modul.


Maybe I understood something wrong, but assuming

Module A
public abstract class A {
  public abstract void foo();
}

Module B (reads A)
public class B extends A {
  public void foo(){}
  public void bar(){}
}

then in Module C to call bar on an instance of B I need a Lookup object 
with the correct rights for B. But I can use the same Lookup object to 
call foo on an instance of B.


If B does not read A when loading the class B I would actually expect 
the module system to deny loading the class... never tested that though. 
Is that not the case?



In the end it might just be simpler to add an open statement for every
package in your module-info. At least for some use cases.


For many cases in which you have to load things dynamically. If not, you 
can figure out these things statically and patch the modules to have the 
right settings, which means to make a tailored module loading in the end.


bye Jochen

Re: module readability and exceptions in api

[Jochen Theodorou]

On 11.10.2018 19:57, Michał Zegan wrote:



W dniu 11.10.2018 o 19:03, Jochen Theodorou pisze:

On 10.10.2018 22:09, Michał Zegan wrote:
[...]

Oh when we are at it, a bit offtopic, can bytecode generation be
replaced with method handle usage, or method handles are still not fast
enough... or whatever?


In my experience they are fast enough in many, if not most cases. But if
you cannot do with constant handles and have to create them at runtime,
then the creation cost is killing your performance.

But they are made mostly for that, fast method calls where you don't
constantly have to create new handles? What about caching? Like
performance of putting such a handle in, say, a map or ... well whatever
more complicated than purely static usage of them...


Not sure I get your question. They are made for fast method calls, yes. 
Does not mean they are cheap to create in the first place. You can even 
build a PIC of definable length by combining multiple handles together 
in a guardWithTest - manner and get pretty good peak performance.


If your performance characteristics are that you run large amounts of 
calls only once and you do not know the handles in advance, then you 
have a problem, depending on what you compare with.


But can runtime bytecode generation still work in times of jigsaw modules?

bye Jochen

Re: module readability and exceptions in api

[Jochen Theodorou]

On 10.10.2018 22:09, Michał Zegan wrote:
[...]

Oh when we are at it, a bit offtopic, can bytecode generation be
replaced with method handle usage, or method handles are still not fast
enough... or whatever?


In my experience they are fast enough in many, if not most cases. But if 
you cannot do with constant handles and have to create them at runtime, 
then the creation cost is killing your performance.


bye Jochen

Re: The baby and the bathwater (the return)

[Jochen Theodorou]

On 03.06.2018 21:53, Jonathan Gibbons wrote:

Rémi,

Generally, you should consider the runtime "module path" to be composed 
of three elements: the upgrade module path (--upgrade-module-path), the 
system modules (--system) and the user module path (--module-path). 
Depending on your requirements, you may want to take --patch-module into 
account as well.  At compile time there is also the source path 
(--source-path or --module-source-path) to consider.


While this may seem complicated, it is analagous of the pre-Jigsaw world of

     -Xbootclasspath/p:  -bootclasspath  -Xbootclasspath/a: -classpath 
(and -sourcepath at compile time)


with one major difference... you rarely had a reason to use 
bootclasspath in any way. The need to test such situations was very very 
limited as well. jlink makes system modules a standard, classpath and 
modulepath have to be tested as well now...


bye Jochen

Re: ServiceLoader.load(Class, ClassLoader) does not load services exposed in modules and loaded by parent CL

[Jochen Theodorou]

Am 24.05.2018 um 19:33 schrieb Robert Scholte:
[...]

I can quote Alan:
Rhino used to be the JS engine in older releases and that may have been 
in rt.jar and so loaded by the boot loader. When Nashorn replaced it (in 
JDK 8) then it was configured to be defined to the extension class 
loader so this is why the code snippet doesn't find it.


how is this handles in JDK 9 now?

bye Jochen

Re: Avoiding sun.misc.Unsafe and embracing modules in Java libraries: missing links

[Jochen Theodorou]

On 03.04.2018 21:26, Henri Tremblay wrote:
[...]

For completeness, there are 4 ways to create a class without calling a
constructor right now that I'm aware of:

- Unsafe.allocateInstance


which is supposed to go away at some point


- sun.reflect.ReflectionFactory.newConstructorForSerialization (my
favorite because it's the fastest one)


which afaik works in java9 but is also one of those critical doomed APIs


- Generate an extending class forgetting to call the super constructor
(so it's not exactly that same class that is instantiated). It requires
-Xverify:none


Is this really an option for a production environment?


- Generate a class extending MagicAccessorImpl that will then
instantiates the wanted class but calling the wrong constructor


Is jdk.internal.reflect.MagicAccessorImpl still usable in Java9+? I 
thought this is no longer exported


Under the premise that all critical API usages will be removed in the 
future and replacement APIs will be created I think we might indeed 
still miss something here


bye Jochen

Re: modules and resources [ was: suggested document enhancement]

[Jochen Theodorou]

Am 02.03.2018 um 15:14 schrieb Bernard Amade:
[...]

2) then what is the simplest way to deploy jars if clicking on a jar does not 
work anymore with modules?
(scripts are platform dependent, so are Jlink images ,...and what happens with 
jars delivered through java webstart?  should we consider that  java webstart 
is going to be  deprecated?)


from 
http://www.oracle.com/technetwork/java/javase/9-deprecated-features-3745636.html



docs/release_notes
 Java Deployment Technologies are deprecated and will be removed in a future release 
Java Applet and WebStart functionality, including the Applet API, The Java plug-in, the Java Applet Viewer, JNLP and Java Web Start including the javaws tool are all deprecated in JDK 9 and will be removed in a future release


So they are deprecated in 9 according to this, I think 10 prints a 
warning still.


bye Jochen

Re: Unsafe and JPMS

[Jochen Theodorou]

On 19.01.2018 19:31, Jeremy Manson wrote:
[...]

I generally agree with the principle of making real APIs to do what we need
to do with Unsafe and migrating.  My concern is not the removal of Unsafe,
it's that the replacement won't be equivalent if it doesn't let you cross
module boundaries.


It won't be equivalent. It can't be or the module system enforced 
encapsulation, the big deal of the jpms, would be rendered useless. But 
maybe the alternatives are good enough for you. You will still have to 
identify the problems points and see if you can replacement with 
something else.


bye Jochen

Re: How does one handle a java method which returns a non-public sub-type via reflection?

[Jochen Theodorou]

Am 18.01.2018 um 14:59 schrieb jeffrey kutcher:

  In this example, if I can access and execute the method 
vbox.getChildren().add() with no warnings or errors, which is accessing a 
non-public class


In Java the static type is the important one. Of course you can do

VBox vbox =  // returns implementation from nonexported package
vbox.getChildren().add()

This will work as long as VBox is accessible. the getChildren call is 
not done using the implementation type, but VBox. And for this it does 
not matter that vbox has a "forbidden" type at runtime at all. 
Conclusively you have to do more at runtime if all you have is the 
runtime type of vbox. You have to check if this type is accessible to 
you and if not go to a super class. You would then maybe also end up in 
Vbox and invoke the method you get for VBox, not for the implementation 
of VBox. You still do a virtual method call, so the implementation 
method is still called


then I should also able to access that very same class and execute the same method using reflection with no warnings or errors and expect the same results. If not, the inconsistency may be a bug, feature or broken. 


design choice I would claim.


Maybe that's the way it's suppose to work. Inconsistency however tells me 
something isn't quite right and if ignored will cause more issues later on.


That is because Reflection has been declared as broken in that it has a 
tradition of allowing too much. Now this has been restricted, leading to 
problems. But in this case you just have to use the type the Java 
compiler would, which is VBOX.



In this particular case, a lock down will completely stop currently working 
code.


You are not alone here.

bye Jochen

Re: How does one handle a java method which returns a non-public sub-type via reflection?

[Jochen Theodorou]

Am 12.01.2018 um 22:49 schrieb Michał Zegan:

well you rather know what getChildren returns, so you know what the
returned object is, like generally.


but as soon as the method you want to call is not on the statically 
declared type, nor within a set of known statically types you are having 
a problem. There is no good way (good in terms of usability and 
performance) to get an arbitrary valid method object without static 
information. You are always forced to traverse the hierarchy to find the 
most accessible type. You will have to check access... and then of 
course there the two bad scenarios where invocation and search are not 
the same class or actual caller and caller are not equal. Which leads to 
a lengthy search using reflection to find a method, transfer it to a 
method handle and then let the invoker do the actual call.


bye Jochen

Re: Module system and services directory in META-INF

[Jochen Theodorou]

On 27.12.2017 13:55, Alan Bateman wrote:

On 27/12/2017 12:22, Cédric Champeau wrote:

:

1. is the spec saying somewhere that no-one should ever use
`META-INF/services` for something else than a service for 
`ServiceLoader`?

If not, then ModuleFinder should probably be patched to recognize that
sometimes it's not the case.
The META-INF/services directory has been used for service configuration 
files for a long time (I think originally specified in Java SE 1.3). 
Adding properties files or other resources in that location may cause 
problems for tools and libraries that scan that location.


At least in the last 4 years nobody bothered reporting a problem with this.

I haven't read the thread that you linked to but I assume it's about JAR 
files on the module path that are treated as automatic modules. In that 
case, the relevant part of the ModuleFinder spec is:


"The contents of entries starting with META-INF/services/ are assumed to 
be service configuration files (see ServiceLoader). If the name of a 
file (that follows META-INF/services/) is a legal class name then it is 
assumed to be the fully-qualified class name of a service type. The 
entries in the file are assumed to be the fully-qualified class names of 
provider classes."


spec by javadoc... takes time getting used to it. At least now I know 
where this is mentioned



2. the 2d aspect of the question is what Jochen described as "would the
service loader infrastructure let us unload services".

Jochen's mail mentions loading modules dynamically so I assume this 
means module layers. If so then unloading is possible and it works well 
with services.


If not keeping a reference to layer, class and loader is enough, then I 
am happy ;)


bye Jochen

Re: Module system and services directory in META-INF

[Jochen Theodorou]

On 27.12.2017 13:22, Cédric Champeau wrote:
[...]

1. is the spec saying somewhere that no-one should ever use
`META-INF/services` for something else than a service for `ServiceLoader`?
If not, then ModuleFinder should probably be patched to recognize that
sometimes it's not the case.


or the javadoc of ServiceLoader should be cleared up as well as the jar 
file spec should contain at least a pointer to the javadoc for "further" 
specification.



2. the 2d aspect of the question is what Jochen described as "would the
service loader infrastructure let us unload services".


I assume, that since the ServiceLoader is looking up services lazy 
(which btw speaks against a requirement in 1), then a reload (which 
really is more a reset) could in theory free resources, since the 
services would first have to be rediscovered. Not sure if that is good 
enough for us.


And before I start ranting how the extensive usage of CallerSensitive 
makes a runtime practically require "all the rights" to run properly and 
thus degrading the java module system or the runtime impossible I better 
stop :(


bye Jochen

Re: Module system and services directory in META-INF

[Jochen Theodorou]

On 26.12.2017 19:30, Jochen Theodorou wrote:

hi all,

Do all files in META-INF/services now have to be well formed according 
to the SPI strucutres in Java, or is it still valid to have other files 
in there as well? Before the module system it was no problem to have in 
there service decriptors, that are services, but do not use the 
ServiceProvider structure. Nothing was disturbed as long as nobody tried 
to load it that way. Now this seems to be different and I am wondering 
where this got specified.


Because the spec does not try to define file layouts so much I have the 
feeling that this is not really specified by the java module system. Was 
it then a decision by some maven plugin writers or maybe even the javac 
guys?


I found 
https://docs.oracle.com/javase/8/docs/technotes/guides/jar/jar.html#Service_Provider 

Is there no Java 9 version? Also the spec is not specific enough for 
this as it sounds more like implicit assumption than specification


bye Jochen

Module system and services directory in META-INF

[Jochen Theodorou]

hi all,

Do all files in META-INF/services now have to be well formed according 
to the SPI strucutres in Java, or is it still valid to have other files 
in there as well? Before the module system it was no problem to have in 
there service decriptors, that are services, but do not use the 
ServiceProvider structure. Nothing was disturbed as long as nobody tried 
to load it that way. Now this seems to be different and I am wondering 
where this got specified.


Because the spec does not try to define file layouts so much I have the 
feeling that this is not really specified by the java module system. Was 
it then a decision by some maven plugin writers or maybe even the javac 
guys?


We have not yet decided if we want to go with the service provider 
mechanism, because I have doubts, that unloading of service classes and 
class loaders associated with those classes will still unload if we use 
a structure as global as the service provider structure. I actually even 
fail to imagine a structure in which you can say, that at this point you 
do no longer require the services in a general and global service 
provider situation - unless there is something like an "unload". In 
other words... if you load a module dynamically, can you let it define 
services and can those services then be unloaded again?


bye Jochen

Should we create a compiler independent module tool?

[Jochen Theodorou]

Hi all,

I am wondering for quite a while already if there should not be an javac 
(and possibly jdk9) independent tool for handling module. I think the 
idea for such a tool would be a bit against several decision that have 
been taken by the jigsaw project, so bare with me, I am trying to be 
constructive here not trolling ;)


What such a tool could do:
* after compiling sources you can add module information
* let the tool check for module access rules
* read and extract module information from multi release jars
* allow alternative compiler to be "enhanced" with jdk9 module creation 
capabilities.. for example joint compilers
* let any build tool use the normal classpath logic to add module 
information in a later step
* being independent of the JDK means (and only on ASM6) to able to use 
JDK7 or 8 and still to be able to produce a module, if no JDK9 specific 
actions are to be taken
* provide a base for other tools for things missing in JDK9, like the 
proper handling of information extraction from multirelease jars (see 
'Scanning Modules(take 2)?' on this list)

* we can have some JDK10 functions in JDK9

If there is interest to create such a tool with a nice license (Apache 2 
for example) I would be happy to try to channel efforts and help 
creating this. I may end up doing this by myself, but then it is likely 
to be used only by my projects and I think it would be a shame not to 
have this wider spread and more people involved.


bye Jochen

Re: trySetAccessible​

[Jochen Theodorou]

On 11.07.2017 12:11, Alan Bateman wrote:

On 11/07/2017 10:16, Uwe Schindler wrote:

[...]


But now it is impossible for us to check, if something is not open by
default.

Module::getDescriptor will return the module descriptor for named
modules. So in this case, Object.class.getModule().getDescriptor()
returns the ModuleDescriptor for java.base so you can inspect the
packages and which are exported and/or opened before being changed by
runtime. So there is enough in the API to determine which packages have
been opened at runtime (either via CLI options or APIs).


we need that on the level of a class member. Not every method in a class 
from an exported package is accessible or has been made accessible via 
runtime mechanisms.


bye Jochen

Re: trySetAccessible​

[Jochen Theodorou]

On 10.07.2017 12:06, Alan Bateman wrote:

On 10/07/2017 10:59, Cédric Champeau wrote:

This is not really practical. Basically it means that for every Gradle
build script on earth, we would either choose to make them fail on JDK
9, or be bloated with warnings, that are actually handled.

My mail was just explaining why it returns `true`.

The right way to get rid of these warnings is to fix the underlying issues.


see my other mail

bye Jochen

Re: trySetAccessible​

[Jochen Theodorou]

On 10.07.2017 10:07, Alan Bateman wrote:
[...]

Remember the purpose of the warnings is to make you or your users aware
that the code will behave differently when access to JDK internals, or
other so-called illegal access, is denied.


In the past we did create a MetaClass, which contains all the accessible 
methods and properties. This meta class is used for method selection 
during invocation, it is used for querying and for runtime meta 
programming.


These warnings now force us to change the semantics and not show the 
accessible methods, but all methods. Invocation will then have to make 
it accessible... something we moved to meta class creation because it 
slows down the invocation mechanism. And then we will still get false 
warnings in reflective code.


Example:

module A:

class Foo {
  protected void foo(){}
}

Groovy program:

class X extends Foo {
  def bar() {return {foo()}}
}

if Foo is in an exported package, then it is legal for X to call foo, 
even though it is protected. But bar() does not directly call foo, it 
will do so in a Closure (not functional closure, not lambdas). In a 
Closure the implicit this has no fixed meaning and is realized as 
anonymous inner class. With our current protocol the call to foo ends up 
in the meta class logic, which then executes the method call. This will 
then cause a warning "WARNING: An illegal reflective access operation 
has occurred".


Now tell me how this is useful for the user. The call is supposed to be 
legal after all. Still we will get a warning. How am I supposed to tell 
our users about correct and wrong warnings?


Of course I am aware that I am supposed to give in a Lookup object and 
return a MethodHandle instead. But without changes to the MOP API this 
cannot be done. And that means Groovy 1.0 code will, for the first time 
in our history, no longer really work on a new Groovy version, once this 
change is done. And here I am not even sure that "giving in the Lookup 
object" is really what I should do. This Lookup object would have full 
access to X after all (I need to be able to call private methods too you 
know). And that does not sound right either


bye Jochen

Re: trySetAccessible​

[Jochen Theodorou]

On 10.07.2017 10:49, Cédric Champeau wrote:

I second Uwe's comment here: I was surprised as well, I expected the
semantics of `trySetAccessible` to be: "let me know if I can do this,
respecting the semantics of modules, and if not, return false"


I suspect canAccess is supposed to be used here... of course the 
semantics of that method are not enough for us.


bye Jochen

trySetAccessible​

[Jochen Theodorou]

Hi all,

since the JVM now prints warnings if you call setAccssible there was the 
advise to move to trySetAccessible​ to prvent these warnings. Now I am 
told those warnings appear here as well.


Now my question would be about what the intended behaviour here is. in 
what mode is a trySetAccessible​ supposed to cause the JVM to issue a 
warning or other message about a module accessing something?


And if the warnings are the intended behaviour, then what is the way to 
avoid this if I cannot just stop trying?


bye Jochen

Re: Compiling with automatic modules

[Jochen Theodorou]

ok, thanks for the answers

On 30.05.2017 23:43, Alex Buckley wrote:

On 5/30/2017 2:08 PM, Jochen Theodorou wrote:

On 30.05.2017 21:42, Alex Buckley wrote:

On 5/26/2017 4:12 AM, Jochen Theodorou wrote:

On 26.05.2017 01:04, Alex Buckley wrote: [...]

The semantics of an observed JAR without module-info.class are
 specified as part of JPMS resolution, and JLS 7.3 explicitly
defers to that, so I believe it is clear how a compiler must
behave when a modular compilation unit 'requires' a module that
turns out to be automatic. (Of course a big part of the
migration story is that the requirer is unaware of whether the
requiree is automatic or explicit.)


Isn't the consequence that I can write a compiler which does only
allow named modules?


You mean a compiler that understands named module and does not
understand unnamed modules?


actually I was wondering more about automatic modules and inexact in
my question.


No, per JLS 7.7.5: "An implementation of the Java SE Platform must
support at least one unnamed module."  The mandates for unnamed
modules in 7.7.5 are essentially identical to the historical
mandates for unnamed packages in 7.4.2.


""" An implementation of the Java SE Platform must support at least
one unnamed module. An implementation may support more than one
unnamed module, but is not required to do so. Which ordinary
compilation units are associated with each unnamed module is
determined by the host system.

The host system may associate ordinary compilation units in a named
package with an unnamed module. """

OK, from this I understand there must be at least one unnamed module.
 Nothing about automatic modules.


Correct. Automatic modules are named modules known to the JPMS, just
declared implicitly by the JPMS rather than explicitly in the Java
language. Where a named module IS declared explicitly in the Java
language, it may reference, in its 'requires' directives, any other
named module known to the JPMS, regardless of whether that other named
module is declared implicitly or explicitly.


What comes after that is a bit confusing to me. Could I for example
say that only compilation units, that declare to be part of a package
with the name "unnamed" will be part of the unnamed module?


Yes, the host system can choose to associate those compilation units
with an unnamed module if it wishes. See JLS9 7.2 and 7.3, paying
attention to the flexibility granted for ordinary compilation units (the
ones in your paragraph) versus no flexibility for modular compilation
units:

-
Each host system determines which compilation units are observable in a
particular compilation (§7.3). Each host system also determines which
observable compilation units are associated with a module.

...

The host system also determines which observable ordinary compilation
units are associated with a module, except <<java.* stuff>>.

The host system must determine that an observable modular compilation
unit is associated with the module declared by the modular compilation
unit.
-


I mean, I understand that the "which" refers to the way the files are
given to javac... But it feels like the JLS allows here many other
variants as well.


Correct. The JLS is a language spec, not a compiler spec. With the
exception of the rule for 'public' types in 7.6, the JLS has
historically imposed very few constraints on a compiler ("host system").

Alex

Re: Compiling with automatic modules

[Jochen Theodorou]

On 30.05.2017 21:42, Alex Buckley wrote:

On 5/26/2017 4:12 AM, Jochen Theodorou wrote:

On 26.05.2017 01:04, Alex Buckley wrote:
[...]

The semantics of an observed JAR without module-info.class are specified
as part of JPMS resolution, and JLS 7.3 explicitly defers to that, so I
believe it is clear how a compiler must behave when a modular
compilation unit 'requires' a module that turns out to be automatic. (Of
course a big part of the migration story is that the requirer is unaware
of whether the requiree is automatic or explicit.)


Isn't the consequence that I can write a compiler which does only allow
named modules?


You mean a compiler that understands named module and does not
understand unnamed modules?


actually I was wondering more about automatic modules and inexact in my 
question.



No, per JLS 7.7.5: "An implementation of the
Java SE Platform must support at least one unnamed module."  The
mandates for unnamed modules in 7.7.5 are essentially identical to the
historical mandates for unnamed packages in 7.4.2.


"""
An implementation of the Java SE Platform must support at least one 
unnamed module. An implementation may support more than one unnamed 
module, but is not required to do so. Which ordinary compilation units 
are associated with each unnamed module is determined by the host system.


The host system may associate ordinary compilation units in a named 
package with an unnamed module.

"""

OK, from this I understand there must be at least one unnamed module. 
Nothing about automatic modules.


What comes after that is a bit confusing to me. Could I for example say 
that only compilation units, that declare to be part of a package with 
the name "unnamed" will be part of the unnamed module? I mean, I 
understand that the "which" refers to the way the files are given to 
javac... But it feels like the JLS allows here many other variants as well.


bye Jochen

Re: Will split-packages ever be supported?

[Jochen Theodorou]

On 30.05.2017 16:21, Remi Forax wrote:
[...]

the big monolithic module is a good transition solution,
i've done that with Aether to get access to Maven Central inside a fully 
modularized project,
but it's like with automatic modules, it's just a temporary solution that you 
can use in your application and not something you can do if you want to publish 
a library.


I see it more as intermediate solution... and like it is with solutions 
like that, it will be the final solution for a long time. Maybe I would 
go with two flavors for the modules. One is the big monolith, the other 
is after a jar has been freed from the split package problem. In that 
case your project can depend on the small module, which will 
transitively depend on the shrunken monolith. This allows the project a 
transition phase of undetermined length and shows it is still future 
proof. One could even consider using a bytecode rewriting tool and let 
the monolith use the old package names.


Probably because nobody really cared about how easily the Java platform 
can be exploited. Well, I dare to say, that we have been especially bad 
here. Not that any of the exploits I know the specifics about, would not 
be working anymore in JDK9... sorry I digress.


Of course if you are in a situation like we are, then all this will not 
help, because we have split package problems with third party projects. 
Our own fault, really, yes... but the split package problem is not 
something many people have been aware of it seems. Especially since in 
our case it was really only a name space question, not a question about 
access rights at all.


bye Jochen

Re: Will split-packages ever be supported?

[Jochen Theodorou]

On 30.05.2017 14:16, Alan Bateman wrote:

On 30/05/2017 11:52, wzberger wrote:


Our project has around 45 jar libraries and we currently using
split-packages as simplest modularization concept. It's a desktop
project with different kind of controls - all have in common that they
are located in the same package e.g. 'com.swing'. So users can add
only required libraries to their project

[...]

To your example, then if the project can't be restructured to address
the split packages issues then it will need to stay on the class path.
There's nothing wrong with that, it should continue to work as before.


Of course staying on the classpath means to get replaced in the future. 
So I don't agree with you here: there is a lot wrong with that imho. 
Nobody really cares in the end if you are used as module or not, but if 
you cannot do it, you are being replaced with something that can, 
because you are not future proof. Breaking the code of about everyone 
depending of you can have of course the same effect.


I mean... assume you want to write a named module... how do you depend 
on a class in an unnamed module? You do not. You can for example not 
extend such a class. javac would not allow that. I can of course still 
compile in classpath mode and then use a yet to be written tool, that 
generates the module information independent of javac... It means your 
module would have an undeclared requirement. I think that is a 
complication you normally would not want to have. It is probably more 
easy to drop the framework and use something else then.


Oh, you forgot the other alternative... make it a big monolith and 
squash the 45 jars into one big module. Then it can move away from the 
classpath too. Sure, making a monolithic module really defies the 
purpose of the module system, but it is still better than breaking code 
or requiring the users to do module system acrobatics.


bye Jochen

Re: Compiling with automatic modules

[Jochen Theodorou]

On 26.05.2017 01:04, Alex Buckley wrote:
[...]

The semantics of an observed JAR without module-info.class are specified
as part of JPMS resolution, and JLS 7.3 explicitly defers to that, so I
believe it is clear how a compiler must behave when a modular
compilation unit 'requires' a module that turns out to be automatic. (Of
course a big part of the migration story is that the requirer is unaware
of whether the requiree is automatic or explicit.)


Isn't the consequence that I can write a compiler which does only allow 
named modules?


bye Jochen

Re: Proposal: Allow illegal reflective access by default in JDK 9

[Jochen Theodorou]

On 19.05.2017 15:38, Stephen Felts wrote:
[...]

Here are a few public ones.

org.python.core.PyJavaClass - Jython

com.sun.xml.bind.v2.runtime.reflect.opt.Injector – standalone jaxb

com.sun.xml.ws.model.Injector – standalone jaxws

net.sf.cglib.core.ReflectUtils$ - cglib



Summarizing the use cases:

1. The code calls setAccessible and correctly catches and ignores the exception 
for those that fail.  It would be a big change to make this call setAccessible 
lazily.

2. The code continues to use the JDK8 approach and if that fails it falls back 
to an approach that works on JDK9 and/or other non-Hotspot platforms.  In some 
cases, this code has been around for a decade or more (e.g., to handle 
platforms without Unsafe).


with the same summary you can include Groovy here.

bye Jochen

Re: Some suggested patches and improvements

[Jochen Theodorou]

On 12.05.2017 18:49, Michael Nascimento wrote:

#1-#3, IMHO, meet the needs of a niche who knows how to work around these
issues using other ways (although inconvenient).


I see me as someone needing this, and only having hints about how to 
make it work, that are based on annotation processors, which is no 
acceptable solution at all.


 bye Jochen

Re: Need help implementing Java modules

[Jochen Theodorou]

On 09.05.2017 20:51, Ralph Goers wrote:

I am attempting to modularize Log4j and am running into some trouble
understanding how this can work.

Log4j supports many Appenders, such as the FlumeAppender,
KafkaAppender, CassandraAppender, and JPAAppender. Each of these use
their respective libraries to cause the log events to be delivered to
their destinations. All of these frameworks perform logging and so
would have a dependency on a logging framework. For example, the
FlumeAppender passes the LogEvent to an embedded Flume agent that
then sends the event to a remote Flume node. The embedded agent
performs logging using the Log4j 1 API and also uses other libraries
that perform logging. So if Flume was modularized it would require
some logging module. That module would almost certainly route logging
calls back to Log4j 2 and so would require Log4j 2 and result in a
circularity.  In the same manner, the KafkaAppender uses the SLF4J
API. When the implementation of that is log4j-slf4j-impl a
circularity will occur. Since the Java module system forbids
circularities I don’t understand how Log4j (or Logback) can
realistically be modularized.


I will try to explain and depend on others to correct me if I am wrong..

Assuming you have your log4j-base module, which will also provide an 
interface to communicate with a implementation of log4j. A FlumeAppender 
would then for example depends on that interface, but log4j-base would 
not depend on the Flume Appender. So the question is how log4j-base can 
then get the FlumeAppender... which is what SPI is for. So every 
Appender would then be a service provider.



We also noticed that System.LoggerFinder [1] now provides a new way
to integrate an external logging framework with Java. If Log4j
creates the service provider for that how will that not cause a
runtime circularity since java.base now is bound to log4j-api and/or
log4j-core and both of those require java.base.

Ralph

1.
http://docs.jboss.org/hibernate/orm/4.3/topical/html/logging/Logging.html



SPI again. Since Log4j depends on java.base for compilation, but 
java.base does still compile without log4j-api or -core it is not a 
circular dependency.


bye Jochen

Re: Alternatives for naming automatic modules, and a proposal (#AutomaticModuleNames)

[Jochen Theodorou]

On 25.04.2017 16:38, David M. Lloyd wrote:
[...]

I think that Maven- and Gradle-based tooling will appear to fill this
need.  The only correct solution (automatic modules or not) to
modularizing arbitrary groups of artifacts will likely involve a
separate assembly step which will inevitably include writing or
rewriting module descriptors.  I expect that the tooling will use the
artifact dependency graph, module descriptors and/or other metadata
present on the artifacts, and some manual configuration as input, and
then produce as output a collection of corrected/re-written module JARs
(and probably a complete wiring report that can be checked over by human
eyes, probably with admonitions relating to potential or definite
problems).


Did you ever think what that means for build times?

If the build tools have to become so powerful to handle all the module 
aspects that the can do dependencies and all that, I really expect them 
to change over to java compilation without the module aspect and at it 
later on when creating a module jar only



I really don't see any other way things could possibly work consistently
for any application which uses external JARs.  Automatic modules are
just going to be a stumbling block to reaching this conclusion.


And let´s not forget about those applications that won´t work as named 
module... after all your read - ability is quite different then.


And another thought... if I make a library, that depends on an automatic 
module... for me that would mean in consequence to employ package 
rewriting and make a fat jar. Otherwise... how can I every know anything 
about the version of the dependency



OSGi, by the way, avoids this situation somewhat more effectively than
Jigsaw can, mainly because of three factors: resolving by package
instead of by bundle name, performing the bulk of resolution work at run
time, and by the simple fact that it is less invasive towards
reflection, thus there is more "wiggle room".

This isn't to say that OSGi is an ideal solution of course, but I just
wanted to point out why the OSGi ecosystem can generally get away with
publishing artifacts containing descriptor information, while Jigsaw
probably won't be able to (or at least, not as cleanly).


agreed

bye Jochen

Re: feedback on --permit-illegal-access

[Jochen Theodorou]

On 07.04.2017 09:11, Alan Bateman wrote:

On 07/04/2017 00:09, Jochen Theodorou wrote:


Hi,

so today I found finally _JAVA_OPTIONS to get our gradle build running
and of course I used --permit-illegal-access and I thought I give some
feedback here.

Running a clean test on our build will result in 44531 warning
messages. Of which 6394 are unique. of course some of those warnings
are actually from gradle and some are from xstream, but that makes
maybe 21 less.

Why do we have such a huge amount of warnings? That is because of the
way we are building our meta classes, which we still have to change
and use JDK9 special code (something we tried to avoid)

>

The implementation makes a best effort to avoid emitting duplicate
warnings. Are these 44531 warnings coming from one run of the VM or is
this the total from many `java` commands run in the build? Another
possibility is code generation. The filtering of duplicates is based on
the perpetrator's call stack and the victim. If Groovy is generating a
lot of classes and the code is say using a shared utility class to hack
in then these usages would not be unique.


both I guess. Many of our tests are in script form, which means they are 
runtime compiled and executed. The tests are executed in parallel, I 
guess on my computer it was around 8 parallel running JVMs. Most 
messages are reported for the same class though, because that is kind of 
our base class for the meta class initialization process.



Running with -Dsun.reflect.debugModuleAccessChecks=access will give you
a full stack trace at each warning and should help debug this.


yes, will check that soon

bye Jochen

what is undesired usage of the Java API?

[Jochen Theodorou]

Hi,

I think the recent discussion about the agent loading mechanism do show 
one fundamental problem that many have with the changes through the 
restricted module system. That is, it is unclear what undesired usage of 
the Java APIs actually is.


So let me write down an unfinished list as discussion base (jigsaw):
(1) usage of internal modules, classes, methods or fields
(2) transforming any of (1)
(3) non-public elements are not to be accessed if normal Java does not 
allow for it


compared with my personal old list (javaSE):
* any sun.* classes

well.. my extended list actually was (library):
* any sun.* classes
* requiring agents
* requiring command line arguments
* requiring special configurations

And in my case, since I violate (1) in the jigsaw list, I get into 
trouble with my old library list. Which means I am in the confusing 
situation of having to do things now, that I considered bad before, 
because of jigsaw.


Anyway... I am sure the jigsaw list above is something to be refined and 
I would love to see people helping me with that.


bye Jochen

Re: Better tools for adjusting to strong encapsulation

[Jochen Theodorou]

I will interpret your answer as that there is no additional per method 
invocation cost due to this.


bye Jochen

On 23.03.2017 09:12, Alan Bateman wrote:

On 22/03/2017 21:07, Jochen Theodorou wrote:


the warnings you are seeing are from the groovy runtime trying to
determine - in a pre JDK9 compatible way - if it is allowed to call
the methods on Objecz. This happens in a first step by setAccessible
on all methods using the array version, and if that fails by using the
normal setAccessible on each method (which will contribute to much
longer startup times). At no point our runtime or the program you
wrote are trying to call finalize, clone or registerNatives.

This sounds like Object.class.getDeclaredMethods() and then invoking
setAccessible(true) on every method, is that right? If so then it will
trigger warnings for non-public methods. Is there any reason why this
code couldn't just look at the modifiers? Claes mentions the new
canAccess which might be useful going forward.



The only one we are to be blamed for is
MethodHandles$Lookup(java.lang.Class,int), for which I haven´t
implemented the alternative yet

If you did other programs... like for example running a gradle
build... you would see many many more such lines

Lots of lines but not clear to me that it's Gradle that wants to hack
into every non-public member of java.util.ArrayList.

$ gradle
NOTE: Picked up the following options via JDK_JAVA_OPTIONS:
  --permit-illegal-access
WARNING: --permit-illegal-access will be removed in the next major release
WARNING: Illegal access by org.gradle.internal.reflect.JavaMethod
(file:/gradle-3.4.1/lib/gradle-base-services-3.4.1.jar) to method
java.lang.ClassLoader.getPackages() (permitted by --permit-illegal-access)
Starting a Gradle Daemon, 1 stopped Daemon could not be reused, use
--status for details
WARNING: Illegal access by org.gradle.internal.reflect.JavaMethod
(file:/gradle-3.4.1/lib/gradle-base-services-3.4.1.jar) to method
java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int)
(permitted by --permit-illegal-access)
WARNING: Illegal access by
org.codehaus.groovy.reflection.CachedClass$3$1
(file:/gradle-3.4.1/lib/groovy-all-2.4.7.jar) to method
java.lang.Object.finalize() (permitted by --permit-illegal-access)
WARNING: Illegal access by
org.codehaus.groovy.reflection.CachedClass$3$1
(file:/gradle-3.4.1/lib/groovy-all-2.4.7.jar) to method
java.lang.Object.clone() (permitted by --permit-illegal-access)
WARNING: Illegal access by
org.codehaus.groovy.reflection.CachedClass$3$1
(file:/gradle-3.4.1/lib/groovy-all-2.4.7.jar) to method
java.lang.Object.registerNatives() (permitted by --permit-illegal-access)
WARNING: Illegal access by org.codehaus.groovy.vmplugin.v7.Java7$1
(file:/gradle-3.4.1/lib/groovy-all-2.4.7.jar) to constructor
java.lang.invoke.MethodHandles$Lookup(java.lang.Class,int) (permitted by
--permit-illegal-access)
WARNING: Illegal access by org.gradle.internal.reflect.JavaMethod
(file:/gradle-3.4.1/lib/gradle-base-services-3.4.1.jar) to method
java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int)
(permitted by --permit-illegal-access)
WARNING: Illegal access by
org.codehaus.groovy.reflection.CachedClass$3$1
(file:/gradle-3.4.1/lib/groovy-all-2.4.7.jar) to method
java.util.AbstractCollection.hugeCapacity(int) (permitted by
--permit-illegal-access)
WARNING: Illegal access by
org.codehaus.groovy.reflection.CachedClass$3$1
(file:/gradle-3.4.1/lib/groovy-all-2.4.7.jar) to method
java.util.AbstractCollection.finishToArray(java.lang.Object[],java.util.Iterator)
(permitted by --permit-illegal-access)
WARNING: Illegal access by
org.codehaus.groovy.reflection.CachedClass$3$1
(file:/gradle-3.4.1/lib/groovy-all-2.4.7.jar) to method
java.util.AbstractList.subListRangeCheck(int,int,int) (permitted by
--permit-illegal-access)
WARNING: Illegal access by
org.codehaus.groovy.reflection.CachedClass$3$1
(file:/gradle-3.4.1/lib/groovy-all-2.4.7.jar) to method
java.util.AbstractList.removeRange(int,int) (permitted by
--permit-illegal-access)
WARNING: Illegal access by
org.codehaus.groovy.reflection.CachedClass$3$1
(file:/gradle-3.4.1/lib/groovy-all-2.4.7.jar) to method
java.util.AbstractList.rangeCheckForAdd(int) (permitted by
--permit-illegal-access)
WARNING: Illegal access by
org.codehaus.groovy.reflection.CachedClass$3$1
(file:/gradle-3.4.1/lib/groovy-all-2.4.7.jar) to method
java.util.AbstractList.outOfBoundsMsg(int) (permitted by
--permit-illegal-access)
WARNING: Illegal access by
org.codehaus.groovy.reflection.CachedClass$3$1
(file:/gradle-3.4.1/lib/groovy-all-2.4.7.jar) to method
java.util.ArrayList.add(java.lang.Object,java.lang.Object[],int)
(permitted by --permit-illegal-access)
WARNING: Illegal access by
org.codehaus.groovy.reflection.CachedClass$3$1
(file:/gradle-3.4.1/lib/groovy-all-2.4.7.jar) to method
java.util.ArrayList.access$000(java.util.ArrayList) (permitted by
--permit-illegal-access)
WARNING: Illegal access by
org.codehaus.groovy.reflection.CachedClass$3$1
(file

Re: Better tools for adjusting to strong encapsulation

[Jochen Theodorou]

On 23.03.2017 01:16, Claes Redestad wrote:
[...]

the warnings you are seeing are from the groovy runtime trying to
determine - in a pre JDK9 compatible way - if it is allowed to call the
methods on Objecz. This happens in a first step by setAccessible on all
methods using the array version, and if that fails by using the normal
setAccessible on each method (which will contribute to much longer
startup times). At no point our runtime or the program you wrote are
trying to call finalize, clone or registerNatives.


For determining capabilities in this manner it might be possible to
use the new AccessibleObject.canAccess[1], which should avoid any
warnings?

/Claes

[1]
http://download.java.net/java/jdk9/docs/api/java/lang/reflect/AccessibleObject.html#canAccess-java.lang.Object-


I guess so. I mean normally we request that you open up the Objects to 
the runtime and the runtime then decides by itself if class X has access 
to class Y via runtime. canAccess will avoid the warnings, and 
completely avoid them I think for the case of everything being on the 
classpath. It will not avoid the add-opens required for the runtime to 
access objects from class X in module M to class Y in the same Module 
via Groovy runtime. Because even if we use MethodHandles, the handle is 
still produced from a Method object in reflection and that will mean we 
still require the runtime accessing this.


And actually I don't think we can exclude the reflective part, because 
we have to do runtime method selection. We cannot query for a method 
directly, we have to query for all methods of a certain name, which is 
impossible, so we have to take a look at all methods of a class. And I 
mean independent of the accessibility modifier.


So does canAccess help? Yes, in the classpath scenario, not in the 
module scenario. But at least that scenario is currently not likely


bye Jochen

Re: Better tools for adjusting to strong encapsulation

[Jochen Theodorou]

On 22.03.2017 12:23, Alan Bateman wrote:
[...]

I assume your interest is Groovy where there are indeed some warnings
that look like Groovy is to blame. For example, I get the following with
a trivial script containing`println "Hello world!"` :

$ groovy hello.groovy
NOTE: Picked up the following options via JDK_JAVA_OPTIONS:
   --permit-illegal-access
WARNING: --permit-illegal-access will be removed in the next major release
WARNING: Illegal access by org.codehaus.groovy.reflection.CachedClass
(file:/groovy-2.4.10/lib/groovy-2.4.10.jar) to method
java.lang.Object.finalize() (permitted by --permit-illegal-access)
WARNING: Illegal access by org.codehaus.groovy.reflection.CachedClass
(file:/groovy-2.4.10/lib/groovy-2.4.10.jar) to method
java.lang.Object.clone() (permitted by --permit-illegal-access)
WARNING: Illegal access by org.codehaus.groovy.reflection.CachedClass
(file:/groovy-2.4.10/lib/groovy-2.4.10.jar) to method
java.lang.Object.registerNatives() (permitted by --permit-illegal-access)
WARNING: Illegal access by org.codehaus.groovy.vmplugin.v7.Java7$1
(file:/groovy-2.4.10/lib/groovy-2.4.10.jar) to constructor
java.lang.invoke.MethodHandles$Lookup(java.lang.Class,int) (permitted by
--permit-illegal-access)
Hello world!

The user's script is innocent but maybe you are saying that if Groovy
spins code and that code calls back into the Groovy runtime to do its
dirty work then Groovy will be blamed?


the warnings you are seeing are from the groovy runtime trying to 
determine - in a pre JDK9 compatible way - if it is allowed to call the 
methods on Objecz. This happens in a first step by setAccessible on all 
methods using the array version, and if that fails by using the normal 
setAccessible on each method (which will contribute to much longer 
startup times). At no point our runtime or the program you wrote are 
trying to call finalize, clone or registerNatives.


The only one we are to be blamed for is 
MethodHandles$Lookup(java.lang.Class,int), for which I haven´t 
implemented the alternative yet


If you did other programs... like for example running a gradle build... 
you would see many many more such lines



On the question of performance then it may have an impact, it might not
as it depends on how much illegal access is going on. We see some
libraries doing some hacking with setAccessible(true) at startup and
then re-using the AccessibleObject over and over, they will hardly
notice anything.


So it will not be on a invocation base with some kind of check if that 
has been locked already? Then I misunderstood



In the case of Groovy then running with
JDK_JAVA_OPTIONS="-Dsun.reflect.debugModuleAccessChecks=true" reveals
several places where  InaccessibleObjectException is being thrown and
silently ignored, e.g:


silently and purposely ignored.


java.lang.reflect.InaccessibleObjectException: Unable to make private
java.lang.invoke.MethodHandles$Lookup(java.lang.Class,int) accessible:
module java.base does not "opens java.lang.invoke" to unnamed module
@43bd930a
 at
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:337)

 at
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:281)

 at
java.base/java.lang.reflect.Constructor.checkCanSetAccessible(Constructor.java:192)

 at
java.base/java.lang.reflect.Constructor.setAccessible(Constructor.java:185)
 at org.codehaus.groovy.vmplugin.v7.Java7$1.run(Java7.java:53)
 at java.base/java.security.AccessController.doPrivileged(Native
Method)
 at org.codehaus.groovy.vmplugin.v7.Java7.(Java7.java:50)
 at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
 at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

 at
java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

 at
java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:488)
 at java.base/java.lang.Class.newInstance(Class.java:558)
 at
org.codehaus.groovy.vmplugin.VMPluginFactory.createPlugin(VMPluginFactory.java:59)

 at
org.codehaus.groovy.vmplugin.VMPluginFactory.(VMPluginFactory.java:40)

 at
org.codehaus.groovy.runtime.metaclass.MetaClassRegistryImpl.(MetaClassRegistryImpl.java:102)

 at
org.codehaus.groovy.runtime.metaclass.MetaClassRegistryImpl.(MetaClassRegistryImpl.java:74)

 at groovy.lang.GroovySystem.(GroovySystem.java:36)
 at
org.codehaus.groovy.runtime.InvokerHelper.(InvokerHelper.java:66)
 at groovy.lang.GroovyObjectSupport.(GroovyObjectSupport.java:34)
 at groovy.lang.Binding.(Binding.java:35)
 at groovy.lang.GroovyShell.(GroovyShell.java:74)
 at groovy.ui.GroovyMain.processOnce(GroovyMain.java:651)
 at groovy.ui.GroovyMain.run(GroovyMain.java:384)
 at groovy.ui.GroovyMain.process(GroovyMain.java:370)
 at 

Re: Better tools for adjusting to strong encapsulation

[Jochen Theodorou]

On 21.03.2017 15:48, mark.reinh...@oracle.com wrote:
[...]

When an illegal reflective access operation succeeds due to the use of
the `--permit-illegal-access` option, or the use of an `--add-opens` or
`--add-exports` option, then a warning message of the following form is
written to the error stream:

 WARNING: Illegal access by $PERPETRATOR to $VICTIM (permitted by $OPTION)

where:

   - $PERPETRATOR is the fully-qualified name of the type containing
 the code that invoked the reflective operation in question plus
 the code source (i.e., JAR-file path), if available,

   - $VICTIM is a string that describes the member being accessed,
 including the fully-qualified name of the enclosing type, and

   - $OPTION is the name of the command-line option that enabled this
 access, when that can be determined, or the first one of those
 options if more than one option had that effect.

The run-time system attempts to suppress duplicate warnings for the same
$PERPETRATOR and $VICTIM, but it's not always practical to do so.


that means we will get a lot of users asking us, why our runtime does 
this, even though it is the "fault" of the code using the runtime and 
not really ours. Because "type that invoked the reflective operation" is 
very likely again something caller sensitive, right? Well, that will 
point to our runtime, not to the real perpetrator. I think there is an 
annotation that could be used to exclude our methods from getting the 
caller information, but if memory serves me right this is no public api.


What will be the performance penalty of this?

bye Jochen

Re: Alternatives to automatic modules as a concept

[Jochen Theodorou]

On 20.03.2017 13:22, Ceki Gülcü wrote:
[...]

As for jigsaw module declarations, I expect it to be trivial assuming
building under Java 9 but targeting Java 6.


Is Java 6 still supported through a Java 9 build? I thought Java 7 will 
be the minimum.


bye Jochen

Re: RFR: 8177086: java.lang.reflect.AccessibleObject::canAccess should share access cache with internal method ::checkAccess

[Jochen Theodorou]

frankly I would migrate more from the normal reflection API to the 
MethodHandles API instead of extending the Reflection API further, but I 
see the use case and reasoning here. thanks.


bye Jochen

On 20.03.2017 10:04, Alan Bateman wrote:

On 19/03/2017 22:47, Jochen Theodorou wrote:



not wanting to hijack the thread, but why is there no canAccess method
that takes a class argument to check if that class can access? Why
always depending so much on caller sensitive methods?

This method is intended to be used in conjunction with
Constructor.newInstance, Method.invoke, Field.get, ...  The idiom in
Peter's mail combines this with trySetAccessible and makes it easy to
gracefully handle cases where access is not allowed.

JEP 274 added Lookup.accessClass and may be closer to what you need.

-Alan

Re: Instrumentation.redefineModule -- extraPackages argument?

[Jochen Theodorou]

On 11.03.2017 10:51, Alan Bateman wrote:

On 10/03/2017 16:12, Michael Rasmussen wrote:


:

It might not be the most common scenario, but creating new packages
during
development is definitely not uncommon, and from my experience with
JRebel
and our customers, I can also say that we do see these kind of changes.
So, if the JVM can automatically do it for unnamed modules, why is
this not
a supported redefinition for (named) modules?


If a module is in development or is being significantly refactored to
add/move code to non-exported packages then it really needs to be
re-loaded (in this design then this means re-instantiating a new module
layer, not specific modules).


reloading means the classes will be loaded into a new classloader? Don´t 
you think that is overkill for just adding a class in a not yet existing 
package - something that was very easy before? I guess you could always 
load the new class in the new package into dynamically created layer. 
But frankly, that is a new class loader, that requires you setting 
relations between layers and all that. That will backfire. Does JDK9 
even have no notion of removing a layer and garbage collection on it?


bye Jochen

Re: A broken setAccessible use case: enhanced diagnostics

[Jochen Theodorou]

On 26.01.2017 13:34, Russell Gold wrote:

The Glassfish Primitive Function Library (PFL) contains an
interesting use case for setAccessible: an enhanced object-to-string
capability. Now, in an ideal world, this shouldn’t be needed, as each
object provides an adequate toString() method; however, it is not
uncommon while trying to diagnose a problem, to need to see the
actual state of objects, as represented by their fields, as the
toString() function often doesn’t show everything. This is especially
important to diagnose problems in production, where one could ask a
customer to enable a debug flag. Until now, this was restricted only
by the security manager. What can we do to keep such a capability in
JDK9?


you have to declare all the module and the packages you want to access 
as open. On the command line this is done by using --add-opens. For 
example --add-opens java.base to open all classes in the java.base 
module. At runtime you have to use the Module and Layer APIs to realize 
this...


bye Jochen

Re: MethodHandle performance

[Jochen Theodorou]

On 12.01.2017 15:23, Stephen Colebourne wrote:
[...]

The real problem however is performance. In my tests, I am seeing a
MethodHandle approach being 2 to 3 times slower than a reflection
approach for identical functionality, which is quite a significant
degradation. (using Java 8 b122)


just asking to be clear about the problem are you talking about the 
execution time or does it include the creation of MethodTypes? Because 
creating those MethodType objects tends to be heavy


bye Jochen

Re: Suggestion: allow accessible reflection on protected methods of exported types.

[Jochen Theodorou]

On 10.01.2017 15:40, Andrew Dinn wrote:
[...]

That's possible because an agent can use its Instrumentation instance to
'export open' the MethodHandles package to a module of its own making
and then create the Lookup from within that module.


but that won't work for me in Groovy as a library. The other modules are 
already alive and there are instance from classes of those I need to 
work with. Even if I created a new Layer depending on bootstrap and add 
there manually all of the modules and add "opens" to them I will still 
get into trouble I assume, because I strongly expect those classes not 
be identical with the original classes. This works then only if I add my 
own module loader, which replaces the one used by jigsaw. Which is again 
highly intrusive. Imagine for example me adding something like that to 
jboss...


But... hmm... if you create a Lookup from within that module, why do you 
still need to add export open? The Lookup then has full rights even 
without "export open" I would have expected.



Obviously this is not a 'legitimate' way to create a Lookup -- hijacking
an internal API is never safe (irrespective of how unlikely it is that
the API might change or disappear). John Rose has raised an issue to
look into providing a /legitimate/ way of making Lookups available to
interested 3rd parties:

  https://bugs.openjdk.java.net/browse/JDK-8162494

If you have any input to provide as to how this might be done without
(as John says) 'giving away the store' I am sure it would be listened to
with interest.


Since the current mechanism with the security manager is deemed to be 
not good enough anymore I actually fail to suggest something, since that 
is exactly something I would expect the SecurityManager to handle.


bye Jochen

Re: Suggestion: allow accessible reflection on protected methods of exported types.

[Jochen Theodorou]

On 10.01.2017 12:28, Uwe Schindler wrote:

Hi,

just coming to the example you brought, because I exercised this a week ago:


[...] For example... can you
call Object#clone on Object using MethodHandles? I actually did not try
this, but since the method is protected and in a module I would assume
no. You had been able to do so by reflection. Let us not discuss about
if that makes sense or not. It just shows for me that (ii) is simply not
there.


Hi, it is impossible to call Object#clone with MethodHandles. Not even in the same module 
unless you unreflect an accessible reflective method. I tried something similar last week 
to fix the Cglib issues with invoking protected ClassLoader#defineClass. The idea was 
"the standard MethodHandle" trick:

Create an abstract subclass of ClassLoader - just to have a static initialization block. 
In this static block you get a (private) Lookup object to lookup the protected 
defineClass method. Which works phantastically! As it is virtual, the MethodHandle should 
now allow to call the method with any ClassLoader as first parameter (the receiver of 
virtual method). This works for all methods, but not protected ones. If you read the 
Javadocs it is in the specs. For protected methods, the methodhandle is 
"modified" and the receiver argument is casted to the Lookup's class. And 
because of this you get a ClassCastException if you don't call with a receiver that is 
assignable to the current Lookup's class.

Too stupid, the designers of MethodHandles were way too intelligent :-)

See my investigations:
https://github.com/cglib/cglib/issues/93#issuecomment-269255311


I guess the trick is that I am using reflection to get a Lookup object 
that has full priviledges... a trick that does no longer work in current 
jigsaw



bye Jochen

Re: Suggestion: allow accessible reflection on protected methods of exported types.

[Jochen Theodorou]

On 09.01.2017 12:47, Andrew Dinn wrote:
[...]

If you carefully review discussion on the subject of this (slightly
false) dichotomy between static and dynamic capabilities I think you
will find that there has been a lot of explicit discussion. In my view,
your summary of the status quo is at best out of date and at worst
misrepresents the goals of the Jigsaw project and the wider goals of the
OpenJDK project as a whole.

There is indeed a desire to limit opportunities for programs to rely on
reflection.


The problem is simply with old reflective code. You tend to have - and I 
have seen that in many applications - a kind of central part doing 
invocation for you. You could call it a reflective runtime. For this 
kind of code doing rights via @CallerSensitive tends to be unusable, 
because you tend to have code that wraps the call itself. And once you 
have that, it is the rights of your runtime, not that of the calling 
code. It was simply not "normal" to do, what you can now do with 
MethodHandles, which is returning something, which will then be invoked 
instead of doing the invocation right away. And considering cases where 
you for example need the post processing of exceptions and such I don´t 
really see how you could have done that with reflection anyway.


Now having these "central invokers" means you have to work completely 
out of context as far @CallerSensitive is considered. And that means you 
are forced to use setAccessible, if you need to handle more than just 
public class members (or public class containers for that matter). And 
this kind of code is now practically crippled.


And as you wrote, that is exactly what was intended to be done.

But that means, that language runtimes tend to have a problem now if 
they depend on such central invoker patterns.


I won´t deny that maybe in some structures things got a bit out of hand. 
But fixing them after years of working without breaking changes is near 
impossible.


That is because I cannot simply change exported API to reach through 
Lookup objects, since they would be a breaking API change. Plus the 
Lookup objects can only do what their context allows them to do, they do 
not automatically give me the right to invoke methods on other objects I 
normally have no access rights for in that context. I would need the 
Lookup object for the goal, which might not be obtainable. And again, 
that is exactly what was intended to be broken.


example... Let us say you work with a GroovyIntereptable:

class A implements GroovyIntercepable {
  def invokeMethod(String name, args) {
   ...
  }
}

class B extends A {
  private someMethod() {}
}

for every method call on an instance of A or B (for example 
b.callSomemethod()) the invokeMethod method is called. I was now 
perfectly able to invoke someMethod on instances of B in the 
invokeMethod method of A by simply doing this.someMethod(). I assume the 
code will still work if B is in the same module as A, but if not, then 
this code will no longer work using central invoker patterns. Of course 
I could now claim that B may expose a helper method of some kind, which 
is automagically called and delegates to someMethod, or that I can 
replace the usage by MethodHandles to some extend. But if class B is 
written in Java, then I fail with the first approach and if the actual 
handling is not done in A, but in a delegate of A, I fail with the 
MethodHandles approach behind the scenes as well. Of course I could have 
the Lookup object in this case. But point being, the API does not allow 
for it without breaking changes.


Then how else can I add such a helper method for my runtime? Agents, as 
Remi said? Can agents add methods to an already loaded class? I am 
unaware of this being possible now. Which means I have to start an agent 
on VM startup, which means a more complicated command line for the user, 
but it means first and for all it counters the usage of Groovy as 
non-intrusive library. Right now, if I do not use anything from Groovy, 
the Groovy runtime won´t be involved. With having to have an agent to 
transform classes on load, this is changed.


In other words, we are destroying one feature to rescue another. And I 
do not see a good solution here, I doubt there can really exist one, 
because it is the very goal of jigsaw to "limit opportunities for 
programs to rely on reflection". And just to make this 100% clear... 
even the invokedynamic based version of Groovy has to use reflection to 
some degree to get everything running. Basically all those things that 
jigsaw forbids now.



That desire is linked to the motivation for implementing
Jigsaw. Reflection as it was provided in JDK8- does not really fit well
with /the/ main goal of modularization i.e. ensuring that module
internals -- in particular, JDK runtime internals -- can be guaranteed
to remain private to client code. The concern is wider than simply
meeting this Jigsaw requirement. That guarantee is critical to ensuring
that module 

Re: Using java.awt.Toolkit.getDefaultToolkit().getScreenSize() reflectively causes InaccessibleObjectException

[Jochen Theodorou]

On 05.01.2017 21:07, Phil Race wrote:

Sort of .. it depends whether you could use getDeclaredMethod instead.

One question I have is, how would you update the code if you
were given the instance "foo", and need to programmatically
work out what is the correct super-type of Foo that exports
the interface method "bar" ?

i.e you don't have the string "java.awt.Toolkit" handy and
need to infer it from the instance class.


staying in old java style I guess something like this:

Method getMethod(Class c, String name, Class... types) {
  if (c==null) return null
  try {
return c.getMethod(name, types)
  } catch (Exception e) {
// ignoring exceptions is fun!
  }
  for (Class ci : c.getInterfaces()) {
Method ret = getMethod(ci, name, types)
if (ret!=null) return ret
  }
  return getMethod(c.getSuperClass(), name, types)
}

If the this getMethod returns a method then you should have something 
you can invoke on. Just searching the super classes is not enough, the 
only exported type might be an interfaces, so you have to go through 
those too. I leave the version with lambdas, flat-mapped streams, 
Optionals and useless generics/semicolons to somebody else ;)


I think a version using MethodHandles would be similar, since I think 
the game is the same if you use Lookup#findVirtual instead of 
Class#getMethod... only that in the end you could of course just return 
a MethodHandle that throws one (of the countless) exceptions this code 
may have produced...


I'd love to see a better solution though..

bye Jochen

Re: Suggestion: allow accessible reflection on protected methods of exported types.

[Jochen Theodorou]

On 03.01.2017 18:04, David M. Lloyd wrote:
[...]

Here's an important point about adoption (these are my own observations,
not the EG's or my employer's).  One of the effects of the way Java is
produced and its historical entrenchment is that no matter what happens,
adoption is more or less inevitable; every prediction of Java's death
has proven to be false (which of course doesn't stop pundits from doing
so anyway).


For sure people will be using Java 9 at some point. If they will 
completely work around the module system or actually use it properly is 
an entirely different matter. At no point did I intend to predict the 
death of Java.



Even if a future Java was terrible by many objective
measures, its adoption will probably be eventually inexorable.  It would
take something highly extraordinary to change these basic truths.  This
is just the nature of the Java ecosystem today.  Lack of adoption is a
consequence that essentially nobody is worried about, and is really not
a deterrent to any given proposed change.


But there is a relative lack of adoption. For example I know only very 
few libraries that actually use nio2 instead of "normal" java.io. But 
just using the old and ignoring the new and otherwise updating to the 
newer version of the JVM would still count as adoption... even though it 
not really is. Java9 now is one of the releases that do not allow you to 
simply ignore the new and continue with the old.



But this just means that it is all the more critical that everyone that
uses Java in their day to day work to take some degree of responsibility
to ensure that every language and JDK specification change is reviewed
for quality.


Quality can mean a lot of things. Any computer virus can contain high 
quality code, still you do not want any. If the intention is wrong, then 
quality control will do nothing.



The JCP EC consists of many parties; it's my personal
belief and hope that there will always be EC members (hopefully the
majority of them in fact) that will listen to the practical concerns of
the community.


For me jigsaw is an incorporation of not being practical. I am not the 
community, so just because nobody is listening to me doesn´t mean the 
community is ignored... but frankly my impression is the community does 
not know - yet. There was an outreach for 2+ years, yes, but if I 
consider the situation 1 year for example, there had been so many open 
questions to the module system, especially with regards to dynamic 
modules and layers, that it was impractical for me to even start working 
on it. And the situation changed again massively with 
#AwkwardStrongEncapsulation.


For me 100% of my work projects and hobby projects are impacted by 
java9. And that is not because of Groovy alone. Jigsaw made me 
seriously consider leaving the JVM for the first time in almost 20 years.


bye Jochen

Re: Suggestion: allow accessible reflection on protected methods of exported types.

[Jochen Theodorou]

On 29.12.2016 18:24, Rafael Winterhalter wrote:

Hei Jochen, thank you for your feedback. I must have missed your
posting, I regularly fail to follow up on the things discussed on the
mailing list and its not easy to seatch for topics, sorry for that.


don´t worry, my complaints have not been ignored, but did not change 
anything either.



In Byte Buddy, I created an option to use subclassing by creating
dedicated class loaders for proxy classes. There is however a row of
disadvantages:

1. Class loaders are not cheap objects, they require some memory and the
performance cost of creating dedicated class loaders is significant. I
have benchmarked this, as well and it costs you about three times the
time to create such "wrapper class loaders" rather than injecting.


agreed


2. It is close to impossible to define a good life-cycle for classes
loaded by such wrapper class loaders. You need to keep a reference to
the proxy class to prohibit its collection which in turn keeps a
reference to the original class loader making it uncollectable. Using
soft or weak references is less then ideal, too and the only option
would be to use ephemerons which are not supported by Java. With
injection, it is as easy as looking up a previously created class from
the proxied class's class loader which dies and lives with this class
loader.


A ClassValue is no ephemeron, but letting it keep a SoftReference to the 
proxy class may work. It means there will be situations in which the 
class will be recreated, but thanks to the SoftReference both classes 
and loaders should still be garbage collectable.



3. Java packages define their identity by name and class loader at
runtime. Many developers use anonymous classes or nested classes which
are package-private and cannot be subclassed by creating a new class
loader but which must live in the same class loader.


package-private is another story... I would be happy enough to have a 
generic solution for protected.


and 4.  if the  loader is defined by a module you will have fun with 
creating layers and export/read edges



Those are the main issues that lead us to strongly prefer injection,
especially in a testing library like Mockito and there are some more
complications that we would like to avoid.


the reply will most probably be about asking why these classes are not 
made public. They can be exported to only the framework module and all that.



I am however especially concerned about upgrading. Esepcially with
mature and wide-spread libraries like Mockito or Hibernate, even minor
changes can cause really big, hard-to-debug issues for our users.


been there and I agree. But it is unlikely that something like the 
implementation of strong encapsulation will be done without trouble for 
bigger projects. Will this lower adoption of Java9? Sure.



Switching from injection to subclassing would be much more than a minor
change. The JVM developers surely know this, otherwise they would not
keep classes like Stack around or fix quirks in classes like ByteBuffer.
Too much code was written on top of these classes and their expected
inner working. If we were however forced to change our APIs that
drastically, I think the real consequences would be much more drastic
than the mentioned changes. At the moment, as a library author, I do
however feel like there are few options to avoid this.


On the other hand... if you think it has to be done at some point, then 
why delay it? Of course the question of if it has to be done is 
questionable in itself, especially since the slim advantages of the 
module system turn into the negative, if weighted against "other 
solutions", the work you will have with it and the limitations it imposes.


bye Jochen

Re: Suggestion: allow accessible reflection on protected methods of exported types.

[Jochen Theodorou]

On 29.12.2016 17:24, Rafael Winterhalter wrote:

Hello everybody,

I did another round of testing a recent build of Java 9 with some
applications and framework that I work with or maintain. I ran into an
issue that I consider rather severe and I want to make a suggestion on how
to potentially solve this problem.

Some libraries entertain utilities that access protected methods of
exported types to enhance their functionality without requiring a user to
provide a specific subclass of such an instance but to allow them to use
“their“ implementation. To access these protected methods, reflection is
used in combination with AccessibleObject::setAccessible.

For example, this approach is used by code generation libraries to invoke
the ClassLoader::defineClass method which allows the injection of proxy
types into the class loader of the proxied class. With the new constraints
enforced by Jigsaw, attempting to make ClassLoader::defineClass accessible
fails with:


while I am on your side and while I did bring this up myself in the 
past, let me play the devil´s advocate here and ask: Why can´t this be 
solved by subclassing? For example you can create a new class loader 
that makes defineClass public and use that class loader as a child of 
the class loader, which defines the class you want to proxy. Basically 
this targets at the question of if these libraries are using the API 
wrong, or if the API is wrong and if an alternative has to be created.. 
which does already exist with Unsafe.


by Jochen

Re: Java 9 build 148 causes trouble in Apache Lucene/Solr/Elasticsearch

[Jochen Theodorou]

Hi,

I strongly hope Paul and Cedric will be able to start the release 
process next week, if not we will have to do it the old way I think.


what would help us a lot would be you testing the GROOVY_2_4_X branch 
with your build system to see if it really does solve your problem. Even 
if it is only locally on your computer


bye Jochen

On 16.12.2016 10:58, Uwe Schindler wrote:

Hi Jochen,

thank you for the information! Is there any plan about a release? I also found 
no JIRA issue about this issue to link it against our JIRA: 
https://issues.apache.org/jira/browse/LUCENE-7596

The problem makes our build system unusable, so it would be very important to 
have a fix quite soon! As our Ant/Ivy-based build relies on Maven Central, it 
would be good to have the bugfix release available there, which requires a 
release. I think the same applies for Gradle users (Elasticsearch).

As a temporary workaround we might be able to use the Apache Snapshot 
repository, but this is not allowed if we do a release of Lucene.

Uwe

-
Uwe Schindler
uschind...@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/


-Original Message-
From: Jochen Theodorou [mailto:blackd...@gmx.org]
Sent: Saturday, December 10, 2016 9:23 AM
To: Uwe Schindler <uschind...@apache.org>; jigsaw-dev@openjdk.java.net;
Core-Libs-Dev <core-libs-...@openjdk.java.net>
Subject: Re: Java 9 build 148 causes trouble in Apache
Lucene/Solr/Elasticsearch

On 09.12.2016 23:32, Uwe Schindler wrote:

Hi,

I updated our Jenkins server for the JDK 9 preview testing to use build 148.

Previously we had build 140 and build 147, which both worked without any
issues. But after the update the following stuff goes wrong:


(1) Unmapping of direct buffers no longer works, although this API was

marked as critical because there is no replacement up to now, so code can
unmap memory mapped files, which is one of the most important things
Apache Lucene needs to use to access huge random access files while
reading the index. Without memory mapping, the slowdown for Lucene
users will be huge


This is caused by the recent Jigsaw changes, published in build 148.

Unfortunately we did not test the Jigsaw builds, so we would have noticed
that earlier. Basically the following peace of code fails now (with or without
doPrivileged and with/without security manager):


   final Class directBufferClass =

Class.forName("java.nio.DirectByteBuffer");


   final Method m = directBufferClass.getMethod("cleaner");
   m.setAccessible(true);
   MethodHandle directBufferCleanerMethod = lookup.unreflect(m);
   Class cleanerClass =

directBufferCleanerMethod.type().returnType();

   // build method handle for unmapping, full code is here:

https://goo.gl/TfQWl6

I guess that is the effect of #AwkwardStrongEncapsulation. I would
advise doing regular checks against the jigsaw builds to know about such
problems in the future earlier... but seeing your code break without an
obvious good solution sure is stressful. I feel with you.

[...]

(2) A second thing  we noticed is that Groovy no longer works and dies with

strange error messages.

That is because versions including Groovy 2.4.7 are using
setAccessible(AccessibleObject[] array, true), and the array will also
include private methods or fields. This worked till
#AwkwardStrongEncapsulation because will then a class was either
exported and its method can all be made accessible or not. For example
on GAE or earlier versions of the module system. Now an exported class
may break this, since its private methods can no longer be made
accessible using setAccessible.

A fix for this is already committed, we are only waiting for release of
Groovy 2.4.8. Of course even with the fix Groovy code can possibly
break... for example if you did the direct buffer access in Groovy.

Btw, do not hesitate to ask about such problems on groovy-user, please.

bye Jochen

Re: Question ad #AwkwardStrongEncapsulation (Re: Moving the changes in jake to jdk9/dev

[Jochen Theodorou]

On 13.12.2016 23:17, Peter Levart wrote:
[...]

You might have access to a protected method, but you can not delegate
that access to a 3rd party unless you make the Method object
.setAccessible(true) and pass it to the 3rd party as a capability. (I
recommend using MethodHandle(s) for such delegation of rights instead of
reflection though).


that is unlikely to happen


But let me explain why .setAccessible(true) can't be allowed for
protected members in general.

Jigsaw establishes strong encapsulation. What that means is that even
without a SecurityManager present, code should not be allowed to gain
access to a member beyond what is allowed by accessibility rules of Java
language unless that member is in a class in an open package or such
access is willingly delegated to code by some other code.


I am aware of this. For me it is more a question of how far strong 
encapsulation is supposed to go and you can understand strong 
encapsulation in a module system in different ways as well.


I don't intend to get access to hidden API after all... just exported.

[...]

You can't perform the access check for a protected
instance member without knowing the 'targetClass' (the runtime class of
the target instance).


sure, I already commented on this part with: why do access checks for 
this at all? Your answer is because of strong encapsulation and my 
comment for that is that you guys are maybe overdoing it a bit. Which 
leaves us with: you want this change and I am unhappy with it. I don't 
see a technical reason for the limit.


bye Jochen

Re: Question ad #AwkwardStrongEncapsulation (Re: Moving the changes in jake to jdk9/dev

[Jochen Theodorou]

On 12.12.2016 20:56, Alex Buckley wrote:
[...]

The ability of protected members to be accessed from outside their
package means they are essentially public members for the purposes of
inheritance and reflection. So, setAccessible should work for protected
members of exported packages. I know what you mean about the receiver
object being of the correct class, but that's outside the capability of
setAccessible to check, so I don't believe it is checked.


why does it have to be checked? why not just allow it? I mean that is 
why I use setAccessible in the first place. I have much less use of 
making something accessible for which I already have access rights


bye Jochen

Re: AccessbileObject setAccessible array version vs non-array version

[Jochen Theodorou]

On 10.12.2016 13:08, Alan Bateman wrote:

On 10/12/2016 09:13, Jochen Theodorou wrote:


Hi all,

motivated by the recent "Java 9 build 148 causes trouble in Apache
Lucene/Solr/Elasticsearch" thead, I thought I ask... there is
AcccessibleObject#setAccessible(boolean), which will ask the
SecurityManager for permissions and then make itself accessible
according to the boolean flag. the array version takes an array of
AccessibleObjects, asks the security manager *once* and then makes all
of them accessible. So if you are in need to make a lot of objects
accessible the array version is superior in performance.

Now with jigsaw it is no longer a all or nothing for the class, now
single methods or fields may no longer be made accessible, even
without security manager. That means that even without a security
manager set using the array version on File for example will fail with
an exception (unless the module is opened from the command line, but
we should leave that out for now)

My question now basically is the following... why is this method not
made deprecated? It is kind of useless now, even misleading I would say.

I'm not sure that I understand your mail. The permission check when
running with a security manager has not changed. If you use the array
version then there is one permission check.

Maybe you mean that the array version will fail when the array contains
at least one element where the access check cannot be suppressed? That
is possible of course. You mentioned File and maybe you mean you the
array has a mix of public methods and non-public members and fields?


yes. You will not need to set them to accessible for public members 
after all


bye Jochen

Re: Java 9 build 148 causes trouble in Apache Lucene/Solr/Elasticsearch

[Jochen Theodorou]

On 09.12.2016 23:32, Uwe Schindler wrote:

Hi,

I updated our Jenkins server for the JDK 9 preview testing to use build 148. 
Previously we had build 140 and build 147, which both worked without any 
issues. But after the update the following stuff goes wrong:

(1) Unmapping of direct buffers no longer works, although this API was marked 
as critical because there is no replacement up to now, so code can unmap memory 
mapped files, which is one of the most important things Apache Lucene needs to 
use to access huge random access files while reading the index. Without memory 
mapping, the slowdown for Lucene users will be huge

This is caused by the recent Jigsaw changes, published in build 148. 
Unfortunately we did not test the Jigsaw builds, so we would have noticed that 
earlier. Basically the following peace of code fails now (with or without 
doPrivileged and with/without security manager):

   final Class directBufferClass = 
Class.forName("java.nio.DirectByteBuffer");

   final Method m = directBufferClass.getMethod("cleaner");
   m.setAccessible(true);
   MethodHandle directBufferCleanerMethod = lookup.unreflect(m);
   Class cleanerClass = directBufferCleanerMethod.type().returnType();
   // build method handle for unmapping, full code is here: 
https://goo.gl/TfQWl6


I guess that is the effect of #AwkwardStrongEncapsulation. I would 
advise doing regular checks against the jigsaw builds to know about such 
problems in the future earlier... but seeing your code break without an 
obvious good solution sure is stressful. I feel with you.


[...]

(2) A second thing  we noticed is that Groovy no longer works and dies with 
strange error messages.


That is because versions including Groovy 2.4.7 are using
setAccessible(AccessibleObject[] array, true), and the array will also 
include private methods or fields. This worked till 
#AwkwardStrongEncapsulation because will then a class was either 
exported and its method can all be made accessible or not. For example 
on GAE or earlier versions of the module system. Now an exported class 
may break this, since its private methods can no longer be made 
accessible using setAccessible.


A fix for this is already committed, we are only waiting for release of 
Groovy 2.4.8. Of course even with the fix Groovy code can possibly 
break... for example if you did the direct buffer access in Groovy.


Btw, do not hesitate to ask about such problems on groovy-user, please.

bye Jochen

Re: Services and Bindings - expected usage scenarios

[Jochen Theodorou]

On 30.11.2016 17:31, mark.reinh...@oracle.com wrote:
[...]

The SE Platform has ServiceLoader rather than a built-in DI mechanism
because the former is more primitive and imposes less overhead, while
the latter can be built on top of the former.  (I don't really know,
nor care, whether java.util.ServiceLoader is officially an instance
of the Service Locator pattern.)


while some form of DI can be built on top of ServiceLoader, it usually 
is not. And that is because it is really annoying to do so. Imagine for 
example in Spring what you have to do to get DI configured only in Java 
running on top of the ServiceLoader architecture. Just imagine how many 
additional ClassLoaders you will need and now also Layers. And then 
imagine how you are going to test all this... And you really, really 
cannot expect that this all will be done without the user noticing 
additional quirks.


I see not a single thing being better for DI with non-static descriptors 
in jigsaw. Only many more complications


bye Jochen

Re: #CyclicDependences with ServiceLoader

[Jochen Theodorou]

On 30.11.2016 10:16, Alan Bateman wrote:

On 30/11/2016 08:53, Jochen Theodorou wrote:


Hi,

for my understanding I would like to ask the following:

Given:
* Module A: uses service S1 and provides service S2
* Module B: uses service S2 and provides service S1

In my understanding this forms a cyclic dependency, but since this is
not directly on the API level I am wondering if this is really one
"per definition as to be seen by jigsaw"?

You need to expand the example a bit to say where S1 and S2 are, it's
otherwise impossible to say if there is a cycle in the dependency graph.

For example, there is no cycles here:

module S {
exports p;
}

module A {
requires S;
uses p.S1;
provides p.S2 with ...
}

module B {
requires S;
uses p.S2;
provides p.S1 with ...
}


if that is no cycle, then you answered my question already. There is no 
dependency between A and B in the sense that A uses a class from B and B 
uses a class from A - not directly at least. But there is one through 
the ServiceLoader, but it is still not cyclic for jigsaw, which looks 
for that only at the "direct" API usage


And it is easy to see that if we drop S and put the service interface 
for S1 in A and the one for S2 in B that we then get a cycle just 
because of "requires"


thanks,
Jochen

#CyclicDependences with ServiceLoader

[Jochen Theodorou]

Hi,

for my understanding I would like to ask the following:

Given:
* Module A: uses service S1 and provides service S2
* Module B: uses service S2 and provides service S1

In my understanding this forms a cyclic dependency, but since this is 
not directly on the API level I am wondering if this is really one "per 
definition as to be seen by jigsaw"?


bye Jochen

Re: Services and Bindings - expected usage scenarios

[Jochen Theodorou]

I am not of the expert group, but I would like to give a partial answer.

Let us think of a kind of plugin architecture. There is a public 
interface from your framework and there are other modules that are going 
to provide plugins by implementing that interface. Now assume the 
framework will also instantiate the plugin. This is a problem, because 
normally a plugin is an implementation detail and its only public aspect 
is supposed to be the interface it implements. But to create an instance 
of it, you need to execute a constructor, which would then originate in 
private API.


Thus the ServiceLoader API and the module logic and descriptor got 
extended to support this.


bye Jochen

On 29.11.2016 14:13, Pisarev, Vitaliy wrote:


What about the intent behind how are developers supposed to use this tool?

For example, when Optional was introduced in Java 8, it was specified that it 
is intended to be used as a methods return type. And people should not
use it as class member or method argument, even though the compiler will not 
prevent them to.

Here is a language feature that comes with guidelines as to how it is intended 
to be used.

Can you tell me how the expert group envisions usage of the ServiceLoader in 
Java 9, with reference to the original question (and any other thing you can 
add with regard to this).

-Original Message-
From: Alan Bateman [mailto:alan.bate...@oracle.com]
Sent: יום ג 29 נובמבר 2016 14:06
To: Pisarev, Vitaliy ; jigsaw-dev@openjdk.java.net
Subject: Re: Services and Bindings - expected usage scenarios

On 29/11/2016 11:35, Pisarev, Vitaliy wrote:


Another best practices question.

I am aware that the ServiceLoader API is not new to Java. What's new
in Java 9 is that is has been put forward to the front of the stage and it is 
now very easy for a service provider to register a service.

The thing is that up until now, the ServiceLoader was a relatively low level 
component, primarily used by infrastructure components that loaded plugins that 
implemented various java specs.
It is no surprise than that the ServiceLoader is something that very few ever 
heard of.

Is it the intent of project Jigsaw that the service loader becomes a central 
and ubiquitous mechanism in the day to day work of plain old java developers?
As far as I am concerned, it is a very useful tool even when I do not
have a plugin architecture and just want a module to expose a certain service 
via an interface without having to jump throw hooks in order to provide the 
consumer a way to instantiate it..

Which leads me to the next question: is the returned instance a singleton or a 
prototype?

Services are a first class part of the module system but are a somewhat 
advanced topic. The ServiceLoader API is part of the story but arguably just a 
small aspect when compared to the architectural and design issues involved in 
moving to services, loose coupling, and improved SoC.

Provider factories can implement singletons where needed, the details are in 
the javadoc [1].

-Alan

[1] http://download.java.net/java/jigsaw/docs/api/index.html

Re: Can't call certain accessible methods using reflection

[Jochen Theodorou]

On 28.11.2016 12:23, Peter Levart wrote:
[...]

// Module m1:

module m1 {
exports pkg1;
}

package internal;
public class InternalImpl {
public void m() {
System.out.println("m()");
}
}

package pkg1;
public class Public extends internal.InternalImpl {
}


is it legal for an exported class to "expose" an internal class in the 
class signature? I would have assumed this will fail compilation





// Module m2:

module m2 {
requires m1;
}

package pkg2;
import pkg1.Public;
import java.lang.reflect.Method;
public class Main {
public static void main(String[] args) throws Exception
Public p = new Public();
// using bytecode
p.m();
// using reflection
Method m = Public.class.getMethod("m");
m.invoke(p);
// IllegalAccessException: class pkg2.Main (in module m2) cannot
access class internal.InternalImpl (in module m1) because module m1 does
not export internal to module m2
}
}


most likely p.m() will do invokevirtual P#m(), while 
Public.class.getMethod("m") will return a Method with the declaring 
class being internal.InternalImpl.


bye Jochen

Re: Few questions

[Jochen Theodorou]

On 26.11.2016 18:29, Michał Zegan wrote:

well...

But jigsaw itself can become a foundation for such a higher level
framework build directly on top of jigsaw that adds such dynamic
capabilities?


if you want more than jigsaw offers out of the box, you have add runtime 
capabilities and you can do so to some extend. Like making a new layer 
and defining modules in their on your own, in which you then set the 
read edges and such.


bye Jochen

Re: Few questions

[Jochen Theodorou]

On 26.11.2016 12:53, Michał Zegan wrote:

I know that osgi sits on top of jigsaw, but osgi was created when jigsaw
did not exist, so it is itself a module system for java that does not
need jigsaw.
So then it would probably look like a module system on top of another
module system... seems weird to me.


That is because jigsaw does not try to be a replacement for OSGI. OSGI 
for example does normally not do anything for you at compile time. 
jigsaw on the other hand does not allow you using multiple versions.



What should I do when I would like to write a new application that would
be modular? Use jigsaw directly or not?


It depends then on your requirements

bye Jochen

Re: Few questions

[Jochen Theodorou]

On 25.11.2016 23:21, Michał Zegan wrote:

Well, I specifically mean setAccessible usage between modules.


If your code really requires setAccessible it is going to break with 
jigsaw. And there is no way to for example emulate that on java.base 
without using command line arguments intended only for the transition to 
"real" modules.



Another question that comes to mind after reading some things: what
about relation of jigsaw and osgi?


they are different beasts


Is jigsaw going to replace osgi?


No


Well, what I exactly mean is, I know you try to make jigsaw and osgi
work together.  I just mean, is jigsaw module system going to work in
such a way that you would be able to use it for modularizing
applications without osgi, assuming that either the app does not use
dynamic services, or it does, but there would be a service manager built
on top of jigsaw?


No. OSGI sits on top of jigsaw.


Specifically, what about multiple versions of the same module per jvm
and multiple packages with the same name per jvm (either private, or
exported non transitively)...


multiple versions of the same module can be done... kinda... Just not at 
compile-time. You will be required to handle all the layers and read 
edges yourself. At compile-time jigsaw does not allow multiple versions 
of the same module - not even two distinct modules that use the same 
packages (I forgot if that was really only for the exported packages, or 
if the non-exported count as well, or if them counting as well was/is a 
bug).


bye Jochen

Re: modules and tests

[Jochen Theodorou]

On 24.11.2016 16:41, Alan Bateman wrote:
[...]

to Alan and Sander:
setting command line arguments or using a build tool to fiddle them
for you is exactly what we do not want here! We want fidelity between
the compile time configuration and the runtime configuration. Having
to play with -Xpatch at runtime is conceptually exactly like setting
the classpath. I don't want to explain to the Java devs that we have
fidelity between compile-time and runtime on source code but not on
test code.


I hope in time that there will be support from the tools, plugins, test
runners ... so that regular developers don't need to be concerned with
this.


I don't understand why it is better if a tool does -Xpatch, than when I 
do it myself. Anything else will require several classes be loaded that 
will change how the modules work and influence the tests with their 
present and their dependencies


bye Jochen

Re: The split package problem

[Jochen Theodorou]

On 05.11.2016 13:56, Alan Bateman wrote:



On 05/11/2016 10:29, Jochen Theodorou wrote:

On 05.11.2016 08:39, Alan Bateman wrote:


The application module reads other modules rather than layers, in this
case it results in the application module reading C, B-1, D, and B-2
(assuming that the application modules `requires C` and `requires D` and
both of these modules `requires transitive B`). This will fail because
the application reads B-1 and B-2, both of which export the same
packages to the application module.


And if that is done at runtime ti still fails?

The exception will be thrown when you attempt to create the
configuration containing the application module. So yes, it's run time,
there is no equivalent at compile or link time because this is
dynamically created configurations + layers.


so it is possible at runtime.

Still I don´t get why it should fail at compile time. if the application 
requires C and D and they require each B...ahh, the misunderstanding is 
probably that B-1 and B-2 are two versions of the module B. Of course I 
would not compile my application with B-1 and B-2 being there at the 
same time. But I can compile using either of them as B. Since there is 
only one of them, there will be no compile time failure... unless 
application uses something from B-1, that does not exist in B-2 and 
something from B-2 that does not exist in B-1. But that would not work 
out at runtime either.


bye Jochen

Re: The split package problem

[Jochen Theodorou]

On 05.11.2016 08:39, Alan Bateman wrote:

On 04/11/2016 16:50, Jochen Theodorou wrote:


:


Attempting to do this with multiple configuration + layers isn't going
to help, it's just not safe, and you'll get the same exception.


One layer with B-1 and C and another with B-2 and D and my application
"reading" from those layers... is that something that cannot be done?
So far I had the impression I can

The application module reads other modules rather than layers, in this
case it results in the application module reading C, B-1, D, and B-2
(assuming that the application modules `requires C` and `requires D` and
both of these modules `requires transitive B`). This will fail because
the application reads B-1 and B-2, both of which export the same
packages to the application module.


And if that is done at runtime ti still fails?

bye Jochen

Re: The split package problem

[Jochen Theodorou]

On 04.11.2016 15:29, Alan Bateman wrote:

On 04/11/2016 13:22, Jochen Theodorou wrote:


:

well... assume you have an application and it requires the library A,
which transitively requires B-1. the application also requires library
C, which transitively requires B-2. B-1 and B-2 are not compatible.
library A and D leak instances of classes of B-1 and B-2 to the
application.


Assuming B-1 and B-2 export the same packages, A `requires transitive
B-1` (because a method in A's API returns a B type), C `requires
transitive B-2` (because a method in C's API returns a B type) then you
will get an exception when attempting to create the configuration. The
exception will tell you that the application module reads two modules
(B-2 and B-2) that export the same package to the application.


I can compile the application if there is only B-1 or B-2 available at 
that time, regardless of if that is actually the wrong version for A or 
C. jigsaw does after all not care about the versions. This would mean 
the application cannot be run with only B-1 or B-2 of course. And it 
means I cannot run the application normally either.



Attempting to do this with multiple configuration + layers isn't going
to help, it's just not safe, and you'll get the same exception.


One layer with B-1 and C and another with B-2 and D and my application 
"reading" from those layers... is that something that cannot be done? So 
far I had the impression I can



You can of course go off-piste and use the reflective API to have the
application read both B-1 and B-2, and if it its your own class loaders
then you can go crazy with split delegation, but that is not something
that you get out of the box.


you rarely get the classloader hell problem out of the box either. And 
configurations like the ones above are not all that rare for me... just 
so far that had been with classloaders only, now it would be with 
classloader and modules and layers


bye Jochen

Re: The split package problem

[Jochen Theodorou]

On 04.11.2016 11:50, Andrew Dinn wrote:
[...]

 - Modules stop you providing two versions of a package in the same
layer, a problem for classpath deployment which, as Remi noted, can
easily lead to you mixing classes from two different versions of a library.


but sometimes you have to do something similar to that. Usually you then 
start with classloader magic and have to find your way around 
"duplicated classes".


What is targeted with modules is that this is done by accident. And 
frankly, my experience was that those accidental cases have been 
resolved pretty fast.



 - You can indeed use a dynamic module loading system based on layers to
introduce versions of the same classes in different layers. However, the
unique package ownership requirements means that those layers cannot
partake in an ancestor relationship again with the result that duplicate
classes cannot then be conflated in the way Remi described.


sure, nobody really needs that on the classpath... except for a poor 
mans patching.



So, it seems from the above that layers are indeed a structured way to
avoid one of the major causes of 'classloader hell' precisely because
they detect and reject deployments which introduce these ambiguities in
resolution.


"classloader hell" goes far beyond the classpath. The problem is rarely 
caused by a classloader that loads two versions of the same library with 
different classes. But happens then more easily if your classloaders 
suddenly have to become forests. Frankly I do not see how modules will 
avoid that.



Are you suggesting that modules will not provide this specific benefit
or that some other problem will render it of no importance? If the
former can you explain why? If the latter can you explain what that
other problem is?


well... assume you have an application and it requires the library A, 
which transitively requires B-1. the application also requires library 
C, which transitively requires B-2. B-1 and B-2 are not compatible. 
library A and D leak instances of classes of B-1 and B-2 to the application.


Problem number 1, how to start this if application, A, D, B-1 and B-2 
are modules? You don't care that the module system does not allow for 
this, you have to run it in that configuration and you have to have a 
way around this. Which means you will have to load modules dynamically 
in different layers. At this point the benefit already become a burden 
and is nullified as benefit.


Problem number 2... the layers in which B-1 and B-2 reside in still have 
their own versions of classes, which are by name equal in B-1 and B-2. 
If application is actually using classes directly from B-1 or B-2 you 
get the classloader hell problem, just the same as you would without the 
module system beyond considering the classpath.


bye Jochen

Re: The split package problem

[Jochen Theodorou]

On 04.11.2016 12:36, Sander Mak wrote:



On 04 Nov 2016, at 12:06, Cédric Champeau  wrote:

It's a very big issue, typically all Gradle plugins written in Groovy would
break. Gradle itself would break. There are things to mitigate that, like
rewriting classes at load time, or an easier solution which is to have a
single Groovy module for all (and come back to the monolith era, sigh...).



Wouldn't a better solution be to provide a groovy-all module that 'requires 
transitive' the other actual groovy modules.


I agree in that we should have done this... but


(probably moving the split packages back together in a module)


that is exactly the problem. if you move them back, you get half of the 
dependencies along, which makes the modules itself kind of obsolete.


bye Jochen

Re: The split package problem

[Jochen Theodorou]

On 04.11.2016 10:33, Alan Bateman wrote:
[...]

This is all part of reliable configuration where you can prove
correctness by construction. Alex's "Under the Hood" session from
JavaOne 2016 [1] is a great resource for understanding the science.

-Alan

[1] http://openjdk.java.net/projects/jigsaw/talks/#j1-2016


I could now go into lengths as of why I do not believe in proven 
correctness by construction, but that is probably too offtopic here ;)


What I see mostly is that all the problems you have now on a per class 
level, you later have on a per module level again... for example needing 
two versions of the same library being active at the same time... and 
the movement away from the static module descriptor to a dynamic module 
loading system with layers and all kinds of shenanigans that will bypass 
these efforts in the end again


bye JCoehn

Re: The split package problem

[Jochen Theodorou]

On 04.11.2016 09:25, Remi Forax wrote:

There are two issues with split packages,
- if you have the same class in each part of the package, the behavior of your 
problem depend on the order in the classpath,
   i've experienced this kind of bugs with two different libraries requiring 
different version of ASM, at runtime, a class of the older version was calling 
a class of the newer version :(
- security, if you allow split packages, you allow anybody to insert any 
classes in any packages.


ok, not sure if I agree that these are reason enough for the annoyance, 
but at least I know the proper reason now ;)


bye Jochen

The split package problem

[Jochen Theodorou]

Hi all,

I do often read about this "split package problem", but I never did see 
a proper explanation about why it matters to jigsaw so much that we do 
not allow it. Can somebody enlighten me?


bye Jochen

Unsafe and AwkwardStrongEncapsulation

[Jochen Theodorou]

On 04.11.2016 00:32, mark.reinh...@oracle.com wrote:

2016/11/1 14:17:43 -0700, neil.bartl...@paremus.com:

...

In scenarios where a module's author
can't foresee the need for such access (e.g., intrusive serialization
frameworks) then the framework's author must take more drastic measures
(e.g., use the legacy unsupported unsafe API).  This balance is intended
to help maintain the integrity of a module as expressed by its author.


Since Unsafe was mentioned here... All the ways I did know to get to 
Unsafe required reflection on private members. Afaik 
#AwkwardStrongEncapsulation will block that. So how can I still use 
Unsafe in the future or did it get some "proper" public way to get hold 
of an Unsafe by now and I missed that?


bye Jochen

Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages

[Jochen Theodorou]

On 01.11.2016 16:44, Alan Bateman wrote:



On 01/11/2016 15:32, Jochen Theodorou wrote:

:

Well... it makes me ask the question: Does #AwkwardStrongEncapsulation
impose additional limitations for the redefinition of already loaded
classes?

No but if the methods in the new class redefinition are doing
setAccessible then it's same checks as would happen if the original
class bytes did the same thing.


If I got Remi's proposal right, I would "only" have to add the 
annotation on the class by instrumentation to gain access, no need for 
setAccessible


bye Jochen

Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages

[Jochen Theodorou]

I must say I am partially to the whole story.

My main problem is code I do not have under control, but suddenly need 
"extended" rights for. If this comes from a another module I will have 
by default no chance of doing this anymore because of 
#AwkwardStrongEncapsulation.


This proposal shows a way around it, but only if I instrument or 
otherwise transform the bytecode to add the annotation. If I have that 
level of control I can do all sorts of things. My simple problem here 
is, that this may work for a framework you program against, but not for 
a library that happens to have to process the class without the class 
knowing anything about that library. And normally you do not want to 
have a bytecode/class transformer, just to use the library properly. Not 
to mention, that the point of time in which you get a hold on the 
object, it might be much too late to do any modifications to the class


Well... it makes me ask the question: Does #AwkwardStrongEncapsulation 
impose additional limitations for the redefinition of already loaded 
classes?


bye Jochen

Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages

[Jochen Theodorou]

On 01.11.2016 15:04, John Rose wrote:

On Nov 1, 2016, at 9:53 AM, David M. Lloyd  wrote:


1. It requires the target class to be initialized
2. It requires the target class to proactively donate MethodHandles or a Lookup 
to the lookup class


Both of these can be overcome, though only by privileged code.
The privileged code would forge (uh, "mint") a legitimate Lookup to the 
not-yet-initialized class.
A "Vault" meta-framework doesn't need to inject a Lookup donation statement into 
anybody's .
Instead, it needs to do the super-user operation of making a trusted lookup.
It must also fulfill the super-user *responsibility* of not leaking such 
lookups, just using them in a predictable, rule-based manner.


Can we clarify "privileged code"? Privileged like in a SecurityManager 
in a PrivilegedAction for example, for privileged like only jdk internal 
code? Just to see it black on white ;)


bye Jochen

Re: JDK 9 Early Access with Project Jigsaw, build 135 on 09-14-2016 (#5500)

[Jochen Theodorou]

On 02.10.2016 14:43, Claes Redestad wrote:



On 2016-10-02 14:26, Jochen Theodorou wrote:



Do you know anything about the ClassLoader::getPackages issue? Is this a
case where the (new) public getDefinedPackages returns the Packages that
Gradle is looking for?


hmm... if I look at
https://github.com/gradle/gradle/blob/master/subprojects/base-services/src/main/java/org/gradle/internal/classloader/MultiParentClassLoader.java

and I imagine one parent being a MultiParentClassLoader...
getDefinedPackages is final, so you cannot do what they did for
getPackages, unless you do it from outside. And for that you need to
know about multiple parents... I don´t know if it is working well enough
for Gradle, but no, getDefinedPackages is no suitable replacement for
getPackages at all if you have to consider multiparent class loaders.


Since the methods reflected upon in ClassLoader here are all protected,
couldn't this specific code be made reflection-free by wrapping all
parent classloaders in a delegating classloader that overrides
getPackage/getPackages as public?


How is the delegating classloader going to access getPackage of the 
loader it delegates to? Just being a classloader subclass does not solve 
the problem. And you cannot be in the same package as well... not to 
mention, that your class is probably defined in a different classloader 
as well. Or is here a Java-trick I do not know of?


bye Jochen

Re: JDK 9 Early Access with Project Jigsaw, build 135 on 09-14-2016 (#5500)

[Jochen Theodorou]

On 02.10.2016 11:42, Alan Bateman wrote:
[...]

In prior releases
when updates to the Java Language would have required to IDEs and some
tools but wouldn't have wide impact on tools. This is the reason for
early access releases and the ongoing outreach to create awkward of
issues and impact.


it really depends on the project of if things are affecting you or not. 
I think since JDK 1.2 only JDK6 did no require us making changes 
somewhere... sometimes because of things changed in maintenance 
versions, sometimes because of new features conflicting with ours.



On Groovy then Jochen has brought up issues here several times. I get
the impression that there is a lot more to this, often it seems like
issues that should be discussed on mlvm-dev. We have not seen anything
like these issues with Nashorn (Javascript), it is of course based on
method handles and a different design.


Well, if you have a 10 year old project you can expect it depending much 
more on Reflection than something like Nashorn... especially when it is 
coupled tightly to Java and does not build up its own independent universe


bye Jochen

Re: JDK 9 Early Access with Project Jigsaw, build 135 on 09-14-2016 (#5500)

[Jochen Theodorou]

On 02.10.2016 12:12, Alan Bateman wrote:

On 02/10/2016 10:20, Jochen Theodorou wrote:
[...] All I can
suggest is bring up the topic on core-libs-dev. I don't wish to get into
the discussion here on whether it's a good idea or not to add such a
method. Also with Project Panama looking good then one has to wonder if
it make sense to add methods like this.


Project Panama is going to be in JDK9? That is new to me. But even if, 
it makes no major difference, since everything you could have done 
there, you could have done with JNI/JNA and you will still need platform 
specific code for that... unless you use a more independent library, 
which then has to be added in different versions for multiple platforms 
again.


But yes, I too suggest the gradle people bring that to core-lib-dev


On the Process API then just to say that it has been updated
significantly for Java SE 9 to support managing of child processes (and
trees of processes). Details in JEP 102. This might not be exactly what
you are looking for here but I just mention that the API has been
updated to address many long standing shortcomings.


for setting environment variables JEP 102 does not look useful. You can 
applaud the effort for fixing many long standing issues, but if you need 
a certain feature and it becomes "disabled" without having a replacement 
at hand... well you are not in the mood to applaud then. I am sure the 
Gradle people will solve all those problems with one hack or another. It 
just means 3.0 and 3.1 will no longer be jdk9 ready people will have to 
wait for 3.2 or even 3.3



Do you know anything about the ClassLoader::getPackages issue? Is this a
case where the (new) public getDefinedPackages returns the Packages that
Gradle is looking for?


hmm... if I look at 
https://github.com/gradle/gradle/blob/master/subprojects/base-services/src/main/java/org/gradle/internal/classloader/MultiParentClassLoader.java 
and I imagine one parent being a MultiParentClassLoader... 
getDefinedPackages is final, so you cannot do what they did for 
getPackages, unless you do it from outside. And for that you need to 
know about multiple parents... I don´t know if it is working well enough 
for Gradle, but no, getDefinedPackages is no suitable replacement for 
getPackages at all if you have to consider multiparent class loaders.


bye Jochen

Re: JDK 9 Early Access with Project Jigsaw, build 135 on 09-14-2016 (#5500)

[Jochen Theodorou]

On 02.10.2016 09:53, Alan Bateman wrote:

On 02/10/2016 01:25, Malachi de Ælfweald wrote:

:

This appears to come from a setAccessible from inside getEnv
https://github.com/adammurdoch/native-platform/blob/master/src/main/java/net/rubygrapefruit/platform/internal/WrapperProcess.java#L113


Indeed, as we've said in other mails, this change is going to expose a
lot hacks. In this specific case then System.getenv() is specified to
return an unmodifiable map but a library that Gradle uses seems to want
to hack into the underlying map so that it can modify it. Stack trace
below.

Jochen may be able to find out more about this, maybe there is a
description somewhere on what the real issue is. It might be something
that can be tackled in other ways.


The real issue is probably Gradle trying to "fork" the process to a 
running daemon. That includes environment variables for example. Java is 
not very "fork"-friendly. Not in the classic posix sense, and not by 
passing the execution to an already running daemon either.


Then you find things like this: 
http://stackoverflow.com/questions/318239/how-do-i-set-environment-variables-from-java


And remember, there is no ProcessBuilder, if you delegate execution to a 
daemon. Only without the daemon startup times may matter much more. And 
even if in JDK9 the JVM starts faster now, that does not mean you get a 
ready setup to work in from that point on... not if you have to load 
another 10k classes and data structures for example. Gradle is 
struggling with this for years already and I am talking about comparing 
execution speeds of gradle with something like make or scons, not with 
ant, maven or sbt.


So now I would be curious as of what you suggest on how to fix the 
issue. I mean even if you start using native code to set the 
environment, will the java process notice those changes? There is no 
guarantee for that and it may break again even if it works now, right?


bye Jochen

Re: JDK 9 Early Access with Project Jigsaw, build 135 on 09-14-2016 (#5500)

[Jochen Theodorou]

On 02.10.2016 02:25, Malachi de Ælfweald wrote:

With build 125, Gradle 3.1 works fine.
With build 138, I have not yet figured out how to get it running.


The Gradle guys had been very proud of their "we support jdk9" story, 
but #AwkwardStrongEncapsulation really destroyed that for them... for 
now at least.



Trying to run 'gradle init' in a bare directory results in
'java.lang.ExceptionInInitializerError'

Adding '-Dsun.reflect.debugModuleAccessChecks=true' it reported:
16:56:47.501 [ERROR] [system.err]
java.lang.reflect.InaccessibleObjectException: Unable to make protected
java.lang.Package[] java.lang.ClassLoader.getPackages() accessible: module
java.base does not "exports private java.lang" to unnamed module @352c1b98

I changed the _JAVA_OPTIONS to
'--add-exports-private=java.base/java.lang=ALL-UNNAMED
--add-modules=ALL-SYSTEM --add-exports=java.base/sun.nio.ch=ALL-UNNAMED'
(the last two are for Dagger and Neo4j compat)
That resulted in:
Unable to get mutable environment variable map.

This appears to come from a setAccessible from inside getEnv
https://github.com/adammurdoch/native-platform/blob/master/src/main/java/net/rubygrapefruit/platform/internal/WrapperProcess.java#L113


adding --add-exports-private java.base/java.util=ALL-UNNAMED can help a 
bit here and there. Adding the same options above to the 
"org.gradle.jvmargs" in gradle.properties if you fork might too.


And even though this will solve the problem in parts of Gradle, it will 
not solve the problem in the Groovy code called by Gradle. Because 
before I had a logic, that if not all methods can be made accessible, I 
skip the class when creating the meta class and use only the superclass 
and interface information. This logic worked fine with SecurityManagers 
and with the module logic before #AwkwardStrongEncapsulation. Now with 
#AwkwardStrongEncapsulation I have to go through each method and make it 
accessible piece by piece. Without this change the --add-exports-private 
is going to be quite long I imagine... and I am not convinced yet, that 
this solution will work in all relevant cases. It for sure is no 
solution for modules written in Groovy - should they ever come. An 
option to completely turn off #AwkwardStrongEncapsulation would be nice.



So how do we get past #AwkwardStrongEncapsulation?  Do I need to fork
Gradle's dependencies and rebuild both of them since I have no code of my
own in this use case?


you can try making Gradle use Groovy master plus the above. I can´t say 
if it will work then though, haven´t tried that yet.


bye Jochen

Re: Class names in java.lang.Module

[Jochen Theodorou]

On 26.09.2016 11:25, Neil Bartlett wrote:

Module is already in the name: “java.lang.module.Configuration”. Wouldn’t 
“java.lang.module.ModuleConfiguration” look really odd?


ah, you mean like List is enough for java.util.List and java.awt.List? 
Configuration is a really common name in projects.


bye Jochen

Re: Proposal: #ReflectiveAccessToNonExportedTypes (revised) & #AwkwardStrongEncapsulation: Weak modules & private exports

[Jochen Theodorou]

On 12.09.2016 17:08, Mark Reinhold wrote:

Issue summary
-

   #ReflectiveAccessToNonExportedTypes --- Some kinds of framework
   libraries require reflective access to members of the non-exported
   types of other modules; examples include dependency injection (Guice),
   persistence (JPA), debugging tools, code-automation tools, and
   serialization (XStream).  In some cases the particular library to be
   used is not known until run time (e.g., Hibernate and EclipseLink both
   implement JPA).  This capability is also sometimes used to work around
   bugs in unchangeable code.  Access to non-exported packages can, at
   present, only be done via command-line flags, which is extremely
   awkward.  Provide an easier way for reflective code to access such
   non-exported types. [1]

   #AwkwardStrongEncapsulation --- A non-public element of an exported
   package can still be accessed via the `AccessibleObject::setAccessible`
   method of the core reflection API.  The only way to strongly
   encapsulate such an element is to move it to a non-exported package.
   This makes it awkward, at best, to encapsulate the internals of a
   package that defines a public API. [2]

[...]

I´d like to give some feedback on this. Our situation was this, that we 
finally managed to get the Groovy build running on JDK9 with the jigsaw 
version before this change here came live. The situation now is, that 
gradle broke and we are not even starting compilation anymore. Well, can 
happen with a change like this. But the issue at hand was about using 
setAccessible (without exporting private) to get access to a protected 
method in java.base. When this proposal mentioned non-public I was 
actually automatically thinking private, but of course there are at 
least two more visibility options to think of.


So I find another awkward point in #AwkwardStrongEncapsulation in that 
you have exported API and you have a protected method in it. You can 
write a class in your own module, that will overwrite or use that method 
given that it is in a subclass. But if I want to use setAccessible to 
invoke the method I cannot do that? This is awkward to me, because it 
degrades the former (and often misused) swiss-army-knife setAccessible 
to something that is even less capable than what I can do by subclassing 
- unless special action are taken. I can understand the idea for private 
methods or package private - but protected is for me always part of the 
API to program against and as such it makes not sense to me to prevent 
setAccessible accessing it here.


bye Jochen

Re: problem with Class#getResource

[Jochen Theodorou]

On 05.09.2016 17:00, Alan Bateman wrote:

On 05/09/2016 15:50, Jochen Theodorou wrote:



getResource did not get caller sensitive, right?

The Class getResource/getResourceAsStream methods are but this is only
relevant if you are invoke them on classes in named modules. This is why
I was asking what "this.class" was in the previous mail.



ok, turns out getResource is doing the right thing for me actually. The 
error was instead in the JVM forking code deeper down, that just 
surfaced strangely.


Sorry for the false alarm

bye Jochen

Re: problem with Class#getResource

[Jochen Theodorou]

On 05.09.2016 15:15, Jochen Theodorou wrote:


On 04.09.2016 22:05, Alan Bateman wrote:

On 04/09/2016 14:35, Jochen Theodorou wrote:


Hi all,


I am using build 9-ea+132

I have a test, that does in the setup this.class.getResource("/jars"),
to get a URL to the jars directory. The parent directory is on the
classpath, but the call returns null... This code is not run in a
named module.

Can anyone tell me why that returns null with jigsaw?

Can you say what "this.class" is in the example? Also do you see any
difference in behavior between the regular JDK 9 downloads and the
Jigsaw EA downloads?


this.class is the same as this.getClass().

Anyway... turns out this is working if I make a standalone example.
Which means the test setup part must do something wrong when forking a
new JVM.


getResource did not get caller sensitive, right?

bye Jochen

Re: problem with Class#getResource

[Jochen Theodorou]

On 04.09.2016 22:05, Alan Bateman wrote:

On 04/09/2016 14:35, Jochen Theodorou wrote:


Hi all,


I am using build 9-ea+132

I have a test, that does in the setup this.class.getResource("/jars"),
to get a URL to the jars directory. The parent directory is on the
classpath, but the call returns null... This code is not run in a
named module.

Can anyone tell me why that returns null with jigsaw?

Can you say what "this.class" is in the example? Also do you see any
difference in behavior between the regular JDK 9 downloads and the
Jigsaw EA downloads?


this.class is the same as this.getClass().

Anyway... turns out this is working if I make a standalone example. 
Which means the test setup part must do something wrong when forking a 
new JVM.


bye Jochen

Re: problem with Class#getResource

[Jochen Theodorou]

hmmm... makes a sad sense to me. Though it seems to extend to 
ClassLoader#getResources(). That can´t be because of multi-release JAR 
files, can it?


bye Jochen

On 04.09.2016 17:32, Uwe Schindler wrote:

Hi,

I think this is related to multi-release JAR files because directories are no longer 
"unique" for the same resource. I am not 100% sure, but according to specs, 
Resources are only files, never directories, so I'd always say the above code is wrong 
and it is just caused by an implementation detail that it works at all. :-)

Uwe

-
Uwe Schindler
uschind...@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/

Sent: Sunday, September 4, 2016 3:36 PM
To: jigsaw-dev@openjdk.java.net
Subject: problem with Class#getResource

Hi all,


I am using build 9-ea+132

I have a test, that does in the setup this.class.getResource("/jars"),
to get a URL to the jars directory. The parent directory is on the
classpath, but the call returns null... This code is not run in a named
module.

Can anyone tell me why that returns null with jigsaw?

bye Jochen

problem with Class#getResource

[Jochen Theodorou]

Hi all,


I am using build 9-ea+132

I have a test, that does in the setup this.class.getResource("/jars"), 
to get a URL to the jars directory. The parent directory is on the 
classpath, but the call returns null... This code is not run in a named 
module.


Can anyone tell me why that returns null with jigsaw?

bye Jochen

Re: Issue with an automatic module

[Jochen Theodorou]

On 02.08.2016 15:19, Peter Levart wrote:

Hi,

Top-package (unnamed package) classes can not be referenced from classes
in named packages anyway, so they usually represent just reflective
entry points (such as Main class with main() method).


In Java that is you mean. The JVM does not care about the package as 
long as the class can be found, nor does a class loader. It is a java 
compiler restriction, an intended one, but only for those wanting to be 
a java compiler.


bye Jochen

automatically downloading and enabling a jigsaw module

[Jochen Theodorou]

Hi all,

as some may know Groovy has this @Grab feature, which works only 
partially in JDK9, because of the change of not using an URLClassLoader 
anymore for the startup class loaders.


Now I am of course wondering what the exact requirements are for a 
program to get a module jar from the Internet (let us assume we have a 
solution for that part), and then load it, so that a program can use 
classes from this module as well as any services the module may provide. 
And of course let us discuss this for Java-only code, I should then be 
able to make the transition to Groovy code quite easily.


So the scenario I would like to sketch out is the following... I want to 
deliver a precompiled jar that will download its dependencies by itself. 
Let us say we will use a sql driver as well as a custom xml parser and 
asciidoc. The driver spec will be given in at the command line. The 
purpose of the program is to load an xml report, transform it to 
asciidoc and then save that in a predefined sql table.


To start off I guess there would be a small gateway program, that 
handles the command line arguments and issues the download of the 
modules. What comes after that? Is it as easy as spawning a new class 
loader and load the modules in that loader as well as the reminder of 
the program? Is there an alternative way of doing this? How do I add the 
modules to the current layer? Or if a different layer... Will my program 
then still have access to the services?



bye Jochen

Re: Using modules in open source libraries

[Jochen Theodorou]

Groovy-core  44 exported and 25 maybe not exported


On 26.07.2016 12:10, Stephen Colebourne wrote:

There has been a lot of discussion about exporting packages - at
compile time vs runtime and also the related problem of resources. I
decided to take a look at some open source projects to see whether a
list of exported packages is useful as currently defined:

These standard open source projects were designed prior to modules:

Joda-Time - 7 exported packages
Joda-Time-Hibernate - 1 exported package
Joda-Time-I18N - 1 exported package
Joda-Time-JspTags - 1 exported package
Joda-Money - 2 exported packages
Joda-Primitives - 10 exported packages
Joda-Convert - 2 exported packages
Joda-Collect - 1 exported package
Joda-Beans - 16 exported packages
Joda-Beans-Maven-Plugin - 1 exported package
Joda-Beans-UI - 5 exported packages
ThreeTen-Extra - 3 exported packages
ThreeTen-Backport - 6 exported packages
Google Guava - 18 exported packages, 1 non-exported package
SLF4J - 3 exported packages
JCommander - 4 exported packages, 1 non-exported package

Looking at the list, it is clear that non-exported packages are a
small use-case.
- 81 exported packages
- 2 non-exported packages

Strata [1] has been designed in Java 8 but with an awareness that
modules are coming:

Strata-Collect - 8 exported packages
Strata-Basics - 7 exported packages
Strata-Data - 2 exported packages
Strata-Calc - 3 exported packages
Strata-Product - 23 exported packages
Strata-Market - 13 exported packages
Strata-Loader - 3 exported packages, 1 only exported at runtime for reflection
Strata-Math - 1 exported package, 17 packages exported only to a
single friend module
Strata-Pricer - 19 exported packages, 10 non-exported packages
Strata-Measure - 15 exported packages

- 94 exported packages
- 17 exported to a single other module
- 11 non-exported packages

I'll use a different thread to express my opinion on what the data
means. Please use this thread to add summaries of other open source
projects.

Stephen

[1] https://github.com/OpenGamma/Strata

Re: usage of com.sun.tools.javac.Main

[Jochen Theodorou]

On 26.07.2016 09:11, Alan Bateman wrote:

On 26/07/2016 07:58, Jochen Theodorou wrote:


Hi all,

when I tried the Groovy build jigsaw about 2 months ago the usage of
com.sun.tools.javac.Main required me to make the module it was defined
in to be accessible for the unnamed module. In the latest version I
tried this was not required anymore, which was I think JDK9 b128.

Did I use a JDK in which this was not enforced or is there a change
for com.sun.tools.javac.Main?

PS: I know I should not use this class and I was about to change this,
when I noticed that I am missing the error messages about this.

The javac man page documents the entry point com.sun.tools.javac.Main
(the "old interface") and com.sun.tools.javac continues to be exported.
I don't recall a time when it wasn't exported. Jon or Jan might know
more about this. In any case, maybe it is time to move to javax.tools?


Around April 10 this year I had to use 
-XaddExports:jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED to get 
access using the jigsaw build, which means there was a module 
jdk.compiler, that did not export this class.


bye Jochen

usage of com.sun.tools.javac.Main

[Jochen Theodorou]

Hi all,

when I tried the Groovy build jigsaw about 2 months ago the usage of 
com.sun.tools.javac.Main required me to make the module it was defined 
in to be accessible for the unnamed module. In the latest version I 
tried this was not required anymore, which was I think JDK9 b128.


Did I use a JDK in which this was not enforced or is there a change for 
com.sun.tools.javac.Main?


PS: I know I should not use this class and I was about to change this, 
when I noticed that I am missing the error messages about this.


bye Jochen

Re: creating proxies for interfaces with default methods

[Jochen Theodorou]

On 23.07.2016 01:12, John Rose wrote:

On May 27, 2016, at 1:16 AM, Alan Bateman > wrote:


I don't think it's possible to create a Lookup via support API with
just the PRIVATE lookup mode. It only seems to work because this code
seems to hack into the non-public constructor. I'm curious if you
invoke toString on this, with JDK 8 and with -esa, as I assume you
will get a similar assertion.


The Lookup API does not have a "setAccessible" analog.
Maybe it should; along the lines of Unsafe.privateLookup(Class).


would be nice, because then I would not have to depend on a hack

bye Jochen