To confirm: If you use POI as a JAR on classpath it will work without modifications: Try it out. For the module system you need to import jdk.unsupported module.
This is hard to figure out, the stack trace gave me the hint. It’s a HUGE difference if you put a JAR file on classpath (there it works) or on modulepath (won't work without importing jdk.unsupported). In both cases, if everything is declared correctly in JAR file no exports/imports or command line options are needed. Lucene does not support Modules at all, so it's no issue for us. Uwe ----- Uwe Schindler [email protected] ASF Member, Apache Lucene PMC / Committer Bremen, Germany https://lucene.apache.org/ > -----Original Message----- > From: Uwe Schindler <[email protected]> > Sent: Friday, August 7, 2020 6:12 PM > To: 'POI Developers List' <[email protected]>; 'Andreas Beeker' > <[email protected]> > Subject: RE: Java 9+ modules / cleaner > > Ah I think I know!The code is fine, it's just how you use it. > > Could it be that you are compiling this as a module and also use it as a > module > (the stack trace implies this)? If you use the code as a "real" module, you > have > to import explicitely in your module-info the "jdk.unsupported" module, > otherwise you won't get access to sun.misc. You may need to add this to the > auto-module descriptor or module-info.java. > > This must not only be done in tests, Your JAR file MUST declare that it needs > "jdk.unsupported" module. If it does not do this CleanerUtil is useless. In > contrast to the classpath approach, where there is a default set of modules > enabled, in a modulepath you only get what you declare - very simple. > > In short: > You code fails, because the Class.forName() in the Java 9 code will not find > "sun.misc.Unsafe", because its shielded by the module system. Because of this > it falls back to the Java 8 code, which won't work at all (causing the > exceptions). > > Uwe > > ----- > Uwe Schindler > [email protected] > ASF Member, Apache Lucene PMC / Committer > Bremen, Germany > https://lucene.apache.org/ > > > -----Original Message----- > > From: Uwe Schindler <[email protected]> > > Sent: Friday, August 7, 2020 5:20 PM > > To: 'POI Developers List' <[email protected]>; 'Andreas Beeker' > > <[email protected]>; [email protected] > > Subject: RE: Java 9+ modules / cleaner > > > > Hi, > > > > sorry I did not read all, so you already use my code: > > > > The exception you see is coming from the fact that the first part for Java 9 > does > > not work at all. It looks like the code uses Java 8 variant (see line > > number in > > exception) instead of the Java 9 variant. > > > > Can you make a breakpoint here: > > > https://svn.apache.org/viewvc/poi/trunk/src/java/org/apache/poi/poifs/nio/Cle > > anerUtil.java?view=markup#l99 > > > > And step through the code? It might fall through to the catch block and > > java 8 > > code. > > > > Uwe > > > > ----- > > Uwe Schindler > > Achterdiek 19, D-28357 Bremen > > https://www.thetaphi.de > > eMail: [email protected] > > > > > -----Original Message----- > > > From: Uwe Schindler <[email protected]> > > > Sent: Friday, August 7, 2020 5:06 PM > > > To: 'Andreas Beeker' <[email protected]>; [email protected] > > > Cc: [email protected] > > > Subject: RE: Java 9+ modules / cleaner > > > > > > Hi, > > > > > > > I currently try to understand, how to call the Cleaner in Java 14 (or > > > > 9+) > > > without > > > > adding the --add-opens JVM options. > > > > > > Yeah, your code won't work correctly with Java 9 at all. You may fix it > > > with > > > some opens, but still types of internal calsses changed, so its just risky > > > (everything is subject to change). > > > > > > > As you've worked on this in LUCENE-6989, you might have a few hints for > > me. > > > > > > I think you can more or less copy the code from Lucene (use the branch_8x > > > version, as Master requires Java 11, so has no Java 8 code anymore, see > here: > > > https://tinyurl.com/y47euqfg). The aproach in Lucene does NOT use > > Reflection > > > at all, it works with Method Handles. The important thing is: Method > handles > > > are linked earyl, once you have built the final method handle to invoke > > > the > > > cleaner, you can call it without requesting any additional right (security > > > manager). In addition you know beforehand if it works at all (it cannot > throw > > > extra reflective exceptions). > > > > > > The main change in Java 9 is (and this is what's officially "supported" by > > > OpenJDK developer): They added a new method to sun.misc.Unsafe (the > > legacy > > > one), Unsafe#invokeCleaner(ByteBuffer). This class is still in java.base > > > and is > > > open to public (if you know how to get the singleton) for the time being. > > > To > > get > > > the singleton, you need reflection and the code must allow to do > > setAccessible > > > on the getter, but once you have it, it's useable. > > > > > > To unmap a Bytebuffer, Lucene creates a MethodHandle with some > signature > > > like "unmap(ByteBuffer b)" complete pre-configued on startup of class > > > depending on Java version: > > > - In Java 9 and above it uses reflection to get the Unsafe instance (this > > requires > > > security manager to allow it). Then it looks up the method > > > "invokeCleaner(ByteBuffer) and binds it to the unsafe singleton, the > > > final > > > methodhandle is casted to me "void unmap(ByteBuffer)" > > > - In Java 8 and before it uses more or less the old approach by checking > > > the > > > private method to get the Cleaner instance. Finally it calls > > > cleaner.clean(). > > This > > > can also be composed to a MethodHandle with exactly same signature > (using > > > the famous MethodHandle transformation and bindings, introducing some > > null > > > checks). The result is also a methodhandle with signature "void > > > unmap(ByteBuffer)". > > > > > > Once all this is done, the methodhandle with platform independent > signature > > > can be called without any exception handling from any code, so be sure to > > keep > > > it safe in private final fields fully internal to your implementaion > > > (otherwise > > it's > > > a security issue). > > > > > > > I've checked the Lucene implementation, but that code is similar to POIs > > > > current implementation. [1] > > > > As I don't see the Runable interface, I might look at the wrong branch. > > > > > > That won't work. It's the same approach like the old one, just with other > class > > > types. You cant work around the internal Cleaner interface. This is why > > > sun.misc.Unsafe#invokeCleaner() was added. > > > > > > > Any ideas? > > > > > > See above. I'd copy the code from Lucene: It's early binding and failsafe, > once > > > you got the MethodHandle. The approach should work with Java 7+. In Java > 7 > > > there is one additional helper method needed for the methodHandle > > regarding > > > the null check! > > > > > > Java 8 is here: https://tinyurl.com/y47euqfg > > > Hack for compatibility with Java 7 is here: https://tinyurl.com/y4drev3k > (this > > > adds a "compatibility method" to replace missing > "Objects#nonNull(Object)", > > > but it's identical otherwise; we added this to make Lucene 5.5 still > compatible > > > with Java 9, long after support ended, because customers need this). > > > > > > Uwe > > > > > > > Best wishes, > > > > Andi > > > > > > > > > > > > [1] > > > > https://github.com/apache/lucene- > > > > > > > > > > solr/blob/master/lucene/core/src/java/org/apache/lucene/store/MMapDirecto > > > > ry.java#L338 > > > > vs. > > > > > > > > > > https://svn.apache.org/viewvc/poi/trunk/src/java/org/apache/poi/poifs/nio/Cle > > > > anerUtil.java?view=markup#l91 > > > > > > > > On 09.07.20 00:34, Andreas Beeker wrote: > > > > > Hi Dominik, > > > > > > > > > > the goal is to have no --add-opens or similar jvm arguments. In this > > > > > case > > we > > > > get the following exception: > > > > > > > > > > [junit] java.lang.reflect.InaccessibleObjectException: Unable to > > > > > make > > > public > > > > jdk.internal.ref.Cleaner java.nio.DirectByteBuffer.cleaner() accessible: > > module > > > > java.base does not "opens java.nio" to module org.apache.poi.poi > > > > > [junit] at > > > > > > > > > > java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleO > > > > bject.java:349) > > > > > [junit] at > > > > > > > > > > java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleO > > > > bject.java:289) > > > > > [junit] at > > > > > > java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:196) > > > > > [junit] at > > > > java.base/java.lang.reflect.Method.setAccessible(Method.java:190) > > > > > [junit] at > > > > > > > > > > org.apache.poi.poi/org.apache.poi.poifs.nio.CleanerUtil.unmapHackImpl(Cleane > > > > rUtil.java:116) > > > > > [junit] at > > > > > > > > > > java.base/java.security.AccessController.doPrivileged(AccessController.java:312 > > > > ) > > > > > [junit] at > > > > > > > > > > org.apache.poi.poi/org.apache.poi.poifs.nio.CleanerUtil.<clinit>(CleanerUtil.jav > > > > a:77) > > > > > [junit] at > > > > > > > > > > org.apache.poi.poi/org.apache.poi.poifs.nio.FileBackedDataSource.unmap(FileB > > > > ackedDataSource.java:189) > > > > > [junit] at > > > > > > > > > > org.apache.poi.poi/org.apache.poi.poifs.nio.FileBackedDataSource.lambda$clos > > > > e$0(FileBackedDataSource.java:162) > > > > > [junit] at > > > > java.base/java.util.IdentityHashMap.forEach(IdentityHashMap.java:1356) > > > > > [junit] at > > > > > > > > > > org.apache.poi.poi/org.apache.poi.poifs.nio.FileBackedDataSource.close(FileBac > > > > kedDataSource.java:162) > > > > > [junit] at > > > > > > > > > > org.apache.poi.poi/org.apache.poi.poifs.filesystem.POIFSFileSystem.close(POIFS > > > > FileSystem.java:764) > > > > > [junit] at > > > > > > > > > > org.apache.poi.poi/org.apache.poi.hpsf.basic.TestWrite.inPlacePOIFSWrite(Test > > > > Write.java:539) > > > > > > > > > > If we use the Runnable aproach mentioned in the lucene bug, this > should > > > > work again. > > > > > > > > > > In the meantime I had to move a few sources from ooxml to main > > > > (crypto.agile) and we need to find a way, e.g. using multi release > > > > classes, > for > > > > our WorkbookFactory, as with JIgsaw the poi main module can't reflect > into > > > > ooxml anymore. This was also a reason for moving the agile crypto into > poi > > > > main and removing the use XmlBeans schemas there. > > > > > > > > > > Best wishes, > > > > > Andi > > > > > > > > > > > > > > > On 07.07.20 18:00, Dominik Stadler wrote: > > > > >> Hi, > > > > >> > > > > >> Sorry for replying late here, not sure if you already did these > > > > >> changes, > > > > >> but the code in CleanerUtil tries to handle this gracefully with not > > > > >> cleaning on JDKs which do not support this. > > > > >> > > > > >> There should be no compile-time dependency on any unsafe object, > > during > > > > >> runtime we try to get a cleaner and simply not unmap buffers cleanly > > > > >> if > > > not > > > > >> possible for some reason. > > > > >> > > > > >> Which new problem do you see with this approach when using the JDK > > > > module > > > > >> system? > > > > >> > > > > >> Thanks... Dominik. > > > > >> > > > > >> On Mon, Jun 29, 2020 at 10:36 PM Andreas Beeker > > > > <[email protected]> > > > > >> wrote: > > > > >> > > > > >>> Hi *, > > > > >>> > > > > >>> I'm facing the same problem as described in > > > > >>> https://issues.apache.org/jira/browse/LUCENE-6989 > > > > >>> > > > > >>> Is it ok for you, if I get our build more or less to run with the > > > > >>> module > > > > >>> path (instead of the classpath) when running in a JDK 9+ and later > > > > >>> try > to > > > > >>> fix the above Cleaner problem? > > > > >>> > > > > >>> I simply would like to focus on one issue now and as we a have > > > > >>> multi-release jars after my commit, a JDK dependent solution > shouldn't > > > be > > > > a > > > > >>> problem anymore. > > > > >>> > > > > >>> Andi > > > > >>> > > > > >>> > > > > >>> > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > To unsubscribe, e-mail: [email protected] > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [email protected] > > > For additional commands, e-mail: [email protected] > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
