Hi, sorry I did not read all, so you already use my code:
The exception you see is coming from the fact that the first part for Java 9 does not work at all. It looks like the code uses Java 8 variant (see line number in exception) instead of the Java 9 variant. Can you make a breakpoint here: https://svn.apache.org/viewvc/poi/trunk/src/java/org/apache/poi/poifs/nio/CleanerUtil.java?view=markup#l99 And step through the code? It might fall through to the catch block and java 8 code. Uwe ----- Uwe Schindler Achterdiek 19, D-28357 Bremen https://www.thetaphi.de eMail: u...@thetaphi.de > -----Original Message----- > From: Uwe Schindler <uschind...@pangaea.de> > Sent: Friday, August 7, 2020 5:06 PM > To: 'Andreas Beeker' <kiwiwi...@apache.org>; uschind...@apache.org > Cc: dev@poi.apache.org > Subject: RE: Java 9+ modules / cleaner > > Hi, > > > I currently try to understand, how to call the Cleaner in Java 14 (or 9+) > without > > adding the --add-opens JVM options. > > Yeah, your code won't work correctly with Java 9 at all. You may fix it with > some opens, but still types of internal calsses changed, so its just risky > (everything is subject to change). > > > As you've worked on this in LUCENE-6989, you might have a few hints for me. > > I think you can more or less copy the code from Lucene (use the branch_8x > version, as Master requires Java 11, so has no Java 8 code anymore, see here: > https://tinyurl.com/y47euqfg). The aproach in Lucene does NOT use Reflection > at all, it works with Method Handles. The important thing is: Method handles > are linked earyl, once you have built the final method handle to invoke the > cleaner, you can call it without requesting any additional right (security > manager). In addition you know beforehand if it works at all (it cannot throw > extra reflective exceptions). > > The main change in Java 9 is (and this is what's officially "supported" by > OpenJDK developer): They added a new method to sun.misc.Unsafe (the legacy > one), Unsafe#invokeCleaner(ByteBuffer). This class is still in java.base and > is > open to public (if you know how to get the singleton) for the time being. To > get > the singleton, you need reflection and the code must allow to do setAccessible > on the getter, but once you have it, it's useable. > > To unmap a Bytebuffer, Lucene creates a MethodHandle with some signature > like "unmap(ByteBuffer b)" complete pre-configued on startup of class > depending on Java version: > - In Java 9 and above it uses reflection to get the Unsafe instance (this > requires > security manager to allow it). Then it looks up the method > "invokeCleaner(ByteBuffer) and binds it to the unsafe singleton, the final > methodhandle is casted to me "void unmap(ByteBuffer)" > - In Java 8 and before it uses more or less the old approach by checking the > private method to get the Cleaner instance. Finally it calls cleaner.clean(). > This > can also be composed to a MethodHandle with exactly same signature (using > the famous MethodHandle transformation and bindings, introducing some null > checks). The result is also a methodhandle with signature "void > unmap(ByteBuffer)". > > Once all this is done, the methodhandle with platform independent signature > can be called without any exception handling from any code, so be sure to keep > it safe in private final fields fully internal to your implementaion > (otherwise it's > a security issue). > > > I've checked the Lucene implementation, but that code is similar to POIs > > current implementation. [1] > > As I don't see the Runable interface, I might look at the wrong branch. > > That won't work. It's the same approach like the old one, just with other > class > types. You cant work around the internal Cleaner interface. This is why > sun.misc.Unsafe#invokeCleaner() was added. > > > Any ideas? > > See above. I'd copy the code from Lucene: It's early binding and failsafe, > once > you got the MethodHandle. The approach should work with Java 7+. In Java 7 > there is one additional helper method needed for the methodHandle regarding > the null check! > > Java 8 is here: https://tinyurl.com/y47euqfg > Hack for compatibility with Java 7 is here: https://tinyurl.com/y4drev3k (this > adds a "compatibility method" to replace missing "Objects#nonNull(Object)", > but it's identical otherwise; we added this to make Lucene 5.5 still > compatible > with Java 9, long after support ended, because customers need this). > > Uwe > > > Best wishes, > > Andi > > > > > > [1] > > https://github.com/apache/lucene- > > > solr/blob/master/lucene/core/src/java/org/apache/lucene/store/MMapDirecto > > ry.java#L338 > > vs. > > > https://svn.apache.org/viewvc/poi/trunk/src/java/org/apache/poi/poifs/nio/Cle > > anerUtil.java?view=markup#l91 > > > > On 09.07.20 00:34, Andreas Beeker wrote: > > > Hi Dominik, > > > > > > the goal is to have no --add-opens or similar jvm arguments. In this case > > > we > > get the following exception: > > > > > > [junit] java.lang.reflect.InaccessibleObjectException: Unable to make > public > > jdk.internal.ref.Cleaner java.nio.DirectByteBuffer.cleaner() accessible: > > module > > java.base does not "opens java.nio" to module org.apache.poi.poi > > > [junit] at > > > java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleO > > bject.java:349) > > > [junit] at > > > java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleO > > bject.java:289) > > > [junit] at > > java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:196) > > > [junit] at > > java.base/java.lang.reflect.Method.setAccessible(Method.java:190) > > > [junit] at > > > org.apache.poi.poi/org.apache.poi.poifs.nio.CleanerUtil.unmapHackImpl(Cleane > > rUtil.java:116) > > > [junit] at > > > java.base/java.security.AccessController.doPrivileged(AccessController.java:312 > > ) > > > [junit] at > > > org.apache.poi.poi/org.apache.poi.poifs.nio.CleanerUtil.<clinit>(CleanerUtil.jav > > a:77) > > > [junit] at > > > org.apache.poi.poi/org.apache.poi.poifs.nio.FileBackedDataSource.unmap(FileB > > ackedDataSource.java:189) > > > [junit] at > > > org.apache.poi.poi/org.apache.poi.poifs.nio.FileBackedDataSource.lambda$clos > > e$0(FileBackedDataSource.java:162) > > > [junit] at > > java.base/java.util.IdentityHashMap.forEach(IdentityHashMap.java:1356) > > > [junit] at > > > org.apache.poi.poi/org.apache.poi.poifs.nio.FileBackedDataSource.close(FileBac > > kedDataSource.java:162) > > > [junit] at > > > org.apache.poi.poi/org.apache.poi.poifs.filesystem.POIFSFileSystem.close(POIFS > > FileSystem.java:764) > > > [junit] at > > > org.apache.poi.poi/org.apache.poi.hpsf.basic.TestWrite.inPlacePOIFSWrite(Test > > Write.java:539) > > > > > > If we use the Runnable aproach mentioned in the lucene bug, this should > > work again. > > > > > > In the meantime I had to move a few sources from ooxml to main > > (crypto.agile) and we need to find a way, e.g. using multi release classes, > > for > > our WorkbookFactory, as with JIgsaw the poi main module can't reflect into > > ooxml anymore. This was also a reason for moving the agile crypto into poi > > main and removing the use XmlBeans schemas there. > > > > > > Best wishes, > > > Andi > > > > > > > > > On 07.07.20 18:00, Dominik Stadler wrote: > > >> Hi, > > >> > > >> Sorry for replying late here, not sure if you already did these changes, > > >> but the code in CleanerUtil tries to handle this gracefully with not > > >> cleaning on JDKs which do not support this. > > >> > > >> There should be no compile-time dependency on any unsafe object, during > > >> runtime we try to get a cleaner and simply not unmap buffers cleanly if > not > > >> possible for some reason. > > >> > > >> Which new problem do you see with this approach when using the JDK > > module > > >> system? > > >> > > >> Thanks... Dominik. > > >> > > >> On Mon, Jun 29, 2020 at 10:36 PM Andreas Beeker > > <kiwiwi...@apache.org> > > >> wrote: > > >> > > >>> Hi *, > > >>> > > >>> I'm facing the same problem as described in > > >>> https://issues.apache.org/jira/browse/LUCENE-6989 > > >>> > > >>> Is it ok for you, if I get our build more or less to run with the module > > >>> path (instead of the classpath) when running in a JDK 9+ and later try > > >>> to > > >>> fix the above Cleaner problem? > > >>> > > >>> I simply would like to focus on one issue now and as we a have > > >>> multi-release jars after my commit, a JDK dependent solution shouldn't > be > > a > > >>> problem anymore. > > >>> > > >>> Andi > > >>> > > >>> > > >>> > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org > > > For additional commands, e-mail: dev-h...@poi.apache.org > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org > For additional commands, e-mail: dev-h...@poi.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional commands, e-mail: dev-h...@poi.apache.org
