Re: Java 9+ modules / cleaner
The warning printed by the code in Lucene (and the fork of Hadoop in POI) informs user. Btw, there's still "Hadoop" in your code. 😁 It's interesting that JDK prints the exception to stderr. This was not happening in Java 9. Looks like a bug. Uwe Am August 7, 2020 9:58:04 PM UTC schrieb Andreas Beeker : >> The problem was only the missing import of the unsupported module >which is required for unmapping. >I think, I should also add the information about the permissions to a >FAQ entry: >https://stackoverflow.com/a/54046774/2066598 > >> I still wonder where the exception came from. >Have a look at AccessibleObject:349 which prints the stacktrace to >System.Err > >> I would recommend to put the module-info.class into the multi release >part >Of course it will be eventually put into meta-inf/versions/9 ... but >my goal was to release with Java 8, so I need to save the .class file >when compiling with Java 9+ before. > >Andi > > >On 07.08.20 23:07, Uwe Schindler wrote: >> CleanerUtil dies not need to be multireleased. Just leave as is. >> >> The problem was only the missing import of the unsupported module >which is required for unmapping. If you are in module mode at runtime >(module path) and the unsupported module is not there, the exception >handler falls through to the Java 8 code. After that it complains that >nothing works. >> >> I still wonder where the exception came from. The code catches all of >them and converts to a message which should get logged. So this is why >I asked how you got the exception! >> >> I would recommend to put the module-info.class into the multi release >part of the jar (below meta-inf/versions/9). >> >> Uwe >> >> Am August 7, 2020 8:18:48 PM UTC schrieb Andreas Beeker >: >>> Thank you Uwe. >>> >>> The "jdk.unsupported" module hint did the trick - at least the build >is >>> now getting over that error. >>> >>> So our build uses classpath mode when run under Java 8 and >modulepath >>> for Java 9+ - actually everything except of the module-info is >compiled >>> in classpath mode and the module-info is then patched into the >classes. >>> To release under Java 8, I compile/update the module-info.class to >the >>> source when compiling under Java 9+ - so this development noise is a >>> bit of a drawback which we might solve by using timestamp checks in >the >>> build. >>> >>> I thought about providing a multi-release version of the >CleanerUtil, >>> so we have only the Java 8 or the Java 9+ code active. >>> But then I'm puzzled how the JRE reacts, if you use classpath mode >in >>> Java 9+. >>> I guess it ignores the module-info modules and the patched >>> multi-release class, effectively running the Java 8 code. >>> So we need both versions with that fallback mechanism. >>> TL;DR: don't change CleanerUtil. >>> >>> >>> >>> >>> On 07.08.20 18:24, Uwe Schindler wrote: >>>>> Sorry for not understanding the whole problem (module system usage >>> instaed >>>>> of classpath). Your code is perfectly valid and works with Java 9, >>> it's just broken >>>> ...with module system as its not fully declared all imports. The >>> compile won't find the issue, as it's in dynamic linking code. -- Uwe Schindler Achterdiek 19, 28357 Bremen https://www.thetaphi.de
Re: Java 9+ modules / cleaner
CleanerUtil dies not need to be multireleased. Just leave as is. The problem was only the missing import of the unsupported module which is required for unmapping. If you are in module mode at runtime (module path) and the unsupported module is not there, the exception handler falls through to the Java 8 code. After that it complains that nothing works. I still wonder where the exception came from. The code catches all of them and converts to a message which should get logged. So this is why I asked how you got the exception! I would recommend to put the module-info.class into the multi release part of the jar (below meta-inf/versions/9). Uwe Am August 7, 2020 8:18:48 PM UTC schrieb Andreas Beeker : >Thank you Uwe. > >The "jdk.unsupported" module hint did the trick - at least the build is >now getting over that error. > >So our build uses classpath mode when run under Java 8 and modulepath >for Java 9+ - actually everything except of the module-info is compiled >in classpath mode and the module-info is then patched into the classes. >To release under Java 8, I compile/update the module-info.class to the >source when compiling under Java 9+ - so this development noise is a >bit of a drawback which we might solve by using timestamp checks in the >build. > >I thought about providing a multi-release version of the CleanerUtil, >so we have only the Java 8 or the Java 9+ code active. >But then I'm puzzled how the JRE reacts, if you use classpath mode in >Java 9+. >I guess it ignores the module-info modules and the patched >multi-release class, effectively running the Java 8 code. >So we need both versions with that fallback mechanism. >TL;DR: don't change CleanerUtil. > > > > >On 07.08.20 18:24, Uwe Schindler wrote: >>> Sorry for not understanding the whole problem (module system usage >instaed >>> of classpath). Your code is perfectly valid and works with Java 9, >it's just broken >> ...with module system as its not fully declared all imports. The >compile won't find the issue, as it's in dynamic linking code. >>
RE: Java 9+ modules / cleaner
> Sorry for not understanding the whole problem (module system usage instaed > of classpath). Your code is perfectly valid and works with Java 9, it's just > broken ...with module system as its not fully declared all imports. The compile won't find the issue, as it's in dynamic linking code. > - > Uwe Schindler > uschind...@apache.org > ASF Member, Apache Lucene PMC / Committer > Bremen, Germany > https://lucene.apache.org/ > > > -Original Message- > > From: Uwe Schindler > > Sent: Friday, August 7, 2020 6:16 PM > > To: 'POI Developers List' ; 'Andreas Beeker' > > > > Subject: RE: Java 9+ modules / cleaner > > > > To confirm: If you use POI as a JAR on classpath it will work without > > modifications: Try it out. > > For the module system you need to import jdk.unsupported module. > > > > This is hard to figure out, the stack trace gave me the hint. It’s a HUGE > > difference if you put a JAR file on classpath (there it works) or on > > modulepath > > (won't work without importing jdk.unsupported). > > > > In both cases, if everything is declared correctly in JAR file no > > exports/imports > > or command line options are needed. > > > > Lucene does not support Modules at all, so it's no issue for us. > > > > Uwe > > > > - > > Uwe Schindler > > uschind...@apache.org > > ASF Member, Apache Lucene PMC / Committer > > Bremen, Germany > > https://lucene.apache.org/ > > > > > -Original Message- > > > From: Uwe Schindler > > > Sent: Friday, August 7, 2020 6:12 PM > > > To: 'POI Developers List' ; 'Andreas Beeker' > > > > > > Subject: RE: Java 9+ modules / cleaner > > > > > > Ah I think I know!The code is fine, it's just how you use it. > > > > > > Could it be that you are compiling this as a module and also use it as a > > module > > > (the stack trace implies this)? If you use the code as a "real" module, > > > you > have > > > to import explicitely in your module-info the "jdk.unsupported" module, > > > otherwise you won't get access to sun.misc. You may need to add this to > > > the > > > auto-module descriptor or module-info.java. > > > > > > This must not only be done in tests, Your JAR file MUST declare that it > needs > > > "jdk.unsupported" module. If it does not do this CleanerUtil is useless. > > > In > > > contrast to the classpath approach, where there is a default set of > > > modules > > > enabled, in a modulepath you only get what you declare - very simple. > > > > > > In short: > > > You code fails, because the Class.forName() in the Java 9 code will not > > > find > > > "sun.misc.Unsafe", because its shielded by the module system. Because of > > this > > > it falls back to the Java 8 code, which won't work at all (causing the > > > exceptions). > > > > > > Uwe > > > > > > - > > > Uwe Schindler > > > uschind...@apache.org > > > ASF Member, Apache Lucene PMC / Committer > > > Bremen, Germany > > > https://lucene.apache.org/ > > > > > > > -Original Message- > > > > From: Uwe Schindler > > > > Sent: Friday, August 7, 2020 5:20 PM > > > > To: 'POI Developers List' ; 'Andreas Beeker' > > > > ; uschind...@apache.org > > > > Subject: RE: Java 9+ modules / cleaner > > > > > > > > Hi, > > > > > > > > sorry I did not read all, so you already use my code: > > > > > > > > The exception you see is coming from the fact that the first part for > > > > Java 9 > > > does > > > > not work at all. It looks like the code uses Java 8 variant (see line > > > > number > in > > > > exception) instead of the Java 9 variant. > > > > > > > > Can you make a breakpoint here: > > > > > > > > > > https://svn.apache.org/viewvc/poi/trunk/src/java/org/apache/poi/poifs/nio/Cle > > > > anerUtil.java?view=markup#l99 > > > > > > > > And step through the code? It might fall through to the catch block and > java > > 8 > > > > code. > > > > > > > > Uwe > > > > > > > > - > > > > U
RE: Java 9+ modules / cleaner
See also here: https://blogs.oracle.com/java/get-ready-jdk9 Sorry for not understanding the whole problem (module system usage instaed of classpath). Your code is perfectly valid and works with Java 9, it's just broken I still wonder why you get this Exception at all. It should be swallowed by the last try/catch block and saved as a warning for "UNMAP_UNSUPPORTED". Or did you hack the code to get the full stack trace? Uwe - Uwe Schindler uschind...@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany https://lucene.apache.org/ > -Original Message- > From: Uwe Schindler > Sent: Friday, August 7, 2020 6:16 PM > To: 'POI Developers List' ; 'Andreas Beeker' > > Subject: RE: Java 9+ modules / cleaner > > To confirm: If you use POI as a JAR on classpath it will work without > modifications: Try it out. > For the module system you need to import jdk.unsupported module. > > This is hard to figure out, the stack trace gave me the hint. It’s a HUGE > difference if you put a JAR file on classpath (there it works) or on > modulepath > (won't work without importing jdk.unsupported). > > In both cases, if everything is declared correctly in JAR file no > exports/imports > or command line options are needed. > > Lucene does not support Modules at all, so it's no issue for us. > > Uwe > > - > Uwe Schindler > uschind...@apache.org > ASF Member, Apache Lucene PMC / Committer > Bremen, Germany > https://lucene.apache.org/ > > > -Original Message- > > From: Uwe Schindler > > Sent: Friday, August 7, 2020 6:12 PM > > To: 'POI Developers List' ; 'Andreas Beeker' > > > > Subject: RE: Java 9+ modules / cleaner > > > > Ah I think I know!The code is fine, it's just how you use it. > > > > Could it be that you are compiling this as a module and also use it as a > module > > (the stack trace implies this)? If you use the code as a "real" module, you > > have > > to import explicitely in your module-info the "jdk.unsupported" module, > > otherwise you won't get access to sun.misc. You may need to add this to the > > auto-module descriptor or module-info.java. > > > > This must not only be done in tests, Your JAR file MUST declare that it > > needs > > "jdk.unsupported" module. If it does not do this CleanerUtil is useless. In > > contrast to the classpath approach, where there is a default set of modules > > enabled, in a modulepath you only get what you declare - very simple. > > > > In short: > > You code fails, because the Class.forName() in the Java 9 code will not find > > "sun.misc.Unsafe", because its shielded by the module system. Because of > this > > it falls back to the Java 8 code, which won't work at all (causing the > > exceptions). > > > > Uwe > > > > - > > Uwe Schindler > > uschind...@apache.org > > ASF Member, Apache Lucene PMC / Committer > > Bremen, Germany > > https://lucene.apache.org/ > > > > > -Original Message- > > > From: Uwe Schindler > > > Sent: Friday, August 7, 2020 5:20 PM > > > To: 'POI Developers List' ; 'Andreas Beeker' > > > ; uschind...@apache.org > > > Subject: RE: Java 9+ modules / cleaner > > > > > > Hi, > > > > > > sorry I did not read all, so you already use my code: > > > > > > The exception you see is coming from the fact that the first part for > > > Java 9 > > does > > > not work at all. It looks like the code uses Java 8 variant (see line > > > number in > > > exception) instead of the Java 9 variant. > > > > > > Can you make a breakpoint here: > > > > > > https://svn.apache.org/viewvc/poi/trunk/src/java/org/apache/poi/poifs/nio/Cle > > > anerUtil.java?view=markup#l99 > > > > > > And step through the code? It might fall through to the catch block and > > > java > 8 > > > code. > > > > > > Uwe > > > > > > - > > > Uwe Schindler > > > Achterdiek 19, D-28357 Bremen > > > https://www.thetaphi.de > > > eMail: u...@thetaphi.de > > > > > > > -Original Message- > > > > From: Uwe Schindler > > > > Sent: Friday, August 7, 2020 5:06 PM > > > > To: 'Andreas Beeker' ; uschind...@apache.org > > > > Cc: dev@poi.apache.org > > > > Subject: RE: Java 9+ modules / cleaner >
RE: Java 9+ modules / cleaner
To confirm: If you use POI as a JAR on classpath it will work without modifications: Try it out. For the module system you need to import jdk.unsupported module. This is hard to figure out, the stack trace gave me the hint. It’s a HUGE difference if you put a JAR file on classpath (there it works) or on modulepath (won't work without importing jdk.unsupported). In both cases, if everything is declared correctly in JAR file no exports/imports or command line options are needed. Lucene does not support Modules at all, so it's no issue for us. Uwe - Uwe Schindler uschind...@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany https://lucene.apache.org/ > -Original Message----- > From: Uwe Schindler > Sent: Friday, August 7, 2020 6:12 PM > To: 'POI Developers List' ; 'Andreas Beeker' > > Subject: RE: Java 9+ modules / cleaner > > Ah I think I know!The code is fine, it's just how you use it. > > Could it be that you are compiling this as a module and also use it as a > module > (the stack trace implies this)? If you use the code as a "real" module, you > have > to import explicitely in your module-info the "jdk.unsupported" module, > otherwise you won't get access to sun.misc. You may need to add this to the > auto-module descriptor or module-info.java. > > This must not only be done in tests, Your JAR file MUST declare that it needs > "jdk.unsupported" module. If it does not do this CleanerUtil is useless. In > contrast to the classpath approach, where there is a default set of modules > enabled, in a modulepath you only get what you declare - very simple. > > In short: > You code fails, because the Class.forName() in the Java 9 code will not find > "sun.misc.Unsafe", because its shielded by the module system. Because of this > it falls back to the Java 8 code, which won't work at all (causing the > exceptions). > > Uwe > > ----- > Uwe Schindler > uschind...@apache.org > ASF Member, Apache Lucene PMC / Committer > Bremen, Germany > https://lucene.apache.org/ > > > -Original Message- > > From: Uwe Schindler > > Sent: Friday, August 7, 2020 5:20 PM > > To: 'POI Developers List' ; 'Andreas Beeker' > > ; uschind...@apache.org > > Subject: RE: Java 9+ modules / cleaner > > > > Hi, > > > > sorry I did not read all, so you already use my code: > > > > The exception you see is coming from the fact that the first part for Java 9 > does > > not work at all. It looks like the code uses Java 8 variant (see line > > number in > > exception) instead of the Java 9 variant. > > > > Can you make a breakpoint here: > > > https://svn.apache.org/viewvc/poi/trunk/src/java/org/apache/poi/poifs/nio/Cle > > anerUtil.java?view=markup#l99 > > > > And step through the code? It might fall through to the catch block and > > java 8 > > code. > > > > Uwe > > > > - > > Uwe Schindler > > Achterdiek 19, D-28357 Bremen > > https://www.thetaphi.de > > eMail: u...@thetaphi.de > > > > > -Original Message- > > > From: Uwe Schindler > > > Sent: Friday, August 7, 2020 5:06 PM > > > To: 'Andreas Beeker' ; uschind...@apache.org > > > Cc: dev@poi.apache.org > > > Subject: RE: Java 9+ modules / cleaner > > > > > > Hi, > > > > > > > I currently try to understand, how to call the Cleaner in Java 14 (or > > > > 9+) > > > without > > > > adding the --add-opens JVM options. > > > > > > Yeah, your code won't work correctly with Java 9 at all. You may fix it > > > with > > > some opens, but still types of internal calsses changed, so its just risky > > > (everything is subject to change). > > > > > > > As you've worked on this in LUCENE-6989, you might have a few hints for > > me. > > > > > > I think you can more or less copy the code from Lucene (use the branch_8x > > > version, as Master requires Java 11, so has no Java 8 code anymore, see > here: > > > https://tinyurl.com/y47euqfg). The aproach in Lucene does NOT use > > Reflection > > > at all, it works with Method Handles. The important thing is: Method > handles > > > are linked earyl, once you have built the final method handle to invoke > > > the > > > cleaner, you can call it without requesting any additional right (security > > > manager). In addition you know beforehand if it works at all (it cannot > throw >
RE: Java 9+ modules / cleaner
Ah I think I know!The code is fine, it's just how you use it. Could it be that you are compiling this as a module and also use it as a module (the stack trace implies this)? If you use the code as a "real" module, you have to import explicitely in your module-info the "jdk.unsupported" module, otherwise you won't get access to sun.misc. You may need to add this to the auto-module descriptor or module-info.java. This must not only be done in tests, Your JAR file MUST declare that it needs "jdk.unsupported" module. If it does not do this CleanerUtil is useless. In contrast to the classpath approach, where there is a default set of modules enabled, in a modulepath you only get what you declare - very simple. In short: You code fails, because the Class.forName() in the Java 9 code will not find "sun.misc.Unsafe", because its shielded by the module system. Because of this it falls back to the Java 8 code, which won't work at all (causing the exceptions). Uwe - Uwe Schindler uschind...@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany https://lucene.apache.org/ > -Original Message- > From: Uwe Schindler > Sent: Friday, August 7, 2020 5:20 PM > To: 'POI Developers List' ; 'Andreas Beeker' > ; uschind...@apache.org > Subject: RE: Java 9+ modules / cleaner > > Hi, > > sorry I did not read all, so you already use my code: > > The exception you see is coming from the fact that the first part for Java 9 > does > not work at all. It looks like the code uses Java 8 variant (see line number > in > exception) instead of the Java 9 variant. > > Can you make a breakpoint here: > https://svn.apache.org/viewvc/poi/trunk/src/java/org/apache/poi/poifs/nio/Cle > anerUtil.java?view=markup#l99 > > And step through the code? It might fall through to the catch block and java 8 > code. > > Uwe > > - > Uwe Schindler > Achterdiek 19, D-28357 Bremen > https://www.thetaphi.de > eMail: u...@thetaphi.de > > > -Original Message- > > From: Uwe Schindler > > Sent: Friday, August 7, 2020 5:06 PM > > To: 'Andreas Beeker' ; uschind...@apache.org > > Cc: dev@poi.apache.org > > Subject: RE: Java 9+ modules / cleaner > > > > Hi, > > > > > I currently try to understand, how to call the Cleaner in Java 14 (or 9+) > > without > > > adding the --add-opens JVM options. > > > > Yeah, your code won't work correctly with Java 9 at all. You may fix it with > > some opens, but still types of internal calsses changed, so its just risky > > (everything is subject to change). > > > > > As you've worked on this in LUCENE-6989, you might have a few hints for > me. > > > > I think you can more or less copy the code from Lucene (use the branch_8x > > version, as Master requires Java 11, so has no Java 8 code anymore, see > > here: > > https://tinyurl.com/y47euqfg). The aproach in Lucene does NOT use > Reflection > > at all, it works with Method Handles. The important thing is: Method handles > > are linked earyl, once you have built the final method handle to invoke the > > cleaner, you can call it without requesting any additional right (security > > manager). In addition you know beforehand if it works at all (it cannot > > throw > > extra reflective exceptions). > > > > The main change in Java 9 is (and this is what's officially "supported" by > > OpenJDK developer): They added a new method to sun.misc.Unsafe (the > legacy > > one), Unsafe#invokeCleaner(ByteBuffer). This class is still in java.base > > and is > > open to public (if you know how to get the singleton) for the time being. To > get > > the singleton, you need reflection and the code must allow to do > setAccessible > > on the getter, but once you have it, it's useable. > > > > To unmap a Bytebuffer, Lucene creates a MethodHandle with some signature > > like "unmap(ByteBuffer b)" complete pre-configued on startup of class > > depending on Java version: > > - In Java 9 and above it uses reflection to get the Unsafe instance (this > requires > > security manager to allow it). Then it looks up the method > > "invokeCleaner(ByteBuffer) and binds it to the unsafe singleton, the final > > methodhandle is casted to me "void unmap(ByteBuffer)" > > - In Java 8 and before it uses more or less the old approach by checking the > > private method to get the Cleaner instance. Finally it calls > > cleaner.clean(). > This > > can also be composed to a MethodHandle with exactly same
RE: Java 9+ modules / cleaner
Hi, sorry I did not read all, so you already use my code: The exception you see is coming from the fact that the first part for Java 9 does not work at all. It looks like the code uses Java 8 variant (see line number in exception) instead of the Java 9 variant. Can you make a breakpoint here: https://svn.apache.org/viewvc/poi/trunk/src/java/org/apache/poi/poifs/nio/CleanerUtil.java?view=markup#l99 And step through the code? It might fall through to the catch block and java 8 code. Uwe - Uwe Schindler Achterdiek 19, D-28357 Bremen https://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Uwe Schindler > Sent: Friday, August 7, 2020 5:06 PM > To: 'Andreas Beeker' ; uschind...@apache.org > Cc: dev@poi.apache.org > Subject: RE: Java 9+ modules / cleaner > > Hi, > > > I currently try to understand, how to call the Cleaner in Java 14 (or 9+) > without > > adding the --add-opens JVM options. > > Yeah, your code won't work correctly with Java 9 at all. You may fix it with > some opens, but still types of internal calsses changed, so its just risky > (everything is subject to change). > > > As you've worked on this in LUCENE-6989, you might have a few hints for me. > > I think you can more or less copy the code from Lucene (use the branch_8x > version, as Master requires Java 11, so has no Java 8 code anymore, see here: > https://tinyurl.com/y47euqfg). The aproach in Lucene does NOT use Reflection > at all, it works with Method Handles. The important thing is: Method handles > are linked earyl, once you have built the final method handle to invoke the > cleaner, you can call it without requesting any additional right (security > manager). In addition you know beforehand if it works at all (it cannot throw > extra reflective exceptions). > > The main change in Java 9 is (and this is what's officially "supported" by > OpenJDK developer): They added a new method to sun.misc.Unsafe (the legacy > one), Unsafe#invokeCleaner(ByteBuffer). This class is still in java.base and > is > open to public (if you know how to get the singleton) for the time being. To > get > the singleton, you need reflection and the code must allow to do setAccessible > on the getter, but once you have it, it's useable. > > To unmap a Bytebuffer, Lucene creates a MethodHandle with some signature > like "unmap(ByteBuffer b)" complete pre-configued on startup of class > depending on Java version: > - In Java 9 and above it uses reflection to get the Unsafe instance (this > requires > security manager to allow it). Then it looks up the method > "invokeCleaner(ByteBuffer) and binds it to the unsafe singleton, the final > methodhandle is casted to me "void unmap(ByteBuffer)" > - In Java 8 and before it uses more or less the old approach by checking the > private method to get the Cleaner instance. Finally it calls cleaner.clean(). > This > can also be composed to a MethodHandle with exactly same signature (using > the famous MethodHandle transformation and bindings, introducing some null > checks). The result is also a methodhandle with signature "void > unmap(ByteBuffer)". > > Once all this is done, the methodhandle with platform independent signature > can be called without any exception handling from any code, so be sure to keep > it safe in private final fields fully internal to your implementaion > (otherwise it's > a security issue). > > > I've checked the Lucene implementation, but that code is similar to POIs > > current implementation. [1] > > As I don't see the Runable interface, I might look at the wrong branch. > > That won't work. It's the same approach like the old one, just with other > class > types. You cant work around the internal Cleaner interface. This is why > sun.misc.Unsafe#invokeCleaner() was added. > > > Any ideas? > > See above. I'd copy the code from Lucene: It's early binding and failsafe, > once > you got the MethodHandle. The approach should work with Java 7+. In Java 7 > there is one additional helper method needed for the methodHandle regarding > the null check! > > Java 8 is here: https://tinyurl.com/y47euqfg > Hack for compatibility with Java 7 is here: https://tinyurl.com/y4drev3k (this > adds a "compatibility method" to replace missing "Objects#nonNull(Object)", > but it's identical otherwise; we added this to make Lucene 5.5 still > compatible > with Java 9, long after support ended, because customers need this). > > Uwe > > > Best wishes, > > Andi > > > > > > [1] > > https://github.com/apache/l
RE: Java 9+ modules / cleaner
Hi, > I currently try to understand, how to call the Cleaner in Java 14 (or 9+) > without > adding the --add-opens JVM options. Yeah, your code won't work correctly with Java 9 at all. You may fix it with some opens, but still types of internal calsses changed, so its just risky (everything is subject to change). > As you've worked on this in LUCENE-6989, you might have a few hints for me. I think you can more or less copy the code from Lucene (use the branch_8x version, as Master requires Java 11, so has no Java 8 code anymore, see here: https://tinyurl.com/y47euqfg). The aproach in Lucene does NOT use Reflection at all, it works with Method Handles. The important thing is: Method handles are linked earyl, once you have built the final method handle to invoke the cleaner, you can call it without requesting any additional right (security manager). In addition you know beforehand if it works at all (it cannot throw extra reflective exceptions). The main change in Java 9 is (and this is what's officially "supported" by OpenJDK developer): They added a new method to sun.misc.Unsafe (the legacy one), Unsafe#invokeCleaner(ByteBuffer). This class is still in java.base and is open to public (if you know how to get the singleton) for the time being. To get the singleton, you need reflection and the code must allow to do setAccessible on the getter, but once you have it, it's useable. To unmap a Bytebuffer, Lucene creates a MethodHandle with some signature like "unmap(ByteBuffer b)" complete pre-configued on startup of class depending on Java version: - In Java 9 and above it uses reflection to get the Unsafe instance (this requires security manager to allow it). Then it looks up the method "invokeCleaner(ByteBuffer) and binds it to the unsafe singleton, the final methodhandle is casted to me "void unmap(ByteBuffer)" - In Java 8 and before it uses more or less the old approach by checking the private method to get the Cleaner instance. Finally it calls cleaner.clean(). This can also be composed to a MethodHandle with exactly same signature (using the famous MethodHandle transformation and bindings, introducing some null checks). The result is also a methodhandle with signature "void unmap(ByteBuffer)". Once all this is done, the methodhandle with platform independent signature can be called without any exception handling from any code, so be sure to keep it safe in private final fields fully internal to your implementaion (otherwise it's a security issue). > I've checked the Lucene implementation, but that code is similar to POIs > current implementation. [1] > As I don't see the Runable interface, I might look at the wrong branch. That won't work. It's the same approach like the old one, just with other class types. You cant work around the internal Cleaner interface. This is why sun.misc.Unsafe#invokeCleaner() was added. > Any ideas? See above. I'd copy the code from Lucene: It's early binding and failsafe, once you got the MethodHandle. The approach should work with Java 7+. In Java 7 there is one additional helper method needed for the methodHandle regarding the null check! Java 8 is here: https://tinyurl.com/y47euqfg Hack for compatibility with Java 7 is here: https://tinyurl.com/y4drev3k (this adds a "compatibility method" to replace missing "Objects#nonNull(Object)", but it's identical otherwise; we added this to make Lucene 5.5 still compatible with Java 9, long after support ended, because customers need this). Uwe > Best wishes, > Andi > > > [1] > https://github.com/apache/lucene- > solr/blob/master/lucene/core/src/java/org/apache/lucene/store/MMapDirecto > ry.java#L338 > vs. > https://svn.apache.org/viewvc/poi/trunk/src/java/org/apache/poi/poifs/nio/Cle > anerUtil.java?view=markup#l91 > > On 09.07.20 00:34, Andreas Beeker wrote: > > Hi Dominik, > > > > the goal is to have no --add-opens or similar jvm arguments. In this case we > get the following exception: > > > >[junit] java.lang.reflect.InaccessibleObjectException: Unable to make > > public > jdk.internal.ref.Cleaner java.nio.DirectByteBuffer.cleaner() accessible: > module > java.base does not "opens java.nio" to module org.apache.poi.poi > > [junit] at > java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleO > bject.java:349) > > [junit] at > java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleO > bject.java:289) > > [junit] at > java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:196) > > [junit] at > java.base/java.lang.reflect.Method.setAccessible(Method.java:190) > > [junit] at > org.apache.poi.poi/org.apache.poi.poifs.nio.CleanerUtil.unmapHackImpl(Cleane > rUtil.java:116) > > [junit] at > java.base/java.security.AccessController.doPrivileged(AccessController.java:312 > ) > > [junit] at > org.apache.poi.poi/org.apache.poi.poifs.nio.CleanerUtil.(CleanerUtil.jav > a:77
Re: Build failed in Jenkins: POI-DSL-1.10 #16
t; [echo] Archives: 0 > [echo] Standards: 3783 > [echo] > [echo] Apache Licensed: 3783 > [echo] Generated Documents: 0 > [echo] > [echo] JavaDocs are generated, thus a license header is optional. > [echo] Generated files do not require license headers. > [echo] > [echo] 0 Unknown Licenses > [echo] > [echo] * > [echo] Files with Apache License headers will be marked AL >[echo] Binary files (which do not require any license headers) will >be marked B > [echo] Compressed archives will be marked A > [echo] Notices, licenses etc. will be marked N > [echo] > [echo] * > >forbidden-apis-check: >[forbiddenapis] Reading bundled API signatures: jdk-unsafe-1.8 >[forbiddenapis] Reading bundled API signatures: jdk-deprecated-1.8 >[forbiddenapis] Reading bundled API signatures: jdk-internal-1.8 >[forbiddenapis] Reading bundled API signatures: jdk-non-portable >[forbiddenapis] Reading bundled API signatures: jdk-reflection >[forbiddenapis] Reading API signatures: ><https://builds.apache.org/job/POI-DSL-1.10/ws/src/resources/devtools/forbidden-signatures.txt> >[forbiddenapis] Loading classes to check... >[forbiddenapis] Scanning classes for violations... > >BUILD FAILED ><https://builds.apache.org/job/POI-DSL-1.10/ws/build.xml>:2133: Check >for forbidden API calls failed: java.lang.ClassNotFoundException: >javax.xml.bind.annotation.XmlAccessorType > at >de.thetaphi.forbiddenapis.Checker.getClassFromClassLoader(Checker.java:322) > at >de.thetaphi.forbiddenapis.Checker.lookupRelatedClass(Checker.java:334) > at >de.thetaphi.forbiddenapis.ClassScanner.checkClassUse(ClassScanner.java:120) > at >de.thetaphi.forbiddenapis.ClassScanner.checkAnnotationDescriptor(ClassScanner.java:216) > at >de.thetaphi.forbiddenapis.ClassScanner.visitAnnotation(ClassScanner.java:255) > at >de.thetaphi.forbiddenapis.asm.ClassReader.accept(ClassReader.java:526) > at >de.thetaphi.forbiddenapis.asm.ClassReader.accept(ClassReader.java:355) > at de.thetaphi.forbiddenapis.Checker.checkClass(Checker.java:622) > at de.thetaphi.forbiddenapis.Checker.run(Checker.java:639) > at de.thetaphi.forbiddenapis.ant.AntTask.execute(AntTask.java:208) > at >org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293) > at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown >Source) > at >java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:564) > at >org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106) > at org.apache.tools.ant.Task.perform(Task.java:348) > at org.apache.tools.ant.Target.execute(Target.java:435) > at org.apache.tools.ant.Target.performTasks(Target.java:456) > at >org.apache.tools.ant.Project.executeSortedTargets(Project.java:1405) > at org.apache.tools.ant.Project.executeTarget(Project.java:1376) > at >org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41) > at org.apache.tools.ant.Project.executeTargets(Project.java:1260) > at org.apache.tools.ant.Main.runBuild(Main.java:857) > at org.apache.tools.ant.Main.startAnt(Main.java:236) > at org.apache.tools.ant.launch.Launcher.run(Launcher.java:287) > at org.apache.tools.ant.launch.Launcher.main(Launcher.java:113) > >Total time: 21 minutes 12 seconds >Build step 'Invoke Ant' marked build as failure >Skipping publisher since build result is FAILURE >Archiving artifacts >[Fast Archiver] No prior successful build to compare, so performing >full copy of artifacts >Skipping publisher since build result is FAILURE >Recording test results > >- >To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org >For additional commands, e-mail: dev-h...@poi.apache.org -- Uwe Schindler Achterdiek 19, 28357 Bremen https://www.thetaphi.de
RE: publishing poi xmlbeans jars
Hi, if you will publish it with another POM coordinate, be sure to also change package names of all Java classes inside. It's hard for Maven/Gradle to detect a classpath package clash with different coordinates. You could end up with 2 versions of xmlbeans on classpath! So separate Java package names is important, too. So better get it out of attic! Uwe ----- Uwe Schindler Achterdiek 19, D-28357 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: pj.fanning [mailto:fannin...@yahoo.com] > Sent: Tuesday, February 20, 2018 2:26 PM > To: dev@poi.apache.org > Subject: RE: publishing poi xmlbeans jars > > I have a small preference from getting xmlbeans out of the attic but I don't > know who to talk to about that. > Would someone be able to point us in then right direction? > > If the attic approach is politcally complicated, then proceeding with > publishing an org.apache.poi:xmlbeans shouldn't be hard. > > > > -- > Sent from: http://apache-poi.1045710.n5.nabble.com/POI-Dev- > f2312866.html > > - > To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org > For additional commands, e-mail: dev-h...@poi.apache.org - To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional commands, e-mail: dev-h...@poi.apache.org
RE: xmlbeans successor
Hi, In addition, Java 9 does not ship with JAXB enabled by default (see my previous mail about that). You have to pass a command line arg to enable the Java EE modules (next to CORBA and other J2EE stuff). Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Andreas Beeker [mailto:kiwiwi...@apache.org] > Sent: Thursday, April 21, 2016 1:46 AM > To: POI Developers List > Subject: xmlbeans successor > > Hi *, > > I'm still struggling with the jaxb replacement for the xml signing classes. > Apart of spending a lot of time to get namespaces/prefix right (... and it > still > doesn't work), > so that the signatures are valid again, I've realized that jaxb in java6 > might be > also a problem ... > > The reason for changing the implementation is of course testing jaxb in a not > so > important area of POI, but is jaxb really a good pick? > > random thoughts: > > - we get more and more android requests, where jaxb is not supported or > only made possible > by custom hacks [1] ... should we try to make POI compatible to Android? > > - specific to the signing, the xmlsec lib also use jaxb itself - if I try to > omit jaxb > for the sake of android compatibility, > I'd likely need to reimplement the relevant parts of xmlsec [2] > > - having a xml serializer independent of the java version could be handy ... > but security issues (e.g. XXE) shouldn't be overlooked > > - although there are other serializer (e.g. SimpleXml [3]) - we still need to > keep the xml-infoset (or unknown native records) ... do we? > would there be a way to have a common / format independent model albeit > preserving unknown records / xml-data? > I think about the performance discussion lately, where we could handle > more data when we transform the data to an internal model > ... but probably ignore unknown elements. (This applies of course to non- > streaming mode ...) > > Just give me your 2 cents, so I see, if it's worth to get my stuff working ... > > Andi > > > [1] http://stackoverflow.com/questions/9734921/is-there-a-need-in-jaxb- > implementation-for-android > http://www.docx4java.org/blog/2012/05/jaxb-can-be-made-to-run-on- > android/ > [2] http://stackoverflow.com/questions/5991290/xml-soap-signing-on- > android > [3] http://simple.sourceforge.net/ > > > - > To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org > For additional commands, e-mail: dev-h...@poi.apache.org - To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional commands, e-mail: dev-h...@poi.apache.org
RE: Finding deprecated methods
Hi, You can also download the JAR files of those releases and use a comparison tool like http://siom79.github.io/japicmp/ to compare them. This tool can be run using CLI and also prints changes in @Deprecated. The CLI syntax is here: http://siom79.github.io/japicmp/CliTool.html I always use "-m" (only show modified stuff) and "-o" and "-n" to specify the old and new JAR file. After that you can grep on something like DEPRECATED. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Nick Burch [mailto:apa...@gagravarr.org] > Sent: Wednesday, June 15, 2016 10:43 AM > To: POI Developers List > Subject: Re: Finding deprecated methods > > On Tue, 14 Jun 2016, Javen O'Neal wrote: > > Is there a smarter way to find deprecated classes besides checking out > > the code from 2 final releases ago and removing any deprecated code that > > still exists in trunk@HEAD? > > I'd probably attempt it with: > * grep trunk for @deprecated with no date, with a -A 4 or similar, then >grab the method name from that > * grab a list of old versions from "svn ls ^/poi/tags/" > * use "svn switch ^/poi/tags//" to quickly walk your tree back through >older versions > * do the same greps, and spot where they vanish > > If that all sounds a bit hairy, let me know and I can try it later in the > week! > > Nick > > - > To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org > For additional commands, e-mail: dev-h...@poi.apache.org - To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional commands, e-mail: dev-h...@poi.apache.org
JAXB and Java 9
Hi, I just wanted to inform you that the last build of JDK9, build 118 (https://jdk9.java.net/download/) had some changes about which of the core modules are exported by default to non-module enabled apps (those deployed on classpath - the standard way to deploy apps today). One change to note is: JAXB and a few others are no longer visible by default from Java 9 on (you get ClassNotFoundEx), so users have to add a command line option to make it visible! For more information, see Alan Bateman's statement: http://mail.openjdk.java.net/pipermail/jdk9-dev/2016-May/004309.html If we plan to migrate to JAXB, we have to think about how to handle that: - Add some up-to-date JAXB impl as Maven dependency. This was also the main reason why JAXB was disabled by default: Code is now able to ship with their own, up-to-date impl of those modules (like Java EE servers). - Document that on Java 9 you have to write your application using the module system or pass an additional command line option to java Uwe ----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de - To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional commands, e-mail: dev-h...@poi.apache.org
RE: Google App Engine compatibility
Hi, > From bug 59194, https://bz.apache.org/bugzilla/show_bug.cgi?id=59194 > > If we want to make POI be compatible with Google App Engine, should we > add > java.awt.* to the forbidden API's list? Needs to be "java.awt.**"; same syntax like Ant for cross directory file matches. Uwe - To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional commands, e-mail: dev-h...@poi.apache.org
RE: Java 6 support
Hi, > I think we should switch to Java 7 post-3.14, I don't think any of those > things will reduce code size a lot, but most of these are quite useful as > soon as you get used to them. Most of them can be applied automatically using Eclipse (e.g. diamonds). Multi-catch is very useful. Also think of refactoring reflection stuff with the new ReflectiveOperationException, common superclass of all reflection exceptions. Those changes reduce code size and improve readability. Try-with-resources should be applied with great care for existing code. New code should always use it where possible (try-with resources won't work for classes that open a file in constructor and close them afterwards). I think there should be some code review that checks for leaks on resources, also making classes Closeable. Uwe > > Dominik. > > On Wed, Dec 23, 2015 at 10:52 AM, Javen O'Neal > wrote: > > > Oracle released Java 6 in 2006, ended support for Java 6 in 2013 and > > released Java 7 in 2011 and ended support in 2015. Java 8 was released > > in 2014. > > > > There are a few language features introduced in Java 7 that would be > > nice to use in POI. This would mean asking everyone to move off of > > Java 6, but seems justifiable. > > > > I'd really like to use some of the Java 8 features, but usage of Java > > 7 is too prevalent today. > > > > A few Java 7 features that would be nice to use in the POI library: > > * switch statements on strings > > * improved generics type inference (diamond operator) > > * try-with-resources > > * try/multi-catch > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org > > For additional commands, e-mail: dev-h...@poi.apache.org > > > > - To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional commands, e-mail: dev-h...@poi.apache.org
RE: svn commit: r1713218 - in /poi/trunk: build.xml src/resources/devtools/forbidden-signatures.txt
I fixed the forbiddenapis issue. The problem was incomplete classpath. Forbiddenapis used the javadoc.classpath, which did not include the excelant.classpath which has the ANT jar files as downloaded from Maven Central. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: cen...@apache.org [mailto:cen...@apache.org] > Sent: Sunday, November 08, 2015 11:02 AM > To: comm...@poi.apache.org > Subject: svn commit: r1713218 - in /poi/trunk: build.xml > src/resources/devtools/forbidden-signatures.txt > > Author: centic > Date: Sun Nov 8 10:01:48 2015 > New Revision: 1713218 > > URL: http://svn.apache.org/viewvc?rev=1713218&view=rev > Log: > Tried to add excelant to forbidden-apis-check, but failed due to issue #82 in > forbidden-api-checks tool. List more pathes that we still not include. Add > description to custom forbidden-api-checks and add commented out checks > for reflectoin as discussed in bug 58597 > > Modified: > poi/trunk/build.xml > poi/trunk/src/resources/devtools/forbidden-signatures.txt > > Modified: poi/trunk/build.xml > URL: > http://svn.apache.org/viewvc/poi/trunk/build.xml?rev=1713218&r1=171321 > 7&r2=1713218&view=diff > == > > --- poi/trunk/build.xml (original) > +++ poi/trunk/build.xml Sun Nov 8 10:01:48 2015 > @@ -1767,19 +1767,32 @@ under the License. > internalRuntimeForbidden="true" > classpathref="javadoc.classpath" > - > suppressAnnotation="org.apache.poi.util.SuppressForbidden" > - > > + suppressAnnotation="org.apache.poi.util.SuppressForbidden" > +> > > > - > - > - > - > - > - > - > + > + > + > + > + > + > + > + > + > + > + > + > > > > > Modified: poi/trunk/src/resources/devtools/forbidden-signatures.txt > URL: > http://svn.apache.org/viewvc/poi/trunk/src/resources/devtools/forbidden- > signatures.txt?rev=1713218&r1=1713217&r2=1713218&view=diff > == > > --- poi/trunk/src/resources/devtools/forbidden-signatures.txt (original) > +++ poi/trunk/src/resources/devtools/forbidden-signatures.txt Sun Nov 8 > 10:01:48 2015 > @@ -20,7 +20,13 @@ > @ignoreUnresolvable > @defaultMessage POI forbidden APIs > > -java.util.Locale#getDefault() > -java.util.Locale#setDefault(java.util.Locale) > -java.util.TimeZone#getDefault() > -java.util.Date#toString() > \ No newline at end of file > +# Locale related interfaces which we want to avoid to not have code which > depends on the locale of the current machine > +java.util.Locale#getDefault() @ Do not use methods that depend on the > current Local, either use Locale.ROOT or let the user define the local, see > class LocaleUtil for details > +java.util.Locale#setDefault(java.util.Locale) @ Do not use methods that > depend on the current Local, either use Locale.ROOT or let the user define > the local, see class LocaleUtil for details > +java.util.TimeZone#getDefault() @ Do not use methods that depend on the > current Local, either use Locale.ROOT or let the user define the local, see > class LocaleUtil for details > +java.util.Date#toString() @ Do not use methods that depend on the current > Local, either use Locale.ROOT or let the user define the local, see class > LocaleUtil for details > + > +# disabled as there are still invocations that we could not remove easily > +#java.lang.reflect.AccessibleObject#setAccessible(java.lang.reflect.Accessi > bleObject[], boolean) @ Reflection usage fails with SecurityManagers and > likely will not work any more in Java 9 > +#java.lang.reflect.AccessibleObject#setAccessible(boolean) @ Reflection > usage fails with SecurityManagers and likely will not work any more in Java 9 > +#java.lang.reflect.Method#invoke(java.lang.Object, java.lang.Object[]) @ > Reflection usage fails with SecurityManagers and likely will not work any > more in Java 9 > > > > - > To unsubscribe, e-mail: commits-unsubscr...@poi.apache.org > For additional commands, e-mail: commits-h...@poi.apache.org - To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional commands, e-mail: dev-h...@poi.apache.org
RE: svn commit: r1709427 - /poi/trunk/src/ooxml/java/org/apache/poi/xdgf/usermodel/section/CombinedIterable.java
Just unrelated to the original fix: According to checks, the EmptyIterator should throw NoSuchElementException. But I would remove the impl completely! It is much simpler to just return Collections.emptySet().iterator() when empty iterators are required; this is also just returning a static singleton, so no speed improvement at all! Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: virtu...@apache.org [mailto:virtu...@apache.org] > Sent: Monday, October 19, 2015 4:56 PM > To: comm...@poi.apache.org > Subject: svn commit: r1709427 - > /poi/trunk/src/ooxml/java/org/apache/poi/xdgf/usermodel/section/Combi > nedIterable.java > > Author: virtuald > Date: Mon Oct 19 14:55:30 2015 > New Revision: 1709427 > > URL: http://svn.apache.org/viewvc?rev=1709427&view=rev > Log: > XDGF: fix jdk < 1.8 compat > > Modified: > > poi/trunk/src/ooxml/java/org/apache/poi/xdgf/usermodel/section/Combin > edIterable.java > > Modified: > poi/trunk/src/ooxml/java/org/apache/poi/xdgf/usermodel/section/Combin > edIterable.java > URL: > http://svn.apache.org/viewvc/poi/trunk/src/ooxml/java/org/apache/poi/xd > gf/usermodel/section/CombinedIterable.java?rev=1709427&r1=1709426&r2 > =1709427&view=diff > == > > --- > poi/trunk/src/ooxml/java/org/apache/poi/xdgf/usermodel/section/Combin > edIterable.java (original) > +++ > poi/trunk/src/ooxml/java/org/apache/poi/xdgf/usermodel/section/Combi > +++ nedIterable.java Mon Oct 19 14:55:30 2015 > @@ -17,7 +17,6 @@ > > package org.apache.poi.xdgf.usermodel.section; > > -import java.util.Collections; > import java.util.Iterator; > import java.util.Map.Entry; > import java.util.NoSuchElementException; @@ -35,14 +34,21 @@ public > class CombinedIterable impleme > > private static final class EmptyIterator implements Iterator { > > +@Override > public boolean hasNext() { > return false; > } > > +@Override > public T next() { > return null; > } > > +@Override > +public void remove() { > +throw new UnsupportedOperationException(); > +} > + > } > > public CombinedIterable(SortedMap baseItems, > > > > - > To unsubscribe, e-mail: commits-unsubscr...@poi.apache.org > For additional commands, e-mail: commits-h...@poi.apache.org - To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional commands, e-mail: dev-h...@poi.apache.org
RE: svn commit: r1706151 - /poi/trunk/build.xml
This was fast :-) I was about to patch this. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: cen...@apache.org [mailto:cen...@apache.org] > Sent: Wednesday, September 30, 2015 10:00 PM > To: comm...@poi.apache.org > Subject: svn commit: r1706151 - /poi/trunk/build.xml > > Author: centic > Date: Wed Sep 30 20:00:14 2015 > New Revision: 1706151 > > URL: http://svn.apache.org/viewvc?rev=1706151&view=rev > Log: > Update forbidden-api checks to new release 2.0 > > Modified: > poi/trunk/build.xml > > Modified: poi/trunk/build.xml > URL: > http://svn.apache.org/viewvc/poi/trunk/build.xml?rev=1706151&r1=170615 > 0&r2=1706151&view=diff > == > > --- poi/trunk/build.xml (original) > +++ poi/trunk/build.xml Wed Sep 30 20:00:14 2015 > @@ -193,8 +193,8 @@ under the License. > > > value="${repository.m2}/maven2/org/apache/rat/apache-rat/0.11/apache- > rat-0.11.jar"/> > - > - value="${repository.m2}/maven2/de/thetaphi/forbiddenapis/1.8/forbidden > apis-1.8.jar"/> > + > + value="${repository.m2}/maven2/de/thetaphi/forbiddenapis/2.0/forbidden > apis-2.0.jar"/> > > > > > > > - > To unsubscribe, e-mail: commits-unsubscr...@poi.apache.org > For additional commands, e-mail: commits-h...@poi.apache.org - To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional commands, e-mail: dev-h...@poi.apache.org
RE: Time Zone Setting
Hi, This may be only slightly related: to work around the forbidden-apis issue, you can still pass Timezone.getDefault() to the Java APIs - this is not forbidden. Forbidden APIs just wants to make sure you know what you are doing. By writing Timezone.getDefault() you explicitely say what you are doing. And maybe add a comment why you do this. The same applies for Locales or Charsets. E.g., At some places in Lucene we explicitely use the charset of the console, so we use Charset.getDefault() at all those places. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Dominik Stadler [mailto:dominik.stad...@gmx.at] > Sent: Thursday, September 03, 2015 4:05 PM > To: POI Developers List > Subject: Re: Time Zone Setting > > Hi, > > Sorry, but I fear that through this we change the default behavior of POI: > * A ThreadLocal can lead to hard-to-find errors when people use POI in a > multi-threading environment. While it is better in some cases than having a > global static, it now requires the developer to ensure that the timezone is > set > for each thread that is started without a way to set it globally. > * I think previously POI used the java default settings which look at the > operating system, this is now broken as we have "UTC" as default, which > potentially leads to many people seeing different results compared to > before, in fact almost anybody would now see UTC-formatted dates instead > of local timezone, or? > * The java default is to either use the system default timezone or if it > should > be different then use -Duser.timezone when invoking the JRE, see e.g. > http://stackoverflow.com/questions/2493749/how-to-set-a-jvm-timezone- > properly > > So my approach would be to: > * Not use the DateUtil.setTimezone() approach as it has quite some potential > for regressions but also not use TimeZone.setDefault() in unit tests at all > * Ensure that we set -Duser.timezone to a fixed value when running unit > tests in the various CI systems. We already do this for user.language and > user.country, so setting the timezone as well here seems a natural choice > > This way we do not change default behavior for users and still make the CI > runs a bit more reproducible time-wise... > > Dominik. > > On Wed, Sep 2, 2015 at 9:38 AM, Andreas Beeker > wrote: > > Hi, > > > > in the code we have some places where we reset the default time zone > [1], like the following: > > > > TimeZone cet = TimeZone.getTimeZone("Europe/Copenhagen"); > > TimeZone.setDefault(cet); > > > > Although this is just in a junit test, I don't think it's a good idea to > > set the jdk > default. > > I've added a new ThreadLocal to DateUtil and although I'm not > > convinced, that this is a definite solution, at least it's pragmatical: > > > > @BeforeClass > > public static void setTimeZone() { > > userTimeZone = DateUtil.getUserTimeZone(); > > DateUtil.setUserTimeZone(TimeZone.getTimeZone("CET")); > > } > > > > Another solution would be, to guess the TimeZone by parsing date cells > > in the workbook and compare them to the stored string values ... of course > this only applies to X/HSLF. > > > > What do you think about it? > > > > Andi. > > > > > > > > [1] org.apache.poi.hssf.usermodel.TestHSSFDateUtil > > > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional > > commands, e-mail: dev-h...@poi.apache.org > > > > - > To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional > commands, e-mail: dev-h...@poi.apache.org - To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional commands, e-mail: dev-h...@poi.apache.org
RE: getting exception on XMLEventFactory.newFatory on using poi 3.11 API.
Hi, In most cases this is caused by some outdated xml-apis.jar or stax-apis.jar, or some very old instance of Xerces or other XML parsers in classpath. As Java 6+ has STAX APIs bundled, putting them into the classpath or the ext/ directory of your JVM breaks things. Make sure that you clean your classpath up! If you use Maven, exclude the useless dependencies on xml-apis.jar or stax-apis.jar with an exclude. Also make sure you use latest versions of XML parsers. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Nick Burch [mailto:apa...@gagravarr.org] > Sent: Thursday, January 08, 2015 4:11 PM > To: POI Developers List > Subject: Re: getting exception on XMLEventFactory.newFatory on using poi > 3.11 API. > > On Thu, 8 Jan 2015, Banerjee, Tapas wrote: > > We are using Apache SXSSF to export Excel. We had some issue in > > exporting large Excel, so we moved to 3.11. The API works fine in my > > PC, however when we deploy to our Unix server - we are getting below > > exception > > > > [1/8/15 9:29:06:111 EST] 00d9 ServletWrappe E > > com.ibm.ws.webcontainer.servle t.ServletWrapper service Uncaught > > service() exception thrown by servlet ExportExcelServlet: > > "java.lang.NoSuchMethodError: > javax/xml/stream/XMLEventFactory.newFac > > tory()Ljavax/xml/stream/XMLEventFactory;" > > Looks like you have a broken / beta JVM. That method should be present on > a spec-compliant Java 6 JVM, see the java 6 javadocs: > http://docs.oracle.com/javase/6/docs/api/javax/xml/stream/XMLEventFact > ory.html#newFactory%28%29 > > > What is the minimum java version we need to run 3.11? > > Spec compliant java 1.6, or newer > > Nick > > - > To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional > commands, e-mail: dev-h...@poi.apache.org - To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional commands, e-mail: dev-h...@poi.apache.org
[ANNOUNCE] Apache POI 3.11-beta2 released
The Apache POI project is pleased to announce the release of POI 3.11-beta2. Featured are a handful of new areas of functionality, and numerous bug fixes. See the downloads page for binary and source distributions: http://poi.apache.org/download.html Note: The Apache Software Foundation uses an extensive mirroring network for distributing releases. It is possible that the mirror you are using may not have replicated the release yet. If that is the case, please try another mirror. This also goes for Maven access. Release Notes Changes This release fixes two security issues with OOXML: - Tidy up the OPC SAX setup code with a new common Helper, preventing external entity expansion (CVE-2014-3529). - On supported XML parser versions (Xerces or JVM built-in, XMLBeans 2.6), enforce sensible limits on entity expansion in OOXML files, and ensure that subsequent normal files still pass fine (CVE-2014-3574). Please note: You should use xmlbeans-2.6.jar (as shipped with this release) instead of the xmlbeans-2.3.jar version from the 3.10-FINAL release to work around CVE-2014-3574. If you have an alternate XML parser like Apache Xerces in classpath, be sure to use a recent version! Older versions are likely to break on setting required security features. Thanks to Stefan Kopf, Mike Boufford, Mohamed Ramadan, and Christian Schneider for reporting these issues! Other notable changes in this release are: - Switch from dom4j to JAXP: dom4j is no longer a dependency of Apache POI, it solely uses xmlbeans-2.6.jar and the DOM implementation as shipped by the JDK. - For XSLF Pictures, provide a way to get the URI of externally linked pictures - Provide a helpful exception, XLSBUnsupportedException, if XSSFWorkbook is passed a .xlsb file A full list of changes is available in the change log: http://poi.apache.org/changes.html. People interested should also follow the dev mailing list to track further progress. Release Contents This release comes in two forms: - pre-built binaries containing compiled versions of all Apache POI components and documentation (poi-bin-3.11-beta2-20140822.zip or poi-bin-3.11-beta2-20140822.tar.gz) - source archive you can build POI from (poi-src-3.11-beta2-20140822.zip or poi-src-3.11-beta2-20140822.tar.gz) Unpack the archive and use the following command to build all POI components with Apache Ant 1.6+ and JDK 1.6 or higher: ant jar Pre-built versions of all POI components are also available in the central Maven repository under Group ID "org.apache.poi" and Version "3.11-beta2" All release artifacts are accompanied by MD5 checksums and a PGP signatures that you can use to verify the authenticity of your download. The public key used for the PGP signature can be found at http://svn.apache.org/repos/asf/poi/tags/REL_3_11_BETA2/KEYS About Apache POI --- Apache POI is well-known in the Java field as a library for reading and writing Microsoft Office file formats, such as Excel, PowerPoint, Word, Visio, Publisher and Outlook. It supports both the older (OLE2) and new (OOXML - Office Open XML) formats. See http://poi.apache.org/ for more details On behalf of the Apache POI PMC, Uwe - Uwe Schindler uschind...@apache.org Apache Lucene PMC Member / Committer Bremen, Germany http://lucene.apache.org/ - To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional commands, e-mail: dev-h...@poi.apache.org
RE: [VOTE] Apache POI 3.11-beta2 release
Hi, The vote has passed after 72 hrs. We got three +1 and one +/-0. In my original mail I stated that the vote is open for longer time till this evening, but that was just to give more time to the members to vote (because it took much longer for the bugfix 3.10.1). I will now start to push the Maven artifacts, to make the Apache TIKA committers happy. I may announce the release tomorrow (although the date in the ZIP files is Aug 22) or should I better wait till Friday (Aug 22). Any suggestions? Thanks to all for voting! Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Uwe Schindler [mailto:u...@thetaphi.de] > Sent: Wednesday, August 20, 2014 9:38 AM > To: 'POI Developers List' > Cc: d...@tika.apache.org > Subject: RE: [VOTE] Apache POI 3.11-beta2 release > > Hi, > > I announced a bit more time for the vote, because it took a bit longer last > time (POI v3.10.1). The "official" 72 hrs are ending today at 12:35 UTC; we > already have enough positive votes. Because TIKA is waiting for building a > release candidate, I will start publishing Maven Artifacts today at this time. > > If somebody wants to give additional votes, he should do now :-) > > Uwe > > - > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > > > -Original Message- > > From: Uwe Schindler [mailto:u...@thetaphi.de] > > Sent: Sunday, August 17, 2014 2:35 PM > > To: dev@poi.apache.org > > Subject: [VOTE] Apache POI 3.11-beta2 release > > > > Hi, > > > > I prepared artifacts for the release of Apache POI 3.11-beta2. The > > most notable changes in this release are: > > > > - Switch from dom4j to JAXP: The dom4j.jar is no longer needed > > - Shipped version of xmlbeans.jar dependency updated to v2.6 > > - For XSLF Pictures, provide a way to get the URI of externally > > linked pictures > > - Provide a helpful exception, XLSBUnsupportedException, if > > XSSFWorkbook is passed a .xlsb file > > > > https://dist.apache.org/repos/dist/dev/poi/3.11-beta2-RC1/ > > > > Please vote to release the artifacts. > > > > I tried the released source and binary artifact, looks good to me. All > > tests pass. ASC files verify and MD5/SHA1 are correct. Docs look fine. > > > > Here is my +1 to release the above artifacts and deploy to Maven > > Central, Uwe > > > > P.S.: The vote keeps open for 72hrs, 2014-08-21, 0:00 UTC, planned > > release announcement date is Friday, 2014-08-22. > > > > - > > Uwe Schindler > > H.-H.-Meier-Allee 63, D-28213 Bremen > > http://www.thetaphi.de > > eMail: u...@thetaphi.de > > > > > > > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional > > commands, e-mail: dev-h...@poi.apache.org > > > - > To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional > commands, e-mail: dev-h...@poi.apache.org - To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional commands, e-mail: dev-h...@poi.apache.org
RE: [VOTE] Apache POI 3.11-beta2 release
Hi, I announced a bit more time for the vote, because it took a bit longer last time (POI v3.10.1). The "official" 72 hrs are ending today at 12:35 UTC; we already have enough positive votes. Because TIKA is waiting for building a release candidate, I will start publishing Maven Artifacts today at this time. If somebody wants to give additional votes, he should do now :-) Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message----- > From: Uwe Schindler [mailto:u...@thetaphi.de] > Sent: Sunday, August 17, 2014 2:35 PM > To: dev@poi.apache.org > Subject: [VOTE] Apache POI 3.11-beta2 release > > Hi, > > I prepared artifacts for the release of Apache POI 3.11-beta2. The most > notable changes in this release are: > > - Switch from dom4j to JAXP: The dom4j.jar is no longer needed > - Shipped version of xmlbeans.jar dependency updated to v2.6 > - For XSLF Pictures, provide a way to get the URI of externally linked > pictures > - Provide a helpful exception, XLSBUnsupportedException, if XSSFWorkbook > is passed a .xlsb file > > https://dist.apache.org/repos/dist/dev/poi/3.11-beta2-RC1/ > > Please vote to release the artifacts. > > I tried the released source and binary artifact, looks good to me. All tests > pass. ASC files verify and MD5/SHA1 are correct. Docs look fine. > > Here is my +1 to release the above artifacts and deploy to Maven Central, > Uwe > > P.S.: The vote keeps open for 72hrs, 2014-08-21, 0:00 UTC, planned release > announcement date is Friday, 2014-08-22. > > - > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > > > > - > To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional > commands, e-mail: dev-h...@poi.apache.org - To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional commands, e-mail: dev-h...@poi.apache.org
[ANNOUNCE] Apache POI 3.10.1 released
The Apache POI project is pleased to announce the release of POI 3.10.1-20140818. This release is a bugfix release to fix two security issues with OOXML. See the downloads page for binary and source distributions: http://poi.apache.org/download.html Note: The Apache Software Foundation uses an extensive mirroring network for distributing releases. It is possible that the mirror you are using may not have replicated the release yet. If that is the case, please try another mirror. This also goes for Maven access. Release Notes Changes The most notable changes in this release are: This release is a bugfix release to fix two security issues with OOXML: - Tidy up the OPC SAX setup code with a new common Helper, preventing external entity expansion (CVE-2014-3529). - On supported XML parser versions (Xerces or JVM built-in, XMLBeans 2.6), enforce sensible limits on entity expansion in OOXML files, and ensure that subsequent normal files still pass fine (CVE-2014-3574). Please note: You should use xmlbeans-2.6.jar (as shipped with this release) instead of the xmlbeans-2.3.jar version from the 3.10-FINAL release to work around CVE-2014-3574. If you have an alternate XML parser like Apache Xerces in classpath, be sure to use a recent version! Older versions are likely to break on setting required security features. Thanks to Stefan Kopf, Mike Boufford, and Christian Schneider for reporting these issues! A full list of changes is available in the change log: http://poi.apache.org/changes.html. People interested should also follow the dev mailing list to track further progress. Release Contents This release comes in two forms: - pre-built binaries containing compiled versions of all Apache POI components and documentation (poi-bin-3.10.1-20140818.zip or poi-bin-3.10.1-20140818.tar.gz) - source archive you can build POI from (poi-src-3.10.1-20140818.zip or poi-src-3.10.1-20140818.tar.gz) Unpack the archive and use the following command to build all POI components with Apache Ant 1.6+ and JDK 1.5 or higher: ant jar Pre-built versions of all POI components are also available in the central Maven repository under Group ID "org.apache.poi" and Version "3.10.1-20140818" All release artifacts are accompanied by MD5 checksums and a PGP signatures that you can use to verify the authenticity of your download. The public key used for the PGP signature can be found at http://svn.apache.org/repos/asf/poi/tags/REL_3_10_1/KEYS About Apache POI --- Apache POI is well-known in the Java field as a library for reading and writing Microsoft Office file formats, such as Excel, PowerPoint, Visio and Word. Since POI 3.5, the new OOXML (Office Open XML) formats introduced in Office 2007 have been supported. See http://poi.apache.org/ for more details On behalf of the Apache POI PMC, Uwe - Uwe Schindler uschind...@apache.org Apache Lucene PMC Member / Committer Apache POI PMC Member / Committer Bremen, Germany http://lucene.apache.org/ - To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional commands, e-mail: dev-h...@poi.apache.org
RE: [VOTE] Apache POI 3.11-beta2 release
Hi, I will add a line to the 3.10.1 release notes in a moment, which is already on Maven Central and most mirrors. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: David kerber [mailto:dcker...@verizon.net] > Sent: Monday, August 18, 2014 6:27 PM > To: POI Developers List > Subject: Re: [VOTE] Apache POI 3.11-beta2 release > > I would vote to stick to your guns on the pre-requisites, and let it fail if > the > user's environment doesn't meet the requirements. > > Maybe put something in the release notes about this so they know what's > going on when they hit this issue. > > > > On 8/18/2014 12:18 PM, Uwe Schindler wrote: > > The question to the others: > > > > This is a dependency problem and not POI's fault. We can provide a > "workaround" (which introduces a security issue on those broken platforms) > - this is why I raised to warning level when adding the workaround. > > I don't think this should hold a beta2 release, XERCES 2.6.1 is 10 (!!!) > > years > old and was released before Java 5, which added > DoucmentBuilderFactory#setFeature(). > > > > Uwe > > > > - > > Uwe Schindler > > H.-H.-Meier-Allee 63, D-28213 Bremen > > http://www.thetaphi.de > > eMail: u...@thetaphi.de > > > > > >> -Original Message- > >> From: Uwe Schindler [mailto:u...@thetaphi.de] > >> Sent: Monday, August 18, 2014 6:08 PM > >> To: 'POI Developers List' > >> Subject: RE: [VOTE] Apache POI 3.11-beta2 release > >> > >> H Dominik, > >> > >> I committed the suggested fix (to both poi and poi-ooxml): > >> > >> http://svn.apache.org/r1618644 > >> > >> Please note: I raised the logging level on failure to "warning", because > >> you > >> make your XML parsing vulnerable to CVE-2014-3574 and CVE-2014-3529 ! > >> > >> POI 3.10.1 should have same issue, but its less severe there, because > >> DocumentHelper is only used for Excel Import/Export in OOXML, not for > >> openxml DOMs. > >> Uwe > >> > >> - > >> Uwe Schindler > >> H.-H.-Meier-Allee 63, D-28213 Bremen > >> http://www.thetaphi.de > >> eMail: u...@thetaphi.de > >> > >> > >>> -Original Message- > >>> From: Dominik Stadler [mailto:dominik.stad...@gmx.at] > >>> Sent: Monday, August 18, 2014 4:09 PM > >>> To: POI Developers List > >>> Subject: Re: [VOTE] Apache POI 3.11-beta2 release > >>> > >>> I agree that it the lib is outdated, but in my case it is pulled in by > >>> some other dependency down the tree, being a large project, it is hard > >>> to update the Xerces dependency without causing more work to update > >>> other dependencies that are not related to POI, thus making a simple > >>> update of POI rather complicated. > >>> > >>> These tests ran fine with POI 3.10 and 3.11-beta1, so we are > >>> introducing this incompatibility with -beta2. A fix is easy, just > >>> catch the AbstractMethodError in that place the same way that we > already > >> catch Exception. > >>> > >>> So my vote is now 0, I do not vote against it, but think we should do > >>> this change for 3.11 final. > >>> > >>> Dominik. > >>> > >>> > >>> On Mon, Aug 18, 2014 at 3:03 PM, Uwe Schindler > >> wrote: > >>>> Hi, > >>>> > >>>> this old Xerces version is not compliant to Java 6 as required as > >>>> minimum > >>> JVM. Since Java 1.4, the JDK requires setFeature() to be available. > >>>> > >>>> The problem you have is: Something is inserting an older version of > >>>> xml- > >>> apis.jar into the classpath or the lib/ext folder of your JDK, that > >>> breaks java 1.4+. > >>>> > >>>> This will also happen with the bug fix release 3.10.1. There is > >>>> nothing we > >>> can do; upgrade to newer XERCES, which is compliant to newer Java > >> versions. > >>>> > >>>> Uwe > >>>> > >>>> - > >>>> Uwe Schindler > >>>> H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de > >>>> eMail: u...@t
RE: [VOTE] Apache POI 3.11-beta2 release
The question to the others: This is a dependency problem and not POI's fault. We can provide a "workaround" (which introduces a security issue on those broken platforms) - this is why I raised to warning level when adding the workaround. I don't think this should hold a beta2 release, XERCES 2.6.1 is 10 (!!!) years old and was released before Java 5, which added DoucmentBuilderFactory#setFeature(). Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Uwe Schindler [mailto:u...@thetaphi.de] > Sent: Monday, August 18, 2014 6:08 PM > To: 'POI Developers List' > Subject: RE: [VOTE] Apache POI 3.11-beta2 release > > H Dominik, > > I committed the suggested fix (to both poi and poi-ooxml): > > http://svn.apache.org/r1618644 > > Please note: I raised the logging level on failure to "warning", because you > make your XML parsing vulnerable to CVE-2014-3574 and CVE-2014-3529 ! > > POI 3.10.1 should have same issue, but its less severe there, because > DocumentHelper is only used for Excel Import/Export in OOXML, not for > openxml DOMs. > Uwe > > - > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > > > -Original Message- > > From: Dominik Stadler [mailto:dominik.stad...@gmx.at] > > Sent: Monday, August 18, 2014 4:09 PM > > To: POI Developers List > > Subject: Re: [VOTE] Apache POI 3.11-beta2 release > > > > I agree that it the lib is outdated, but in my case it is pulled in by > > some other dependency down the tree, being a large project, it is hard > > to update the Xerces dependency without causing more work to update > > other dependencies that are not related to POI, thus making a simple > > update of POI rather complicated. > > > > These tests ran fine with POI 3.10 and 3.11-beta1, so we are > > introducing this incompatibility with -beta2. A fix is easy, just > > catch the AbstractMethodError in that place the same way that we already > catch Exception. > > > > So my vote is now 0, I do not vote against it, but think we should do > > this change for 3.11 final. > > > > Dominik. > > > > > > On Mon, Aug 18, 2014 at 3:03 PM, Uwe Schindler > wrote: > > > Hi, > > > > > > this old Xerces version is not compliant to Java 6 as required as > > > minimum > > JVM. Since Java 1.4, the JDK requires setFeature() to be available. > > > > > > The problem you have is: Something is inserting an older version of > > > xml- > > apis.jar into the classpath or the lib/ext folder of your JDK, that > > breaks java 1.4+. > > > > > > This will also happen with the bug fix release 3.10.1. There is > > > nothing we > > can do; upgrade to newer XERCES, which is compliant to newer Java > versions. > > > > > > Uwe > > > > > > - > > > Uwe Schindler > > > H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de > > > eMail: u...@thetaphi.de > > > > > > > > >> -Original Message- > > >> From: Dominik Stadler [mailto:dominik.stad...@gmx.at] > > >> Sent: Monday, August 18, 2014 2:52 PM > > >> To: POI Developers List > > >> Subject: Re: [VOTE] Apache POI 3.11-beta2 release > > >> > > >> Hi, > > >> > > >> I get the following, which looks like the change to remove dom4j is > > >> not fully working yet for some versions of Xerces XML Parser: > > >> > > >> java.lang.AbstractMethodError: > > >> > > > javax.xml.parsers.DocumentBuilderFactory.setFeature(Ljava/lang/String;Z)V > > >> at > > >> > > > org.apache.poi.util.DocumentHelper.trySetSAXFeature(DocumentHelper.ja > > >> v > > >> a:62) > > >> at > > org.apache.poi.util.DocumentHelper.(DocumentHelper.java:56) > > >> at > > >> org.apache.poi.openxml4j.opc.internal.marshallers.ZipPartMarshaller.m > > >> arsh > > >> allRelationshipPart(ZipPartMarshaller.java:120) > > >> at > > >> > org.apache.poi.openxml4j.opc.ZipPackage.saveImpl(ZipPackage.java:464) > > >> at > > >> > org.apache.poi.openxml4j.opc.OPCPackage.save(OPCPackage.java:1425) > > >> at > org.apache.poi.POIXMLDocument.write(POIXMLDocument.java:201) > > >> at > > >> > com.xxx.diagnostics.repo
RE: [VOTE] Apache POI 3.11-beta2 release
H Dominik, I committed the suggested fix (to both poi and poi-ooxml): http://svn.apache.org/r1618644 Please note: I raised the logging level on failure to "warning", because you make your XML parsing vulnerable to CVE-2014-3574 and CVE-2014-3529 ! POI 3.10.1 should have same issue, but its less severe there, because DocumentHelper is only used for Excel Import/Export in OOXML, not for openxml DOMs. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Dominik Stadler [mailto:dominik.stad...@gmx.at] > Sent: Monday, August 18, 2014 4:09 PM > To: POI Developers List > Subject: Re: [VOTE] Apache POI 3.11-beta2 release > > I agree that it the lib is outdated, but in my case it is pulled in by some > other > dependency down the tree, being a large project, it is hard to update the > Xerces dependency without causing more work to update other > dependencies that are not related to POI, thus making a simple update of > POI rather complicated. > > These tests ran fine with POI 3.10 and 3.11-beta1, so we are introducing this > incompatibility with -beta2. A fix is easy, just catch the AbstractMethodError > in that place the same way that we already catch Exception. > > So my vote is now 0, I do not vote against it, but think we should do this > change for 3.11 final. > > Dominik. > > > On Mon, Aug 18, 2014 at 3:03 PM, Uwe Schindler wrote: > > Hi, > > > > this old Xerces version is not compliant to Java 6 as required as minimum > JVM. Since Java 1.4, the JDK requires setFeature() to be available. > > > > The problem you have is: Something is inserting an older version of xml- > apis.jar into the classpath or the lib/ext folder of your JDK, that breaks > java > 1.4+. > > > > This will also happen with the bug fix release 3.10.1. There is nothing we > can do; upgrade to newer XERCES, which is compliant to newer Java versions. > > > > Uwe > > > > - > > Uwe Schindler > > H.-H.-Meier-Allee 63, D-28213 Bremen > > http://www.thetaphi.de > > eMail: u...@thetaphi.de > > > > > >> -Original Message- > >> From: Dominik Stadler [mailto:dominik.stad...@gmx.at] > >> Sent: Monday, August 18, 2014 2:52 PM > >> To: POI Developers List > >> Subject: Re: [VOTE] Apache POI 3.11-beta2 release > >> > >> Hi, > >> > >> I get the following, which looks like the change to remove dom4j is > >> not fully working yet for some versions of Xerces XML Parser: > >> > >> java.lang.AbstractMethodError: > >> > javax.xml.parsers.DocumentBuilderFactory.setFeature(Ljava/lang/String;Z)V > >> at > >> > org.apache.poi.util.DocumentHelper.trySetSAXFeature(DocumentHelper.ja > >> v > >> a:62) > >> at > org.apache.poi.util.DocumentHelper.(DocumentHelper.java:56) > >> at > >> org.apache.poi.openxml4j.opc.internal.marshallers.ZipPartMarshaller.m > >> arsh > >> allRelationshipPart(ZipPartMarshaller.java:120) > >> at > >> org.apache.poi.openxml4j.opc.ZipPackage.saveImpl(ZipPackage.java:464) > >> at > >> org.apache.poi.openxml4j.opc.OPCPackage.save(OPCPackage.java:1425) > >> at org.apache.poi.POIXMLDocument.write(POIXMLDocument.java:201) > >> at > >> com.xxx.diagnostics.report.excel.ExcelRenderer.reportDashboard(ExcelR > >> ep > >> ortRenderer.java:99) > >> at > >> com.xxx.diagnostics.report.excel.ExcelRendererTest.testReportDashboar > >> dW > >> ithTooManyTableRowsXLSX(ExcelReportRendererTest.java:2268) > >> > >> This is a larger set of tests with some POI-related tests, due to > >> other dependencies an older version of Xerces XML Parser is pulled: > >> > >> documentBuilderFactory is a > >> org.apache.xerces.jaxp.DocumentBuilderFactoryImpl and not a > >> javax.xml.parsers.DocumentBuilderFactory which is provided with Java > itself. > >> > >> Test-Case is simply: > >> > >> @Test > >> public void testCrash() throws IOException { > >> System.out.println("Java: " + > >> System.getProperty("java.version")); > >> > >> try (Workbook wb = new XSSFWorkbook()) { > >> FileOutputStream out = new FileOutputStream(new > >> File("C:\\temp\\test.xlsx")); > >> try { > >> wb.write(out); > >> } finally { > >>
RE: [VOTE] Apache POI 3.11-beta2 release
Hi, this old Xerces version is not compliant to Java 6 as required as minimum JVM. Since Java 1.4, the JDK requires setFeature() to be available. The problem you have is: Something is inserting an older version of xml-apis.jar into the classpath or the lib/ext folder of your JDK, that breaks java 1.4+. This will also happen with the bug fix release 3.10.1. There is nothing we can do; upgrade to newer XERCES, which is compliant to newer Java versions. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Dominik Stadler [mailto:dominik.stad...@gmx.at] > Sent: Monday, August 18, 2014 2:52 PM > To: POI Developers List > Subject: Re: [VOTE] Apache POI 3.11-beta2 release > > Hi, > > I get the following, which looks like the change to remove dom4j is not fully > working yet for some versions of Xerces XML Parser: > > java.lang.AbstractMethodError: > javax.xml.parsers.DocumentBuilderFactory.setFeature(Ljava/lang/String;Z)V > at > org.apache.poi.util.DocumentHelper.trySetSAXFeature(DocumentHelper.jav > a:62) > at org.apache.poi.util.DocumentHelper.(DocumentHelper.java:56) > at > org.apache.poi.openxml4j.opc.internal.marshallers.ZipPartMarshaller.marsh > allRelationshipPart(ZipPartMarshaller.java:120) > at > org.apache.poi.openxml4j.opc.ZipPackage.saveImpl(ZipPackage.java:464) > at > org.apache.poi.openxml4j.opc.OPCPackage.save(OPCPackage.java:1425) > at org.apache.poi.POIXMLDocument.write(POIXMLDocument.java:201) > at > com.xxx.diagnostics.report.excel.ExcelRenderer.reportDashboard(ExcelRep > ortRenderer.java:99) > at > com.xxx.diagnostics.report.excel.ExcelRendererTest.testReportDashboardW > ithTooManyTableRowsXLSX(ExcelReportRendererTest.java:2268) > > This is a larger set of tests with some POI-related tests, due to other > dependencies an older version of Xerces XML Parser is pulled: > > documentBuilderFactory is a > org.apache.xerces.jaxp.DocumentBuilderFactoryImpl and not a > javax.xml.parsers.DocumentBuilderFactory which is provided with Java itself. > > Test-Case is simply: > > @Test > public void testCrash() throws IOException { > System.out.println("Java: " + System.getProperty("java.version")); > > try (Workbook wb = new XSSFWorkbook()) { > FileOutputStream out = new FileOutputStream(new > File("C:\\temp\\test.xlsx")); > try { > wb.write(out); > } finally { > out.close(); > } > } > } > > > At least xerces-2.6.1 is not providing the "setFeature()" method, > xerces-2.11 and 2.9.1 seem to have it, I did not check intermediate versions. > > I vote that we avoid this crash by either also catching the > AbstractMethodError or not calling that method on older versions of Xerces > that do not yet have "setFeature". Customers will run POI in all sorts of > environments and thus it is likely that older versions of Xerces are still > present in a number of them. > > Thus -1 from me unless it can be explained as being a local problem in my > environment. > > Dominik. > > On Sun, Aug 17, 2014 at 11:45 PM, Andreas Beeker > wrote: > > +1 from my side > > > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional > > commands, e-mail: dev-h...@poi.apache.org > > > > - > To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional > commands, e-mail: dev-h...@poi.apache.org - To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional commands, e-mail: dev-h...@poi.apache.org
RE: [VOTE] Apache POI 3.10.1 release
Hi, Thanks to all, this vote has passed. I will publish the artifacts and Maven now. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Nick Burch [mailto:n...@apache.org] > Sent: Sunday, August 17, 2014 4:20 PM > To: POI Developers List > Subject: Re: [VOTE] Apache POI 3.10.1 release > > On 14/08/14 01:07, Uwe Schindler wrote: > > I prepared artifacts for the release of Apache POI 3.10.1, a bugfix release > > of > the 3.10 branch. It contains 2 important fixes about XML entity handling in > OOXML parsers. Also the recommended (and shipped) version of > xmlbeans.jar was upgraded from 2.3 to 2.6. > > > > https://dist.apache.org/repos/dist/dev/poi/3.10.1-RC2/ > > +1 from myself on this release > > Nick > > - > To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional > commands, e-mail: dev-h...@poi.apache.org - To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional commands, e-mail: dev-h...@poi.apache.org
[VOTE] Apache POI 3.11-beta2 release
Hi, I prepared artifacts for the release of Apache POI 3.11-beta2. The most notable changes in this release are: - Switch from dom4j to JAXP: The dom4j.jar is no longer needed - Shipped version of xmlbeans.jar dependency updated to v2.6 - For XSLF Pictures, provide a way to get the URI of externally linked pictures - Provide a helpful exception, XLSBUnsupportedException, if XSSFWorkbook is passed a .xlsb file https://dist.apache.org/repos/dist/dev/poi/3.11-beta2-RC1/ Please vote to release the artifacts. I tried the released source and binary artifact, looks good to me. All tests pass. ASC files verify and MD5/SHA1 are correct. Docs look fine. Here is my +1 to release the above artifacts and deploy to Maven Central, Uwe P.S.: The vote keeps open for 72hrs, 2014-08-21, 0:00 UTC, planned release announcement date is Friday, 2014-08-22. - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de - To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional commands, e-mail: dev-h...@poi.apache.org
RE: [VOTE] Apache POI 3.10.1 release
Hi, Just a friendly reminder: Please check and test the release artifacts! Many thanks, Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Uwe Schindler [mailto:u...@thetaphi.de] > Sent: Thursday, August 14, 2014 2:08 AM > To: dev@poi.apache.org; priv...@poi.apache.org > Subject: [VOTE] Apache POI 3.10.1 release > > Hi, > > I prepared artifacts for the release of Apache POI 3.10.1, a bugfix release of > the 3.10 branch. It contains 2 important fixes about XML entity handling in > OOXML parsers. Also the recommended (and shipped) version of > xmlbeans.jar was upgraded from 2.3 to 2.6. > > https://dist.apache.org/repos/dist/dev/poi/3.10.1-RC2/ > > Please vote to release the artifacts. Please take especially care, that they > can > work as drop-in replacements for 3.10-FINAL. > > I tried the release, looks good to me. All tests pass. ASC files verify and > MD5/SHA1 are correct. Docs look fine. > > I did a test with Solr 4.9: I replaced the JAR files of POI-3.10-beta2 > (shipped > with Solr 4.9) by those new ones (drop-in replacement), which worked > without any problems. The entity expansion fixes are working correctly! > While doing those test with my first build (RC1), I recognized that there were > more XML parsers used, not respecting the new sensible defaults used for > parsing, so I committed some more fixes and built a new RC (RC2). > > Here is my +1 to release the above artifacts and deploy to Maven Central, > Uwe > > P.S.: The vote keeps open for 72hrs, 2014-08-17, 0:00 UTC, planned release > announcement date is Monday, 2014-08-18. > > - > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > > > > - > To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional > commands, e-mail: dev-h...@poi.apache.org - To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional commands, e-mail: dev-h...@poi.apache.org
[VOTE] Apache POI 3.10.1 release
Hi, I prepared artifacts for the release of Apache POI 3.10.1, a bugfix release of the 3.10 branch. It contains 2 important fixes about XML entity handling in OOXML parsers. Also the recommended (and shipped) version of xmlbeans.jar was upgraded from 2.3 to 2.6. https://dist.apache.org/repos/dist/dev/poi/3.10.1-RC2/ Please vote to release the artifacts. Please take especially care, that they can work as drop-in replacements for 3.10-FINAL. I tried the release, looks good to me. All tests pass. ASC files verify and MD5/SHA1 are correct. Docs look fine. I did a test with Solr 4.9: I replaced the JAR files of POI-3.10-beta2 (shipped with Solr 4.9) by those new ones (drop-in replacement), which worked without any problems. The entity expansion fixes are working correctly! While doing those test with my first build (RC1), I recognized that there were more XML parsers used, not respecting the new sensible defaults used for parsing, so I committed some more fixes and built a new RC (RC2). Here is my +1 to release the above artifacts and deploy to Maven Central, Uwe P.S.: The vote keeps open for 72hrs, 2014-08-17, 0:00 UTC, planned release announcement date is Monday, 2014-08-18. - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de - To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional commands, e-mail: dev-h...@poi.apache.org
Final Call: Release of 3.11-beta2
Hi, I would like to push a release for POI 3.11-beta2, because we have some important changes missing in beta1. Please commit your changes in the next 24 to 36 hours, so I can start with the release process in 2 days. Thanks, Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de - To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org For additional commands, e-mail: dev-h...@poi.apache.org