A move to java 11 means it is time to move to Hadoop 3.3.x as the minimum release, anything 17+ means java 3.4.x. Which before long will go making java 11 its minimum version.
That is: - cut the hadoop2 version/profile which is really java7. - be prepared to move to 3.4.x if some java11/17 incompatibility surfaces as any issues there will only be on branch-3.4 and trunk branches. Cutting the hadoop2 profile can only be good as it reduces the test matrix and means that if problems do surface they won't be rejected as WONTFIX + /upgrade There are still some bits outstanding in the hadoop "support java 17" JIRAs, but they should be all server-side and test issues, and *should* be nothing to worry about. https://issues.apache.org/jira/browse/HADOOP-17177 that is: filesystem clients should all be good. And the CI tooling is getting set up for java17 test runs https://github.com/apache/hadoop/pull/6914 Meanwhile the first java 23 issue has surfaced: Oracle filed a bug report warning that the whole Subject class is going to break so UserGroupInformation.doAs() will go. https://issues.apache.org/jira/browse/HADOOP-19212 That's going to be traumatic across everything. Reflection is going to be needed to use the successor classes/methods, but because the UGI calls are common in hive and any other service which support different users in the same process, getting adoption of the new operations will be tricking. steve Java21 would be interesting; their new memory On Thu, 11 Jul 2024 at 09:09, Gabor Kaszab <gaborkas...@apache.org> wrote: > Hey, > > About the part where we plan to drop the Hive module. This seems like a > thing that has already been discussed and decided, so I might have missed > that conversation. Would it make sense to involve Hive devs into this > decision, though, so that we can make sure there is no difficulty on Hive > side due to this? Mayba a vote (if we hadn't had it already)? > > Cheers, > Gabor > > > On Wed, Jul 10, 2024 at 5:34 PM Robert Stupp <sn...@snazy.de> wrote: > >> +1 on deprecating Java 8 in the next "catchable" Iceberg release. >> >> +1 on removing Java 8 in Iceberg 2.0 >> >> +1 on _building_ for Java 11 in Iceberg 2.0 >> >> +1 on _building_ with Java 17 in Iceberg 2.0 >> >> >> I'd recommend to _build_ for Java 11 wherever possible (`javac --release >> 11`), because many users are concerned about bumping 2 Java LTS versions (8 >> ---> 17). >> >> >> From our experience, it's fine to bump the JDK in CI to Java 17, that's >> the Java version that works for Spark 3.3..3.5 (with some JPMS options). >> I'm less concerned about Spark 4 requiring Java 17 (`--release 17` can be >> configured for Spark 4, leaving everything at a lower version). >> >> I'm quite concerned about Avro 1.12 requiring Java 17 - maybe we can >> stick w/ Avro 1.11.x. >> >> >> On 10.07.24 15:13, Piotr Findeisen wrote: >> >> Hi, >> >> Thanks for additional feedback. It sounds like we're OK enabling builds >> and testing with JDK 21 with the caveats that formatter is off. >> The non-negotiable condition is that CI still checks format. >> I think this is exactly what this PR >> <https://github.com/apache/iceberg/pull/10474> is doing, which means we >> maybe can merge it. One small step forward. >> >> BTW there is no risk of `./gradlew spotlessApply` producing different >> results when being executed under JDK 21. >> The spotless plugin needs to be fully disabled, and then this command >> will fail saying the target is not found. >> It would be even better to provide custom spotlessApply command stub that >> would produce an actionable error message. >> I will add something like this to the above PR. >> >> re dropping Java 8 >> we need to figure out exact deprecation path and I don't know yet the >> customs to drive this fully. >> But happy to help as much as I can nonetheless. >> >> >> Best >> Piotr >> >> >> >> >> >> >> On Wed, 10 Jul 2024 at 11:19, Eduard Tudenhöfner < >> etudenhoef...@apache.org> wrote: >> >>> >>>> It has a caveat (we can't run formatter on 21 and 8, and we need to >>>>> choose one). >>>> >>>> >>>> Would it format differently? I would go for 21 since that's the path >>>> forward, but I'm also fine with JB's suggestion 👍 >>>> >>>>> >>>>> >>> Yeah it would format differently, because the underlying >>> *google-java-format* picks a different formatting depending on the JDK >>> version and that's why we had to pin the *google-java-format* version >>> to the one that still supports JDK8. >>> >>> I think it's ok to disable the formatter when running with JDK 21 for >>> people that want to use that JDK but we need to make sure that CI checks >>> still run JDK < 21 in order to make sure code is properly formatted. >>> >>> The one risk I see is that it will be difficult to understand why >>> running the build with JDK 21 passes locally but fails on CI (where an >>> older JDK is used). >>> Maybe it's sufficient to mention in the docs that *./gradlew >>> spotlessApply* needs to be executed locally with an older JDK. >>> >>> Eduard >>> >> -- >> Robert Stupp >> @snazy >> >>
