Thanks Piotr for raising this and summing it up so far. The timing of deprecating is always hard, but it looks like there is a lot of traction within the Java data ecosystem to move to a later version:
- Avro 1.12.0 will be JDK17+ - Spark 4.x will be JDK17+ - Arrow 18 will be JDK11+ - Parquet-Java has the same discussion right now. Next to Hive, we also need to compile against a later version of Hadoop. Currently, it compiles against Hadoop 2.7.3 <https://github.com/apache/iceberg/blob/main/gradle/libs.versions.toml#L47>, which we need to drop, and go straight to Hadoop 3.3. Regarding the deprecation, how would this look like? I think the version just won't be available for a certain Hive version, similar to Spark 3.2 where it stops at Iceberg 1.4.x <https://mvnrepository.com/artifact/org.apache.iceberg/iceberg-spark-runtime-3.2> . It has a caveat (we can't run formatter on 21 and 8, and we need to choose > one). Would it format differently? I would go for 21 since that's the path forward, but I'm also fine with JB's suggestion 👍 Kind regards, Fokko Op wo 10 jul 2024 om 10:19 schreef Jean-Baptiste Onofré <j...@nanthrax.net>: > Hi Piotr, > > Even if it might fail at the beginning, I think we can start Java21 on > a profile/CI even if we still use Java8 by default. > So, agree to: > 1. Even Java21 in a profile without formatter for now), the default is > still Java8 > 2. When Java8 is dropped and we enable Java21 by default, we also > switch the formatter. > > Regards > JB > > On Wed, Jul 10, 2024 at 10:14 AM Piotr Findeisen > <piotr.findei...@gmail.com> wrote: > > > > Hi, > > > > Thank you all for your comments and perspectives. > > > > Summing up so far: > > It is clear that dropping JDK 8 is imminent, but it is also inevitably > painful for some users. > > We don't have precise date/version when Java 8 can be dropped (along > with Hive module) and Iceberg 1.7 was proposed for that. > > > > We have consensus that after we drop Java 8, we can add Java 21 support > and testing. This is awesome. > > > > What we didn't focus on yet, is this question: > > can we add Java 21 support and testing before we drop Java 8? > > It has a caveat (we can't run formatter on 21 and 8, and we need to > choose one). > > I am proposing this because it seems to me an incremental improvement > over the current state. > > Increasing reliability or releases for those who already moved ahead and > preventing any potential regressions/ > > What do you think? > > > > Best, > > Piotr > > > > > > > > > > > > > > On Wed, 10 Jul 2024 at 10:09, Jean-Baptiste Onofré <j...@nanthrax.net> > wrote: > >> > >> Hi Piotr > >> > >> I already commented on the PR directly, so let me share here: I'm in > >> favor of dropping Java8 and directly jumping to Java21. > >> However, to do this jump, I would need to remove "old" modules, like > >> Hive. I think it's totally acceptable on a new major version. I shared > >> this in the thread about Iceberg Java 2.0, but we didn't have a lot of > >> discussions about it. > >> > >> So, I would suggest: > >> Step 0. Agree in which branch/major version we plan this > >> Step 1. Remove old modules requiring Java8 (hive) > >> Step 2. Upgrade to JDK21 (build + CI) and Java Format > >> > >> Regards > >> JB > >> > >> On Tue, Jul 9, 2024 at 2:31 PM Piotr Findeisen > >> <piotr.findei...@gmail.com> wrote: > >> > > >> > Hi, > >> > > >> > Java 21 is the latest "LTS version" released GA in September 2023. > >> > Some Iceberg users already run with Java 21 on production (and FWIW > Trino runs with 22 already) > >> > I thought it would be nice to add support for building and testing > Iceberg with Java 21. > >> > > >> > Conceptually this is simple (see PR), but there is a caveat worth > discussing: > >> > There seems to be no version of Google Java Format library that can > run under JDK 8 and JDK 21. > >> > Choosing Google Java Format version dynamically is not an option, > because different versions have slightly different formatting preferences, > so updating formatter version requires updating the code in a handful of > places. > >> > > >> > Question: > >> > do we want to add support for building and testing with Java 21? > >> > Ability to test with Java 21 would match what some of Iceberg users > are doing. > >> > If we choose so, we would simply disable spotless formatter when > build runs on Java 21 (or 8 if this is preferred instead) > >> > > >> > or we prefer to wait until we can drop Java 8 support first, and only > then add Java 21 support? > >> > > >> > Pre-existing context: > >> > the topic has been discussed on the PR here: > https://github.com/apache/iceberg/pull/10474#discussion_r1658513019 > >> > and it was proposed there to bring this to Dev group attention. > >> > > >> > Best, > >> > PF > >> > >
