Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
On Tue, Mar 28, 2023 at 1:56 PM Steve Loughran wrote: > Now that hadoop 3.3.5 is out, i want to propose something new > > we switch branch-3.3 and trunk to being java11 only > I agree that maintaining support for Java8 is becoming burdensome. Over in HBase, we maintain support for both Java8 and Hadoop2 :( We could roll-forward the code lines: branch from rel/3.3.5 as a new branch-3.4 and trunk version bumps to 3.5 -- assuming the compatibility guidelines permit dropping the JVM version on a minor release. If not, rel/3.3.5 branches to branch-4 and trunk changes accordingly. 1. java 11 has been out for years >2. oracle java 8 is no longer available under "premier support"; you >can't really get upgrades >https://www.oracle.com/java/technologies/java-se-support-roadmap.html >3. openJDK 8 releases != oracle ones, and things you compile with them >don't always link to oracle java 8 (some classes in java.nio have added >more overrides) >4. more and more libraries we want to upgrade to/bundle are java 11 only >5. moving to java 11 would cut our yetus build workload in half, and >line up for adding java 17 builds instead. > > > I know there are some outstanding issues still in > https://issues.apache.org/jira/browse/HADOOP-16795 -but are they blockers? > Could we just move to java11 and enhance at our leisure, once java8 is no > longer a concern. >
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
I would also vote for targeting 3.4 and have a long term version of Java there. On Tue, Mar 28, 2023 at 11:52 AM Igor Dvorzhak wrote: > +1 to re-focusing on 3.4 branch and upgrading it to Java 11/17, instead of > making potentially breaking changes to 3.3. > > On Tue, Mar 28, 2023 at 11:17 AM Chris Nauroth wrote: > >> In theory, I like the idea of setting aside Java 8. Unfortunately, I don't >> know that upgrading within the 3.3 line adheres to our binary >> compatibility >> policy [1]. I don't see specific discussion of the Java version there, but >> it states that you should be able to drop in minor upgrades and have >> existing apps keep working. Users might find it surprising if they try to >> upgrade a cluster that has JDK 8. >> >> There is also the question of impact on downstream projects [2]. We'd have >> to check plans with our consumers. >> >> What about the idea of shooting for a 3.4 release on JDK 11 (or even 17)? >> The downside is that we'd probably need to set boundaries on end of >> life/limited support for 3.2 and 3.3 to keep the workload manageable. >> >> [1] >> >> https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#Java_Binary_compatibility_for_end-user_applications_i.e._Apache_Hadoop_ABI >> [2] https://github.com/apache/spark/blob/v3.3.2/pom.xml#L109 >> >> Chris Nauroth >> >> >> On Tue, Mar 28, 2023 at 11:10 AM Ayush Saxena wrote: >> >> > > >> > > it's already hard to migrate from JDK8 why not retarget JDK17. >> > > >> > >> > +1, makes sense to me, sounds like a win-win situation to me, though >> there >> > would be some additional issues to chase now :) >> > >> > -Ayush >> > >> > >> > On Tue, 28 Mar 2023 at 23:29, Wei-Chiu Chuang >> wrote: >> > >> > > My random thoughts. Probably bad takes: >> > > >> > > There are projects experimenting with JDK17 now. >> > > JDK11 active support will end in 6 months. If it's already hard to >> > migrate >> > > from JDK8 why not retarget JDK17. >> > > >> > > On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena >> > wrote: >> > > >> > >> I know Jersey upgrade as a blocker. Some folks were chasing that last >> > >> year during 3.3.4 time, I don’t know where it is now, didn’t see then >> > >> what’s the problem there but I remember there was some intitial PR >> which >> > >> did it for HDFS atleast, so I never looked beyond that… >> > >> >> > >> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as >> > >> java-11 only branch may be, but that is something later to decide, >> once >> > we >> > >> get the code sorted… >> > >> >> > >> -Ayush >> > >> >> > >> > On 28-Mar-2023, at 9:16 PM, Steve Loughran >> > >> > >> wrote: >> > >> > >> > >> > well, how about we flip the switch and get on with it. >> > >> > >> > >> > slf4j seems happy on java11, >> > >> > >> > >> > side issue, anyone seen test failures on zulu1.8; somehow my test >> run >> > is >> > >> > failing and i'm trying to work out whether its a mismatch in >> command >> > >> > line/ide jvm versions, or the 3.3.5 JARs have been built with an >> > openjdk >> > >> > version which requires IntBuffer implements an overridden method >> > >> IntBuffer >> > >> > rewind(). >> > >> > >> > >> > java.lang.NoSuchMethodError: >> > >> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer; >> > >> > >> > >> > at >> > >> >> org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341) >> > >> > at >> > >> > >> > >> >> > >> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308) >> > >> > at >> org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257) >> > >> > at >> org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202) >> > >> > at java.io.DataInputStream.read(DataInputStream.java:149) >> > >> > >> > >> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani >> > wrote: >> > >> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2, >> > >> jersey 1 >> > >> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test >> > >> stability. >> > >> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran >> > >> > > >> >> wrote: >> > >> >>> Now that hadoop 3.3.5 is out, i want to propose something new >> > >> >>> we switch branch-3.3 and trunk to being java11 only >> > >> >>> 1. java 11 has been out for years >> > >> >>> 2. oracle java 8 is no longer available under "premier support"; >> you >> > >> >>> can't really get upgrades >> > >> >>> >> > https://www.oracle.com/java/technologies/java-se-support-roadmap.html >> > >> >>> 3. openJDK 8 releases != oracle ones, and things you compile with >> > them >> > >> >>> don't always link to oracle java 8 (some classes in java.nio have >> > >> >> added >> > >> >>> more overrides) >> > >> >>> 4. more and more libraries we want to upgrade to/bundle are java >> 11 >> > >> >> only >> > >> >>> 5. moving to java 11 would cut our yetus build workload in half, >> and >> > >> >>> line up for adding java 17 builds instead. >> > >> >>> I know there are some outstanding issues still in >> >
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
+1 to re-focusing on 3.4 branch and upgrading it to Java 11/17, instead of making potentially breaking changes to 3.3. On Tue, Mar 28, 2023 at 11:17 AM Chris Nauroth wrote: > In theory, I like the idea of setting aside Java 8. Unfortunately, I don't > know that upgrading within the 3.3 line adheres to our binary compatibility > policy [1]. I don't see specific discussion of the Java version there, but > it states that you should be able to drop in minor upgrades and have > existing apps keep working. Users might find it surprising if they try to > upgrade a cluster that has JDK 8. > > There is also the question of impact on downstream projects [2]. We'd have > to check plans with our consumers. > > What about the idea of shooting for a 3.4 release on JDK 11 (or even 17)? > The downside is that we'd probably need to set boundaries on end of > life/limited support for 3.2 and 3.3 to keep the workload manageable. > > [1] > > https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#Java_Binary_compatibility_for_end-user_applications_i.e._Apache_Hadoop_ABI > [2] https://github.com/apache/spark/blob/v3.3.2/pom.xml#L109 > > Chris Nauroth > > > On Tue, Mar 28, 2023 at 11:10 AM Ayush Saxena wrote: > > > > > > > it's already hard to migrate from JDK8 why not retarget JDK17. > > > > > > > +1, makes sense to me, sounds like a win-win situation to me, though > there > > would be some additional issues to chase now :) > > > > -Ayush > > > > > > On Tue, 28 Mar 2023 at 23:29, Wei-Chiu Chuang > wrote: > > > > > My random thoughts. Probably bad takes: > > > > > > There are projects experimenting with JDK17 now. > > > JDK11 active support will end in 6 months. If it's already hard to > > migrate > > > from JDK8 why not retarget JDK17. > > > > > > On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena > > wrote: > > > > > >> I know Jersey upgrade as a blocker. Some folks were chasing that last > > >> year during 3.3.4 time, I don’t know where it is now, didn’t see then > > >> what’s the problem there but I remember there was some intitial PR > which > > >> did it for HDFS atleast, so I never looked beyond that… > > >> > > >> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as > > >> java-11 only branch may be, but that is something later to decide, > once > > we > > >> get the code sorted… > > >> > > >> -Ayush > > >> > > >> > On 28-Mar-2023, at 9:16 PM, Steve Loughran > > > > >> wrote: > > >> > > > >> > well, how about we flip the switch and get on with it. > > >> > > > >> > slf4j seems happy on java11, > > >> > > > >> > side issue, anyone seen test failures on zulu1.8; somehow my test > run > > is > > >> > failing and i'm trying to work out whether its a mismatch in command > > >> > line/ide jvm versions, or the 3.3.5 JARs have been built with an > > openjdk > > >> > version which requires IntBuffer implements an overridden method > > >> IntBuffer > > >> > rewind(). > > >> > > > >> > java.lang.NoSuchMethodError: > > >> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer; > > >> > > > >> > at > > >> > org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341) > > >> > at > > >> > > > >> > > > org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308) > > >> > at > org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257) > > >> > at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202) > > >> > at java.io.DataInputStream.read(DataInputStream.java:149) > > >> > > > >> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani > > wrote: > > >> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2, > > >> jersey 1 > > >> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test > > >> stability. > > >> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran > > >> > >> >> wrote: > > >> >>> Now that hadoop 3.3.5 is out, i want to propose something new > > >> >>> we switch branch-3.3 and trunk to being java11 only > > >> >>> 1. java 11 has been out for years > > >> >>> 2. oracle java 8 is no longer available under "premier support"; > you > > >> >>> can't really get upgrades > > >> >>> > > https://www.oracle.com/java/technologies/java-se-support-roadmap.html > > >> >>> 3. openJDK 8 releases != oracle ones, and things you compile with > > them > > >> >>> don't always link to oracle java 8 (some classes in java.nio have > > >> >> added > > >> >>> more overrides) > > >> >>> 4. more and more libraries we want to upgrade to/bundle are java > 11 > > >> >> only > > >> >>> 5. moving to java 11 would cut our yetus build workload in half, > and > > >> >>> line up for adding java 17 builds instead. > > >> >>> I know there are some outstanding issues still in > > >> >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they > > >> >> blockers? > > >> >>> Could we just move to java11 and enhance at our leisure, once > java8 > > >> is no > > >> >>> longer a concern. > > >> > > >>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
In theory, I like the idea of setting aside Java 8. Unfortunately, I don't know that upgrading within the 3.3 line adheres to our binary compatibility policy [1]. I don't see specific discussion of the Java version there, but it states that you should be able to drop in minor upgrades and have existing apps keep working. Users might find it surprising if they try to upgrade a cluster that has JDK 8. There is also the question of impact on downstream projects [2]. We'd have to check plans with our consumers. What about the idea of shooting for a 3.4 release on JDK 11 (or even 17)? The downside is that we'd probably need to set boundaries on end of life/limited support for 3.2 and 3.3 to keep the workload manageable. [1] https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#Java_Binary_compatibility_for_end-user_applications_i.e._Apache_Hadoop_ABI [2] https://github.com/apache/spark/blob/v3.3.2/pom.xml#L109 Chris Nauroth On Tue, Mar 28, 2023 at 11:10 AM Ayush Saxena wrote: > > > > it's already hard to migrate from JDK8 why not retarget JDK17. > > > > +1, makes sense to me, sounds like a win-win situation to me, though there > would be some additional issues to chase now :) > > -Ayush > > > On Tue, 28 Mar 2023 at 23:29, Wei-Chiu Chuang wrote: > > > My random thoughts. Probably bad takes: > > > > There are projects experimenting with JDK17 now. > > JDK11 active support will end in 6 months. If it's already hard to > migrate > > from JDK8 why not retarget JDK17. > > > > On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena > wrote: > > > >> I know Jersey upgrade as a blocker. Some folks were chasing that last > >> year during 3.3.4 time, I don’t know where it is now, didn’t see then > >> what’s the problem there but I remember there was some intitial PR which > >> did it for HDFS atleast, so I never looked beyond that… > >> > >> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as > >> java-11 only branch may be, but that is something later to decide, once > we > >> get the code sorted… > >> > >> -Ayush > >> > >> > On 28-Mar-2023, at 9:16 PM, Steve Loughran > > >> wrote: > >> > > >> > well, how about we flip the switch and get on with it. > >> > > >> > slf4j seems happy on java11, > >> > > >> > side issue, anyone seen test failures on zulu1.8; somehow my test run > is > >> > failing and i'm trying to work out whether its a mismatch in command > >> > line/ide jvm versions, or the 3.3.5 JARs have been built with an > openjdk > >> > version which requires IntBuffer implements an overridden method > >> IntBuffer > >> > rewind(). > >> > > >> > java.lang.NoSuchMethodError: > >> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer; > >> > > >> > at > >> org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341) > >> > at > >> > > >> > org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308) > >> > at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257) > >> > at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202) > >> > at java.io.DataInputStream.read(DataInputStream.java:149) > >> > > >> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani > wrote: > >> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2, > >> jersey 1 > >> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test > >> stability. > >> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran > >> >> >> wrote: > >> >>> Now that hadoop 3.3.5 is out, i want to propose something new > >> >>> we switch branch-3.3 and trunk to being java11 only > >> >>> 1. java 11 has been out for years > >> >>> 2. oracle java 8 is no longer available under "premier support"; you > >> >>> can't really get upgrades > >> >>> > https://www.oracle.com/java/technologies/java-se-support-roadmap.html > >> >>> 3. openJDK 8 releases != oracle ones, and things you compile with > them > >> >>> don't always link to oracle java 8 (some classes in java.nio have > >> >> added > >> >>> more overrides) > >> >>> 4. more and more libraries we want to upgrade to/bundle are java 11 > >> >> only > >> >>> 5. moving to java 11 would cut our yetus build workload in half, and > >> >>> line up for adding java 17 builds instead. > >> >>> I know there are some outstanding issues still in > >> >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they > >> >> blockers? > >> >>> Could we just move to java11 and enhance at our leisure, once java8 > >> is no > >> >>> longer a concern. > >> > >> - > >> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > >> For additional commands, e-mail: common-dev-h...@hadoop.apache.org > >> > >> >
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
> > it's already hard to migrate from JDK8 why not retarget JDK17. > +1, makes sense to me, sounds like a win-win situation to me, though there would be some additional issues to chase now :) -Ayush On Tue, 28 Mar 2023 at 23:29, Wei-Chiu Chuang wrote: > My random thoughts. Probably bad takes: > > There are projects experimenting with JDK17 now. > JDK11 active support will end in 6 months. If it's already hard to migrate > from JDK8 why not retarget JDK17. > > On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena wrote: > >> I know Jersey upgrade as a blocker. Some folks were chasing that last >> year during 3.3.4 time, I don’t know where it is now, didn’t see then >> what’s the problem there but I remember there was some intitial PR which >> did it for HDFS atleast, so I never looked beyond that… >> >> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as >> java-11 only branch may be, but that is something later to decide, once we >> get the code sorted… >> >> -Ayush >> >> > On 28-Mar-2023, at 9:16 PM, Steve Loughran >> wrote: >> > >> > well, how about we flip the switch and get on with it. >> > >> > slf4j seems happy on java11, >> > >> > side issue, anyone seen test failures on zulu1.8; somehow my test run is >> > failing and i'm trying to work out whether its a mismatch in command >> > line/ide jvm versions, or the 3.3.5 JARs have been built with an openjdk >> > version which requires IntBuffer implements an overridden method >> IntBuffer >> > rewind(). >> > >> > java.lang.NoSuchMethodError: >> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer; >> > >> > at >> org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341) >> > at >> > >> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308) >> > at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257) >> > at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202) >> > at java.io.DataInputStream.read(DataInputStream.java:149) >> > >> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani wrote: >> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2, >> jersey 1 >> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test >> stability. >> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran >> > >> wrote: >> >>> Now that hadoop 3.3.5 is out, i want to propose something new >> >>> we switch branch-3.3 and trunk to being java11 only >> >>> 1. java 11 has been out for years >> >>> 2. oracle java 8 is no longer available under "premier support"; you >> >>> can't really get upgrades >> >>> https://www.oracle.com/java/technologies/java-se-support-roadmap.html >> >>> 3. openJDK 8 releases != oracle ones, and things you compile with them >> >>> don't always link to oracle java 8 (some classes in java.nio have >> >> added >> >>> more overrides) >> >>> 4. more and more libraries we want to upgrade to/bundle are java 11 >> >> only >> >>> 5. moving to java 11 would cut our yetus build workload in half, and >> >>> line up for adding java 17 builds instead. >> >>> I know there are some outstanding issues still in >> >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they >> >> blockers? >> >>> Could we just move to java11 and enhance at our leisure, once java8 >> is no >> >>> longer a concern. >> >> - >> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org >> For additional commands, e-mail: common-dev-h...@hadoop.apache.org >> >>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
My random thoughts. Probably bad takes: There are projects experimenting with JDK17 now. JDK11 active support will end in 6 months. If it's already hard to migrate from JDK8 why not retarget JDK17. On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena wrote: > I know Jersey upgrade as a blocker. Some folks were chasing that last year > during 3.3.4 time, I don’t know where it is now, didn’t see then what’s the > problem there but I remember there was some intitial PR which did it for > HDFS atleast, so I never looked beyond that… > > I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as java-11 > only branch may be, but that is something later to decide, once we get the > code sorted… > > -Ayush > > > On 28-Mar-2023, at 9:16 PM, Steve Loughran > wrote: > > > > well, how about we flip the switch and get on with it. > > > > slf4j seems happy on java11, > > > > side issue, anyone seen test failures on zulu1.8; somehow my test run is > > failing and i'm trying to work out whether its a mismatch in command > > line/ide jvm versions, or the 3.3.5 JARs have been built with an openjdk > > version which requires IntBuffer implements an overridden method > IntBuffer > > rewind(). > > > > java.lang.NoSuchMethodError: > java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer; > > > > at > org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341) > > at > > > org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308) > > at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257) > > at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202) > > at java.io.DataInputStream.read(DataInputStream.java:149) > > > >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani wrote: > >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2, > jersey 1 > >> to 2 and junit 4 to 5) are blockers for java 11 compile + test > stability. > >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran > >> wrote: > >>> Now that hadoop 3.3.5 is out, i want to propose something new > >>> we switch branch-3.3 and trunk to being java11 only > >>> 1. java 11 has been out for years > >>> 2. oracle java 8 is no longer available under "premier support"; you > >>> can't really get upgrades > >>> https://www.oracle.com/java/technologies/java-se-support-roadmap.html > >>> 3. openJDK 8 releases != oracle ones, and things you compile with them > >>> don't always link to oracle java 8 (some classes in java.nio have > >> added > >>> more overrides) > >>> 4. more and more libraries we want to upgrade to/bundle are java 11 > >> only > >>> 5. moving to java 11 would cut our yetus build workload in half, and > >>> line up for adding java 17 builds instead. > >>> I know there are some outstanding issues still in > >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they > >> blockers? > >>> Could we just move to java11 and enhance at our leisure, once java8 is > no > >>> longer a concern. > > - > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > >
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
I know Jersey upgrade as a blocker. Some folks were chasing that last year during 3.3.4 time, I don’t know where it is now, didn’t see then what’s the problem there but I remember there was some intitial PR which did it for HDFS atleast, so I never looked beyond that… I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as java-11 only branch may be, but that is something later to decide, once we get the code sorted… -Ayush > On 28-Mar-2023, at 9:16 PM, Steve Loughran > wrote: > > well, how about we flip the switch and get on with it. > > slf4j seems happy on java11, > > side issue, anyone seen test failures on zulu1.8; somehow my test run is > failing and i'm trying to work out whether its a mismatch in command > line/ide jvm versions, or the 3.3.5 JARs have been built with an openjdk > version which requires IntBuffer implements an overridden method IntBuffer > rewind(). > > java.lang.NoSuchMethodError: java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer; > > at org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341) > at > org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308) > at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257) > at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202) > at java.io.DataInputStream.read(DataInputStream.java:149) > >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani wrote: >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2, jersey 1 >> to 2 and junit 4 to 5) are blockers for java 11 compile + test stability. >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran > wrote: >>> Now that hadoop 3.3.5 is out, i want to propose something new >>> we switch branch-3.3 and trunk to being java11 only >>> 1. java 11 has been out for years >>> 2. oracle java 8 is no longer available under "premier support"; you >>> can't really get upgrades >>> https://www.oracle.com/java/technologies/java-se-support-roadmap.html >>> 3. openJDK 8 releases != oracle ones, and things you compile with them >>> don't always link to oracle java 8 (some classes in java.nio have >> added >>> more overrides) >>> 4. more and more libraries we want to upgrade to/bundle are java 11 >> only >>> 5. moving to java 11 would cut our yetus build workload in half, and >>> line up for adding java 17 builds instead. >>> I know there are some outstanding issues still in >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they >> blockers? >>> Could we just move to java11 and enhance at our leisure, once java8 is no >>> longer a concern. - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
well, how about we flip the switch and get on with it. slf4j seems happy on java11, side issue, anyone seen test failures on zulu1.8; somehow my test run is failing and i'm trying to work out whether its a mismatch in command line/ide jvm versions, or the 3.3.5 JARs have been built with an openjdk version which requires IntBuffer implements an overridden method IntBuffer rewind(). java.lang.NoSuchMethodError: java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer; at org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341) at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308) at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257) at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202) at java.io.DataInputStream.read(DataInputStream.java:149) On Tue, 28 Mar 2023 at 15:52, Viraj Jasani wrote: > IIRC some of the ongoing major dependency upgrades (log4j 1 to 2, jersey 1 > to 2 and junit 4 to 5) are blockers for java 11 compile + test stability. > > > On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran > > wrote: > > > Now that hadoop 3.3.5 is out, i want to propose something new > > > > we switch branch-3.3 and trunk to being java11 only > > > > > >1. java 11 has been out for years > >2. oracle java 8 is no longer available under "premier support"; you > >can't really get upgrades > >https://www.oracle.com/java/technologies/java-se-support-roadmap.html > >3. openJDK 8 releases != oracle ones, and things you compile with them > >don't always link to oracle java 8 (some classes in java.nio have > added > >more overrides) > >4. more and more libraries we want to upgrade to/bundle are java 11 > only > >5. moving to java 11 would cut our yetus build workload in half, and > >line up for adding java 17 builds instead. > > > > > > I know there are some outstanding issues still in > > https://issues.apache.org/jira/browse/HADOOP-16795 -but are they > blockers? > > Could we just move to java11 and enhance at our leisure, once java8 is no > > longer a concern. > > >
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
IIRC some of the ongoing major dependency upgrades (log4j 1 to 2, jersey 1 to 2 and junit 4 to 5) are blockers for java 11 compile + test stability. On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran wrote: > Now that hadoop 3.3.5 is out, i want to propose something new > > we switch branch-3.3 and trunk to being java11 only > > >1. java 11 has been out for years >2. oracle java 8 is no longer available under "premier support"; you >can't really get upgrades >https://www.oracle.com/java/technologies/java-se-support-roadmap.html >3. openJDK 8 releases != oracle ones, and things you compile with them >don't always link to oracle java 8 (some classes in java.nio have added >more overrides) >4. more and more libraries we want to upgrade to/bundle are java 11 only >5. moving to java 11 would cut our yetus build workload in half, and >line up for adding java 17 builds instead. > > > I know there are some outstanding issues still in > https://issues.apache.org/jira/browse/HADOOP-16795 -but are they blockers? > Could we just move to java11 and enhance at our leisure, once java8 is no > longer a concern. >
