> I cannot estimate the effort to backport large features like the new layouts that are currently being added (e.g. RunEndEncoding, ListView, etc.).
In my mind we are only talking about patch releases for security fixes or similarly critical issues as otherwise the effort to maintain 'v14' (but actually arrow-latest) would surely overshadow any gains made by deprecating jdk 8? On Wed, Oct 11, 2023 at 3:31 AM Gang Wu <ust...@gmail.com> wrote: > I agree that we have to move on. It seems that patch release to > Arrow v14 is a good idea, though I cannot estimate the effort to > backport large features like the new layouts that are currently > being added (e.g. RunEndEncoding, ListView, etc.). > > As an Arrow developer, I am always happy to drop JDK 8. My > employer has leveraged Apache Arrow in the internal engine > and depends on Arrow Java in the Java SDK. For end users > who cannot get away with JDK 8, we might need to prepare > different Java SDKs and use features that are available in the > Arrow v14, or let the server side chooses which subset of > features based on the SDK version. > > Thanks, > Gang > > > > On Wed, Oct 11, 2023 at 12:40 AM Dane Pitkin <d...@voltrondata.com.invalid > > > wrote: > > > To summarize the discussion so far: > > > > * Some Arrow Java users are still on JDK 8 > > * Arrow v14 is proposed as the final version with JDK 8 support > > * Arrow v14 can support patch releases if necessary for JDK 8 users > > * There is an open question to decide if JDK 11 should be dropped > > simultaneously > > > > Gang Wu, I'm curious what are your thoughts given your initial concerns? > > > > -Dane > > > > On Sat, Oct 7, 2023 at 12:00 AM Jacob Wujciak-Jens > > <ja...@voltrondata.com.invalid> wrote: > > > > > From a release engineer perspective (without java knowledge) I agree > with > > > Micah, I'd rather make a patch release for an older version if needed > but > > > modernize the codebase and simplify CI! > > > > > > > > > On Sat, Oct 7, 2023 at 5:27 AM Micah Kornfield <emkornfi...@gmail.com> > > > wrote: > > > > > > > I think given the stability of Arrow Java, dropping support probably > > > makes > > > > sense. If a bug comes up or consumers really need to new features we > > can > > > > always make a patch release of an older version. > > > > > > > > On Thu, Oct 5, 2023 at 3:13 PM Dane Pitkin > > <d...@voltrondata.com.invalid > > > > > > > > wrote: > > > > > > > > > I also learned today that Apache Spark has dropped support for > Java 8 > > > and > > > > > 11 for their next release (v4.0)[1]. Should we consider dropping > Java > > > 11 > > > > as > > > > > well? > > > > > > > > > > [1]https://github.com/apache/spark/pull/43005 > > > > > > > > > > -Dane > > > > > > > > > > On Thu, Oct 5, 2023 at 3:30 PM Dane Pitkin <d...@voltrondata.com> > > > wrote: > > > > > > > > > > > I created a GH issue[1] proposing the removal of Java 8 support. > It > > > > > > would target the Arrow v15 release (~Jan 2024). > > > > > > > > > > > > IMO it would be in the best interest of the project for two major > > > > > reasons: > > > > > > 1. Unblock the Java Platform Module System (JPMS)[2] > > implementation. > > > > > > 2. Unblock Arrow from upgrading dependencies that no longer > support > > > > Java > > > > > > 8. (See [1] for examples) > > > > > > > > > > > > Since Arrow Java has been quite stable, will Java 8 users be okay > > > with > > > > > > pinning Arrow to the last supported release (v14) if the Arrow > > > project > > > > > > ultimately decides to remove Java 8 support? > > > > > > > > > > > > > > > > > > [1]https://github.com/apache/arrow/issues/38051 > > > > > > [2]https://en.wikipedia.org/wiki/Java_Platform_Module_System > > > > > > > > > > > > -Dane > > > > > > > > > > > > On Fri, Sep 15, 2023 at 12:26 PM Dane Pitkin < > d...@voltrondata.com > > > > > > > > wrote: > > > > > > > > > > > >> - As a low level library, users have to add specific flags to > use > > > > > >>> Java 9 and up with Arrow to resolve issues with java.nio. This > > has > > > > > >>> been annoying for our customers constantly. If this is not > > > resolved, > > > > > >>> I would say we may see a lot of complaints in the future. > > > > > >>> > > > > > >> I filed issue 37739[1] to track this, but it sounds like this > > can't > > > be > > > > > >> changed until Java 21 or 24. > > > > > >> > > > > > >> - It seems that the EOL of Java 8 from Oracle is Dec 2030 [2]. A > > lot > > > > > >>> users will still stay on it for a long time. At least this is > > true > > > > for > > > > > >>> our > > > > > >>> customers. So I am afraid we may not upgrade to newer versions > > > > > >>> of Arrow if it no longer supports Java 8. > > > > > >>> > > > > > >> Java 8 does have a long Extended Support timeline, but a recent > > > > > >> report shows Java 11 increasing in adoption vs Java 8. "More > than > > > 56% > > > > of > > > > > >> applications are now using Java 11 in production (up from 48% in > > > 2022 > > > > > and > > > > > >> 11% in 2020). Java 8 is a close second with nearly 33% of > > > applications > > > > > >> using it in production (down from 46% in 2022)."[2] > > > > > >> I expect the Java ecosystem will find a way to move on from > Java 8 > > > > much > > > > > >> sooner than 2030, meaning many of Arrow's dependencies could > drop > > > > > support > > > > > >> for Java 8 before then. At this point, Arrow may be forced to > > > support > > > > a > > > > > >> higher minimum Java version. > > > > > >> > > > > > >> That being said, it's hard to argue against real use cases. I'd > be > > > > > >> curious to hear what Java version other users of Arrow are using > > > (and > > > > if > > > > > >> there is a timeline to upgrade if on Java 8). > > > > > >> > > > > > >> > > > > > >> [1]https://github.com/apache/arrow/issues/37739 > > > > > >> [2] > > > > > >> > > > > > > > > > > > > > > > https://newrelic.com/sites/default/files/2023-04/new-relic-2023-state-of-the-java-ecosystem-2023-04-20.pdf > > > > > >> > > > > > >> > > > > > >> -Dane > > > > > >> > > > > > >> > > > > > >> On Thu, Sep 14, 2023 at 11:45 AM Gang Wu <ust...@gmail.com> > > wrote: > > > > > >> > > > > > >>> Thanks for bringing this up! > > > > > >>> > > > > > >>> I have two concerns of dropping Java 8 support: > > > > > >>> - As a low level library, users have to add specific flags [1] > to > > > use > > > > > >>> Java 9 and up with Arrow to resolve issues with java.nio. This > > has > > > > > >>> been annoying for our customers constantly. If this is not > > > resolved, > > > > > >>> I would say we may see a lot of complaints in the future. > > > > > >>> - It seems that the EOL of Java 8 from Oracle is Dec 2030 [2]. > A > > > lot > > > > > >>> users will still stay on it for a long time. At least this is > > true > > > > for > > > > > >>> our > > > > > >>> customers. So I am afraid we may not upgrade to newer versions > > > > > >>> of Arrow if it no longer supports Java 8. > > > > > >>> > > > > > >>> [1] > > > > https://arrow.apache.org/docs/java/install.html#java-compatibility > > > > > >>> [2] > > > > > >>> > > > > > https://www.oracle.com/java/technologies/java-se-support-roadmap.html > > > > > >>> > > > > > >>> Best, > > > > > >>> Gang > > > > > >>> > > > > > >>> > > > > > >>> > > > > > >>> On Thu, Sep 14, 2023 at 11:14 PM David Dali Susanibar Arce < > > > > > >>> davi.sar...@gmail.com> wrote: > > > > > >>> > > > > > >>> > Hi Arrow Java developers, > > > > > >>> > > > > > > >>> > I would like to propose a timeline for dropping support for > > Java > > > 8: > > > > > >>> > - Propose to drop JDK8 in Arrow v15 (2 releases from now) > > > > > >>> > - JDK 21 support will be added before removal of JDK8 > > > > > >>> > > > > > > >>> > Why? > > > > > >>> > - Java 8 no longer receives Premier Support (1) > > > > > >>> > - Some Arrow Java (test) dependencies have already started to > > > drop > > > > > >>> > Java 8 support, forcing us to pin to older packager versions > > > > > >>> > > > > > > >>> > Also note: > > > > > >>> > - gRPC Java may drop support for a JDK version when that > > version > > > is > > > > > no > > > > > >>> > longer receiving Premier Support from Oracle (2), more detail > > at > > > > Java > > > > > >>> > 8 / Java 11 support timeline in gRPC here (3) > > > > > >>> > - Spark plans to tentatively drop JDK 8 support in Spark 4.0 > > (4), > > > > > >>> > which has a release timeline of approximately 2024-06 (5). Is > > it > > > > fine > > > > > >>> > for us to drop JDK 8 support before spark? > > > > > >>> > > > > > > >>> > (1) > > > > > >>> > > > > > https://www.oracle.com/java/technologies/java-se-support-roadmap.html > > > > > >>> > (2) > > > > > >>> > > > > > > >>> > > > > > > > > > > > > > > > https://github.com/grpc/proposal/pull/283/files#:~:text=gRPC%20Java%20may,support%5D > > > > > >>> > . > > > > > >>> > (3) > > > > https://groups.google.com/g/grpc-io/c/-XK6Kd_19YQ/m/-4s07TzdAgAJ > > > > > >>> > (4) https://issues.apache.org/jira/browse/SPARK-44112 > > > > > >>> > (5) > > > > https://www.mail-archive.com/dev@spark.apache.org/msg30460.html > > > > > >>> > > > > > > >>> > Consider: > > > > > >>> > - JDK8 deprecation is currently not mandatory. We simply want > > to > > > > > >>> > devote more time to development of Java LTS versions 11, 17 > and > > > 21. > > > > > >>> > - Java 11 is dropping Premier Support this month. > > > > > >>> > > > > > > >>> > Best regards, > > > > > >>> > > > > > > >>> > -- > > > > > >>> > David Susanibar > > > > > >>> > > > > > > >>> > > > > > >> > > > > > > > > > > > > > > >
