Hi, IMHO, Apache Parquet Java [1] cannot drop Java 8 in all 1.x releases to keep maximum backward compatibility. There was a discussion on the 2.x major release [2] and v3 format [3]. I think it is a good chance to drop Java 8 from the 2.x release.
[1] https://github.com/apache/parquet-java [2] https://lists.apache.org/thread/kttwbl5l7opz6nwb5bck2gghc2y3td0o [3] https://lists.apache.org/thread/5jyhzkwyrjk9z52g0b49g31ygnz73gxo Best, Gang On Fri, May 24, 2024 at 11:47 PM Weston Pace <weston.p...@gmail.com> wrote: > No vote is required from an ASF perspective (this is not a release) > No vote is required from Arrow conventions (this is not a spec change and > does not impact more than one implementation) > > I will send a message to the parquet ML to solicit feedback. > > On Fri, May 24, 2024 at 8:22 AM Laurent Goujon <laur...@dremio.com.invalid > > > wrote: > > > I would say so because it is akin to removing a large feature but maybe > > some PMC can chime in? > > > > Laurent > > > > On Tue, May 21, 2024 at 12:16 PM Dane Pitkin <dpit...@apache.org> wrote: > > > > > I haven't been active in Apache Parquet, but I did not see any prior > > > discussions on this topic in their Jira or dev mailing list. > > > > > > Do we think a vote is needed before officially moving forward with > Java 8 > > > deprecation? > > > > > > On Mon, May 20, 2024 at 12:50 PM Laurent Goujon > > <laur...@dremio.com.invalid > > > > > > > wrote: > > > > > > > I also mentioned Apache Parquet and haven't seen someone mentioned > > > if/when > > > > Apache Parquet would transition. > > > > > > > > > > > > > > > > On Fri, May 17, 2024 at 9:07 AM Dane Pitkin <dpit...@apache.org> > > wrote: > > > > > > > > > Fokko, thank you for these datapoints! It's great to see how other > > low > > > > > level Java OSS projects are approaching this. > > > > > > > > > > JB, I believe yes we have formal consensus to drop Java 8 in Arrow. > > > There > > > > > was no contention in current discussions across [GitHub issues | > > Arrow > > > > > Mailing List | Community Syncs]. > > > > > > > > > > We can save Java 11 deprecation for a future discussion. For users > on > > > > Java > > > > > 11, I do anticipate this discussion to come shortly after Java 8 > > > > > deprecation is released. > > > > > > > > > > On Fri, May 17, 2024 at 10:02 AM Fokko Driesprong < > fo...@apache.org> > > > > > wrote: > > > > > > > > > > > I was traveling the last few weeks, so just a follow-up from my > > end. > > > > > > > > > > > > Fokko, can you elaborate on the discussions held in other OSS > > > projects > > > > to > > > > > >> drop Java <17? How did they weigh the benefits/drawbacks for > > > dropping > > > > > both > > > > > >> Java 8 and 11 LTS versions? I'd also be curious if other > projects > > > plan > > > > > to > > > > > >> support older branches with security patches. > > > > > > > > > > > > > > > > > > So, the ones that I'm involved with (including a TLDR): > > > > > > > > > > > > - Avro: > > > > > > - (April 2024: Consensus on moving to 11+, +1 for moving to > > > 17+) > > > > > > > > > https://lists.apache.org/thread/6vbd3w5qk7mpb5lyrfyf2s0z1cymjt5w > > > > > > - (Jan 2024: Consensus on dropping 8) > > > > > > > > > https://lists.apache.org/thread/bd39zhk655pgzfctq763vp3z4xrjpx58 > > > > > > - Iceberg: > > > > > > - (Jan 2023: Concerns about Hive): > > > > > > > > > https://lists.apache.org/thread/hr7rdxvddw3fklfyg3dfbqbsy81hzhyk > > > > > > - (Feb 2024: Concensus to drop Hadoop 2.x, and move to > > JDK11+, > > > > > > also +1's for moving to 17+): > > > > > > > > > https://lists.apache.org/thread/ntrk2thvsg9tdccwd4flsdz9gg743368 > > > > > > > > > > > > I think the most noteworthy (slow-moving in general): > > > > > > > > > > > > - Spark 4 supports JDK 17+ > > > > > > - Hive 4 is still on Java 8 > > > > > > <https://github.com/apache/hive?tab=readme-ov-file#java> > > > > > > > > > > > > It looks like most of the projects are looking at each other. > Keep > > in > > > > > > mind, that projects that still support older versions of Java, > can > > > > still > > > > > > use older versions of Arrow. > > > > > > > > > > > > [image: spiderman-pointing-at-spiderman.jpeg] > > > > > > (in case the image doesn't come through, that's Spiderman > pointing > > at > > > > > > Spiderman) > > > > > > > > > > > > Concerning the Java 11 support, some data: > > > > > > > > > > > > - Oracle 11: support until January 2032 (extended fee has been > > > > waived) > > > > > > - Cornetto 11: September 2027 > > > > > > - Adoptium 11: At least Oct 2027 > > > > > > - Zulu 11: Jan 2032 > > > > > > - OpenJDK11: October 2024 > > > > > > > > > > > > I think it is fair to support 11 for the time being, but at some > > > point, > > > > > we > > > > > > also have to move on and start exploiting the new features and > make > > > > sure > > > > > > that we keep up to date. For example, Java 8 also has extended > > > support > > > > > > until 2030. Dependabot on the Iceberg project > > > > > > < > > > > > > > > > > > > > > > https://github.com/apache/iceberg/pulls?q=is%3Aopen+is%3Apr+label%3Adependencies > > > > > > > > > > > > nicely shows which projects are already at JDK11+ :) > > > > > > > > > > > > Thanks Dane for driving this! > > > > > > > > > > > > Kind regards, > > > > > > Fokko > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Op vr 17 mei 2024 om 07:44 schreef Jean-Baptiste Onofré < > > > > j...@nanthrax.net > > > > > >: > > > > > > > > > > > >> Hi Dane > > > > > >> > > > > > >> Do we have a formal consensus about Java version in regards of > > arrow > > > > > >> version ? > > > > > >> I agree with the plan but just wondering if it’s ok from > everyone > > > with > > > > > the > > > > > >> community. > > > > > >> > > > > > >> Regards > > > > > >> JB > > > > > >> > > > > > >> Le jeu. 16 mai 2024 à 18:05, Dane Pitkin <dpit...@apache.org> a > > > > écrit : > > > > > >> > > > > > >> > To wrap up this thread on Java 8 deprecation, here is my > current > > > > plan > > > > > of > > > > > >> > action: > > > > > >> > > > > > > >> > 1) Arrow v17 will be the last version supporting Java 8 and > the > > > > > release > > > > > >> > notes will warn of its impending deprecation. > > > > > >> > 2) Arrow v18 will be the first release supporting min version > > Java > > > > 11. > > > > > >> > > > > > > >> > I have updated the GH issue[1] to reflect this. > > > > > >> > > > > > > >> > [1]https://github.com/apache/arrow/issues/38051 > > > > > >> > > > > > > >> > On Wed, May 8, 2024 at 5:46 PM Dane Pitkin > > > > > <d...@voltrondata.com.invalid > > > > > >> > > > > > > >> > wrote: > > > > > >> > > > > > > >> > > Thank you all for your valuable input. The consensus from my > > > > > >> > understanding > > > > > >> > > is that dropping Java 8 is not contentious, so we will move > > > > forward > > > > > >> here. > > > > > >> > > > > > > > >> > > We won't drop Java 11 yet, but there's a chance it will > happen > > > > > sooner > > > > > >> > than > > > > > >> > > later. I brought up Java 8 & 11 deprecation in the community > > > sync > > > > > >> again > > > > > >> > > today. The summary is that the ASF could be enforcing > stricter > > > > > >> security > > > > > >> > > practices in the near future. Arrow Java may be forced to > drop > > > > Java > > > > > >> 11 if > > > > > >> > > any of its dependencies no longer support Java 11. This is > > > > something > > > > > >> > we'll > > > > > >> > > have to investigate and monitor. When the time is right, we > > > should > > > > > >> start > > > > > >> > a > > > > > >> > > new thread on the mailing list to discuss. > > > > > >> > > > > > > > >> > > Thanks, > > > > > >> > > Dane > > > > > >> > > > > > > > >> > > On Sat, May 4, 2024 at 2:51 AM <martin.trave...@icloud.com > > > > .invalid> > > > > > >> > wrote: > > > > > >> > > > > > > > >> > > > Hi, > > > > > >> > > > > > > > > >> > > > We were originally expecting to keep Java 11 to the 2026 > EOL > > > > date > > > > > >> for > > > > > >> > > > extended support, but now that date is moved to 2032 which > > > feels > > > > > >> like > > > > > >> > > more > > > > > >> > > > time than we need. The issue for us is that getting > > technology > > > > > >> approved > > > > > >> > > for > > > > > >> > > > use in an enterprise can have ridiculously long lead > times, > > so > > > > > >> having a > > > > > >> > > > minimum supported version that is only 2 years old, while > > > > probably > > > > > >> ok > > > > > >> > in > > > > > >> > > > most case, would be a bit aggressive. We use optional > > > > dependencies > > > > > >> > where > > > > > >> > > we > > > > > >> > > > can, so e.g. the Java 17 dependency for Spark 4 would only > > > > affect > > > > > >> > clients > > > > > >> > > > using Spark 4, and they could wait to upgrade. But we > chose > > to > > > > use > > > > > >> > Arrow > > > > > >> > > in > > > > > >> > > > the core of our product, it is the internal format > > everything > > > > else > > > > > >> goes > > > > > >> > > > through. On the compliance side we have to keep current > with > > > > > >> security > > > > > >> > > > updates, so there is no option to stick on an old version. > > > > > >> > > > > > > > > >> > > > If we were to drop Java 11 after the next LTS comes out, > > i.e. > > > > > 2025 / > > > > > >> > > 2026, > > > > > >> > > > then the three latest LTS versions would be supported and > > the > > > > > >> minimum > > > > > >> > > > version would have been available for 4 - 5 years. I think > > it > > > > > would > > > > > >> be > > > > > >> > > very > > > > > >> > > > hard to argue 17 can’t be made available at that point. If > > > Arrow > > > > > >> forces > > > > > >> > > our > > > > > >> > > > hand then obviously we’ll have to go sooner, but it > wouldn’t > > > be > > > > > >> ideal > > > > > >> > for > > > > > >> > > > us. > > > > > >> > > > > > > > > >> > > > Lastly just on language capabilities, the only things > we’re > > > > really > > > > > >> > > > interested in are performance related, probably virtual > > > threads > > > > > and > > > > > >> > > foreign > > > > > >> > > > memory would be the main ones. Both of the those could be > > > > optional > > > > > >> > > > dependencies, in the case of FFM we’d rely on either > > > yourselves > > > > or > > > > > >> > Netty > > > > > >> > > > anyway to provide an allocator. So in fact there is very > > > little > > > > > >> benefit > > > > > >> > > for > > > > > >> > > > us to drop Java 11 early, all it costs us is one extra CI > > job. > > > > > >> > > > > > > > > >> > > > Hope some of this is helpful - apologies for the high > > latency, > > > > > busy > > > > > >> as > > > > > >> > > > always!! > > > > > >> > > > > > > > > >> > > > Martin. > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > On 1 May 2024, at 22:38, Dane Pitkin > > > > > <d...@voltrondata.com.INVALID > > > > > >> > > > > > > >> > > > wrote: > > > > > >> > > > > > > > > > >> > > > > Thanks, Martin. It's great to hear of real-world use > > cases. > > > Do > > > > > you > > > > > >> > > > > anticipate any timeline for dropping Java 11 for your > > > product? > > > > > If > > > > > >> > Arrow > > > > > >> > > > did > > > > > >> > > > > drop Java 11, then it sounds like pinning Arrow Java to > an > > > > older > > > > > >> > > version > > > > > >> > > > > wouldn't be an ideal option if security patches are not > > > > > >> backported. > > > > > >> > > > > > > > > > >> > > > > Fokko, can you elaborate on the discussions held in > other > > > OSS > > > > > >> > projects > > > > > >> > > to > > > > > >> > > > > drop Java <17? How did they weigh the benefits/drawbacks > > for > > > > > >> dropping > > > > > >> > > > both > > > > > >> > > > > Java 8 and 11 LTS versions? I'd also be curious if other > > > > > projects > > > > > >> > plan > > > > > >> > > to > > > > > >> > > > > support older branches with security patches. > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > On Tue, Apr 30, 2024 at 4:14 PM < > > martin.trave...@icloud.com > > > > > >> .invalid> > > > > > >> > > > wrote: > > > > > >> > > > > > > > > > >> > > > >> Speaking for my own product we would like to see Java > 11 > > > > > >> support, we > > > > > >> > > > rely > > > > > >> > > > >> heavily on Arrow and have Java 11 as our minimum > > supported > > > > > >> version. > > > > > >> > > We’d > > > > > >> > > > >> like to keep doing that if possible. Our clients are > big > > > > > >> enterprises > > > > > >> > > > with > > > > > >> > > > >> notoriously sluggish update cycles, so we want to offer > > > > maximum > > > > > >> > > > >> compatibility. Once security patches are no longer > > > available > > > > on > > > > > >> the > > > > > >> > > > regular > > > > > >> > > > >> public channels then there is a compliance issue, so we > > > > > generally > > > > > >> > > follow > > > > > >> > > > >> the EOL schedule of our dependencies. > > > > > >> > > > >> > > > > > >> > > > >> Corretto, Adoptium and Zulu all have recent public > builds > > > of > > > > > >> both 8 > > > > > >> > > and > > > > > >> > > > 11 > > > > > >> > > > >> and look set to support them with public builds for > many > > > > years > > > > > to > > > > > >> > > come. > > > > > >> > > > >> Several organisations I have worked with switched away > > from > > > > > >> Oracle > > > > > >> > > when > > > > > >> > > > >> they made their licensing blunder with Java 8 and > > although > > > > that > > > > > >> is > > > > > >> > > > >> rectified now, the change seems to have stuck in quite > a > > > few > > > > > >> places > > > > > >> > > (at > > > > > >> > > > >> least in my anecdotal experience). > > > > > >> > > > >> > > > > > >> > > > >> A major practical difference to me in Java 17 is the > > strong > > > > > >> > > > encapsulation > > > > > >> > > > >> of internals. Since that affects the majority of > serious > > > Java > > > > > >> > > > applications > > > > > >> > > > >> then perhaps most people have figured out by now to add > > the > > > > JVM > > > > > >> > params > > > > > >> > > > that > > > > > >> > > > >> let Java continue working. Still, it could be a > > > > consideration, > > > > > if > > > > > >> > > > Java17 > > > > > >> > > > >> is the baseline supported version. > > > > > >> > > > >> > > > > > >> > > > >> Regards, > > > > > >> > > > >> Martin. > > > > > >> > > > >> > > > > > >> > > > >> - In case anyone is curious why we don’t support Java 8 > > per > > > > our > > > > > >> own > > > > > >> > > > >> policy, it’s because of the “var” keyword - seriously, > > why > > > > did > > > > > >> Java > > > > > >> > > > take so > > > > > >> > > > >> long with that, even C++ got there sooner! > > > > > >> > > > >> > > > > > >> > > > >>> On 30 Apr 2024, at 16:20, Jacob Wujciak < > > > > > assignu...@apache.org> > > > > > >> > > wrote: > > > > > >> > > > >>> > > > > > >> > > > >>> Hello everyone! > > > > > >> > > > >>> Great to see this move forward! > > > > > >> > > > >>> +1 on dropping both 8 and 11 unless there is very good > > > > reason > > > > > to > > > > > >> > keep > > > > > >> > > > 11 > > > > > >> > > > >>> around. > > > > > >> > > > >>> Otherwise people will just move to 11 and then have > the > > > pain > > > > > of > > > > > >> > > > migration > > > > > >> > > > >>> again when we drop that (which will happen soon > > regardless > > > > > imo). > > > > > >> > > > >>> > > > > > >> > > > >>> Am Di., 30. Apr. 2024 um 16:18 Uhr schrieb Dane Pitkin > > > > > >> > > > >>> <d...@voltrondata.com.invalid>: > > > > > >> > > > >>> > > > > > >> > > > >>>> Thanks, JB. Are we aware of any downstream > dependencies > > > > that > > > > > >> would > > > > > >> > > > >> benefit > > > > > >> > > > >>>> from maintaining Java 11 support? Apache Spark jumped > > > > > straight > > > > > >> to > > > > > >> > > Java > > > > > >> > > > >> 17. > > > > > >> > > > >>>> It seems other projects are dropping both 8 and 11 at > > the > > > > > same > > > > > >> > time > > > > > >> > > as > > > > > >> > > > >>>> mentioned by Fokko. From a maintenance perspective, > it > > > > would > > > > > be > > > > > >> > nice > > > > > >> > > > to > > > > > >> > > > >>>> drop both. > > > > > >> > > > >>>> > > > > > >> > > > >>>> On Mon, Apr 29, 2024 at 11:20 AM Jean-Baptiste > Onofré < > > > > > >> > > > j...@nanthrax.net> > > > > > >> > > > >>>> wrote: > > > > > >> > > > >>>> > > > > > >> > > > >>>>> Hi > > > > > >> > > > >>>>> > > > > > >> > > > >>>>> I think it's time to drop JDK8 support. I would say > > that > > > > we > > > > > >> > should > > > > > >> > > > >>>>> keep Java11 (jumping directly to Java17 would be > > > > problematic > > > > > >> > > > >>>>> potentially for some users I guess). > > > > > >> > > > >>>>> > > > > > >> > > > >>>>> Regards > > > > > >> > > > >>>>> JB > > > > > >> > > > >>>>> > > > > > >> > > > >>>>> On Thu, Apr 25, 2024 at 10:21 PM James Duong > > > > > >> > > > >>>>> <james.du...@improving.com.invalid> wrote: > > > > > >> > > > >>>>>> > > > > > >> > > > >>>>>> If we dropped JDK 8, we could use the JDK to > compile > > > > > >> > > > module-info.java > > > > > >> > > > >>>>> files. Then we could remove the custom maven plugin > > > we’re > > > > > >> using > > > > > >> > for > > > > > >> > > > >>>>> compiling module-info.java files for JPMS support > and > > > get > > > > > >> better > > > > > >> > > IDE > > > > > >> > > > >>>>> integration (as what we’re doing currently somewhat > > > > > shoe-horns > > > > > >> > > module > > > > > >> > > > >>>>> information alongside JDK8 bytecode). > > > > > >> > > > >>>>>> > > > > > >> > > > >>>>>> From: Dane Pitkin <d...@voltrondata.com.INVALID> > > > > > >> > > > >>>>>> Date: Thursday, April 25, 2024 at 1:02 PM > > > > > >> > > > >>>>>> To: dev@arrow.apache.org <dev@arrow.apache.org> > > > > > >> > > > >>>>>> Subject: [DISCUSS] Drop Java 8 support > > > > > >> > > > >>>>>> Hi all, > > > > > >> > > > >>>>>> > > > > > >> > > > >>>>>> I would like to revisit the discussion of dropping > > > Java 8 > > > > > >> (and > > > > > >> > > maybe > > > > > >> > > > >>>> 11) > > > > > >> > > > >>>>>> from Arrow's Java implementation. See GH issue[1] > > > below. > > > > > This > > > > > >> > was > > > > > >> > > > also > > > > > >> > > > >>>>>> discussed in the last Arrow community sync meeting > on > > > > > >> > 2024-04-24. > > > > > >> > > > >>>>>> > > > > > >> > > > >>>>>> For context, this was discussed[2] last year on > this > > > > > mailing > > > > > >> > list. > > > > > >> > > > We > > > > > >> > > > >>>>>> decided to revisit the discussion around the June > > 2024 > > > > > >> release > > > > > >> > > > (Arrow > > > > > >> > > > >>>>> v17). > > > > > >> > > > >>>>>> The timing coincides with the initial release of > > Apache > > > > > Spark > > > > > >> > > 4.0.0, > > > > > >> > > > >>>>> which > > > > > >> > > > >>>>>> drops both Java 8 and 11 support. > > > > > >> > > > >>>>>> > > > > > >> > > > >>>>>> For background, we chose not to drop Java 8 support > > > last > > > > > year > > > > > >> > > > because > > > > > >> > > > >>>>> Arrow > > > > > >> > > > >>>>>> is seen as a low level library that should support > as > > > > many > > > > > >> > > > >> environments > > > > > >> > > > >>>>> as > > > > > >> > > > >>>>>> possible. Nowadays, we see more enthusiasm for > > dropping > > > > > Java > > > > > >> 8 > > > > > >> > > (and > > > > > >> > > > >> 11) > > > > > >> > > > >>>>> as > > > > > >> > > > >>>>>> exemplified by Apache Spark as well as Apache > > > Iceberg[3]. > > > > > >> > > > >>>>>> > > > > > >> > > > >>>>>> Is it time to consider dropping Java 8? Should we > > drop > > > > Java > > > > > >> 11 > > > > > >> > and > > > > > >> > > > >> skip > > > > > >> > > > >>>>>> straight to Java 17 as our minimum version? What > > > > > >> implications do > > > > > >> > > we > > > > > >> > > > >>>> need > > > > > >> > > > >>>>> to > > > > > >> > > > >>>>>> be aware of? > > > > > >> > > > >>>>>> > > > > > >> > > > >>>>>> Thanks, > > > > > >> > > > >>>>>> Dane > > > > > >> > > > >>>>>> > > > > > >> > > > >>>>>> [1]https://github.com/apache/arrow/issues/38051 > > > > > >> > > > >>>>>> [2] > > > > > >> > > > > > https://lists.apache.org/thread/s07jx58yw4mkl54t3bkggnyg0sftcrr8 > > > > > >> > > > >>>>>> [3] > > > > > >> > > > > > https://lists.apache.org/thread/ntrk2thvsg9tdccwd4flsdz9gg743368 > > > > > >> > > > >>>>> > > > > > >> > > > >>>> > > > > > >> > > > >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > > > >
