Thanks all for the discussion here. Based on this I think we'll stick with Java 8 for now and then upgrade to Java 11 around or after Spark 4.
On Thu, Jun 8, 2023, at 07:17, Sean Owen wrote: > Noted, but for that you'd simply run your app on Java 17. If Spark works, and > your app's dependencies work on Java 17 because you compile it for 17 (and > jakarta.* classes for example) then there's no issue. > > On Thu, Jun 8, 2023 at 3:13 AM Martin Andersson <martin.anders...@kambi.com> > wrote: >> There are some reasons to drop Java 11 as well. Java 17 included a large >> change, breaking backwards compatibility with their transition from Java EE >> to Jakarta EE >> <https://blogs.oracle.com/javamagazine/post/transition-from-java-ee-to-jakarta-ee>. >> This means that any users using Spark 4.0 together with Spring 6.x or any >> recent version of servlet containers such as Tomcat or Jetty will experience >> issues. (For security reasons it's beneficial to float your dependencies to >> the latest version of these libraries/frameworks) >> >> I'm not explicitly saying Java 11 should be dropped in Spark 4, just thought >> I'd bring this issue to your attention. >> >> Best Regards, Martin >> >> >> *From:* Jungtaek Lim <kabhwan.opensou...@gmail.com> >> *Sent:* Wednesday, June 7, 2023 23:19 >> *To:* Sean Owen <sro...@gmail.com> >> *Cc:* Dongjoon Hyun <dongj...@apache.org>; Holden Karau >> <hol...@pigscanfly.ca>; dev <dev@spark.apache.org> >> *Subject:* Re: JDK version support policy? >> >> >> >> EXTERNAL SENDER. Do not click links or open attachments unless you recognize >> the sender and know the content is safe. DO NOT provide your username or >> password. >> >> >> >> +1 to drop Java 8 but +1 to set the lowest support version to Java 11. >> >> Considering the phase for only security updates, 11 LTS would not be EOLed >> in very long time. Unless that’s coupled with other deps which require >> bumping JDK version (hope someone can bring up lists), it doesn’t seem to >> buy much. And given the strong backward compatibility JDK provides, that’s >> less likely. >> >> Purely from the project’s source code view, does anyone know how much >> benefits we can leverage for picking up 17 rather than 11? I lost the track, >> but some of their proposals are more likely catching up with other >> languages, which don’t make us be happy since Scala provides them for years. >> >> 2023년 6월 8일 (목) 오전 2:35, Sean Owen <sro...@gmail.com>님이 작성: >>> I also generally perceive that, after Java 9, there is much less breaking >>> change. So working on Java 11 probably means it works on 20, or can be >>> easily made to without pain. Like I think the tweaks for Java 17 were quite >>> small. >>> >>> Targeting Java >11 excludes Java 11 users and probably wouldn't buy much. >>> Keeping the support probably doesn't interfere with working on much newer >>> JVMs either. >>> >>> On Wed, Jun 7, 2023, 12:29 PM Holden Karau <hol...@pigscanfly.ca> wrote: >>>> So JDK 11 is still supported in open JDK until 2026, I'm not sure if we're >>>> going to see enough folks moving to JRE17 by the Spark 4 release unless we >>>> have a strong benefit from dropping 11 support I'd be inclined to keep it. >>>> >>>> On Tue, Jun 6, 2023 at 9:08 PM Dongjoon Hyun <dongj...@apache.org> wrote: >>>>> I'm also +1 on dropping both Java 8 and 11 in Apache Spark 4.0, too. >>>>> >>>>> Dongjoon. >>>>> >>>>> On 2023/06/07 02:42:19 yangjie01 wrote: >>>>> > +1 on dropping Java 8 in Spark 4.0, and I even hope Spark 4.0 can only >>>>> > support Java 17 and the upcoming Java 21. >>>>> > >>>>> > 发件人: Denny Lee <denny.g....@gmail.com> >>>>> > 日期: 2023年6月7日 星期三 07:10 >>>>> > 收件人: Sean Owen <sro...@gmail.com> >>>>> > 抄送: David Li <lidav...@apache.org>, "dev@spark.apache.org" >>>>> > <dev@spark.apache.org> >>>>> > 主题: Re: JDK version support policy? >>>>> > >>>>> > +1 on dropping Java 8 in Spark 4.0, saying this as a fan of the >>>>> > fast-paced (positive) updates to Arrow, eh?! >>>>> > >>>>> > On Tue, Jun 6, 2023 at 4:02 PM Sean Owen >>>>> > <sro...@gmail.com<mailto:sro...@gmail.com>> wrote: >>>>> > I haven't followed this discussion closely, but I think we could/should >>>>> > drop Java 8 in Spark 4.0, which is up next after 3.5? >>>>> > >>>>> > On Tue, Jun 6, 2023 at 2:44 PM David Li >>>>> > <lidav...@apache.org<mailto:lidav...@apache.org>> wrote: >>>>> > Hello Spark developers, >>>>> > >>>>> > I'm from the Apache Arrow project. We've discussed Java version support >>>>> > [1], and crucially, whether to continue supporting Java 8 or not. As >>>>> > Spark is a big user of Arrow in Java, I was curious what Spark's policy >>>>> > here was. >>>>> > >>>>> > If Spark intends to stay on Java 8, for instance, we may also want to >>>>> > stay on Java 8 or otherwise provide some supported version of Arrow for >>>>> > Java 8. >>>>> > >>>>> > We've seen dependencies dropping or planning to drop support. gRPC may >>>>> > drop Java 8 at any time [2], possibly this September [3], which may >>>>> > affect Spark (due to Spark Connect). And today we saw that Arrow had >>>>> > issues running tests with Mockito on Java 20, but we couldn't update >>>>> > Mockito since it had dropped Java 8 support. (We pinned the JDK version >>>>> > in that CI pipeline for now.) >>>>> > >>>>> > So at least, I am curious if Arrow could start the long process of >>>>> > migrating Java versions without impacting Spark, or if we should >>>>> > continue to cooperate. Arrow Java doesn't see quite so much activity >>>>> > these days, so it's not quite critical, but it's possible that these >>>>> > dependency issues will start to affect us more soon. And looking >>>>> > forward, Java is working on APIs that should also allow us to ditch the >>>>> > --add-opens flag requirement too. >>>>> > >>>>> > [1]: >>>>> > https://lists.apache.org/thread/phpgpydtt3yrgnncdyv4qdq1gf02s0yj<https://mailshield.baidu.com/check?q=Nz%2bGj2hdKguk92URjA7sg0PfbSN%2fXUIMgrHTmW45gOOKEr3Shre45B7TRzhEpb%2baVsnyuRL%2fl%2f0cu7IVGHunSGDVnxM%3d> >>>>> > [2]: >>>>> > https://github.com/grpc/proposal/blob/master/P5-jdk-version-support.md<https://mailshield.baidu.com/check?q=s89S3eo8GCJkV7Mpx7aG1SXId7uCRYGjQMA6DeLuX9duS86LhIODZMJfeFdGMWdFzJ8S7minyHoC7mCrzHagbJXCXYTBH%2fpZBpfTbw%3d%3d> >>>>> > [3]: >>>>> > https://github.com/grpc/grpc-java/issues/9386<https://mailshield.baidu.com/check?q=R0HtWZIkY5eIxpz8jtqHLzd0ugNbcaXIKW2LbUUxpIn0t9Y9yAhuHPuZ4buryfNwRnnJTA%3d%3d> >>>>> > >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >>>> >>>> >>>> -- >>>> Twitter: https://twitter.com/holdenkarau >>>> Books (Learning Spark, High Performance Spark, etc.): >>>> https://amzn.to/2MaRAG9 >>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
