Thanks, Okumin.

I'll build the distro and check whether it's being packaged. If it's
not, I believe it's permissible — but I'll run it by Legal to be
completely sure.
Also, from a quick scan of other Apache repositories, it seems some
projects are using this as well [1].

That said, I'll explore whether we can switch to GraalVM and eliminate
this dependency altogether.
Let me dig a bit deeper and circle back with a clearer plan to address this.

-Ayush

[1] https://github.com/search?q=org%3Aapache+nashorn-core&type=code&p=1

On Sun, 1 Jun 2025 at 16:12, Shohei Okumiya <oku...@apache.org> wrote:
>
> Hi,
>
> Thanks for making it. It's a massive step for us!
>
> I found the compile dependency on "nashorn-core" was newly added,
> which is licensed under "gpl-v2-with-the-classpath-exception-license".
> I'm not confident that we can distribute the jar file on a release.
>
> I'm still testing the master branch with JDK 17. So far, so good.
>
> Best,
> Okumin
>
>
>
> On Sat, May 31, 2025 at 4:02 AM Simhadri G <simhad...@apache.org> wrote:
> >
> > Congratulations guys! That's really awesome!
> >
> > Will test it out and let you know if I run into any bugs!
> >
> > Thanks everyone !
> > Simhadri G
> >
> >
> > On Sat, May 31, 2025, 12:28 AM Ayush Saxena <ayush...@gmail.com> wrote:
> >>
> >> Hi Folks,
> >>
> >> The Hive master branch has now been upgraded to JDK 17. This means
> >> that building or running the latest code from the master branch will
> >> require JDK 17 going forward.
> >> You're encouraged to try it out and share your feedback. If you
> >> encounter any issues, please feel free to report them here or create a
> >> ticket with the details.
> >>
> >> A big thank you to all the contributors who helped make this
> >> transition possible!
> >>
> >> Looking ahead, Hive 4.1.0 is expected to be released by the end of
> >> June or early July, and will officially support JDK 17.
> >>
> >> Best regards,
> >> Ayush
> >>
> >> ---------- Forwarded message ---------
> >> From: Vlad Rozov <vro...@apache.org>
> >> Date: Tue, 6 May 2025 at 20:29
> >> Subject: Re: Move to JDK-11
> >> To: <dev@hive.apache.org>
> >>
> >>
> >> I am looking to upgrade Apache Spark 4.x to use Apache Hive 4.x as
> >> both compile and runtime dependencies. Spark 4.x requires JDK 17 and
> >> while working on upgrade I have not encountered issues caused by Hive
> >> using JDK 8. I will keep an eye and report any, though I don't expect
> >> to see a problem till Spark upgrades to JDK 21.
> >>
> >> Thank you,
> >>
> >> Vlad
> >>
> >> On 2025/04/08 14:27:33 Butao Zhang wrote:
> >> > The upgrade to JDK 17 is a significant change that introduces 
> >> > compatibility and adaptation requirements for both Apache Tez and Apache 
> >> > Hive. Relevant work is currently in progress—thanks to all community 
> >> > contributors who have already been involved in adaptation and fixes. We 
> >> > encourage everyone to actively report any issues encountered during 
> >> > usage or integration, so that we can collaborate on discussions and 
> >> > resolutions, and accelerate the overall transition to JDK 17.
> >> >
> >> > Thanks,
> >> > Butao Zhang
> >> >
> >> > On 2025/04/07 11:25:33 Shohei Okumiya wrote:
> >> > > Hi Ayush and all people working on the JDK upgrade,
> >> > >
> >> > > Thanks for sharing the timeline and detailing the current progress. I'm
> >> > > more than happy to hear the good news and look forward to seeing Hive
> >> > > running on Java 17!
> >> > >
> >> > > Mid-May is not so far away. I don't have a firm preference about how to
> >> > > package 4.1. If anyone is interested, let's discuss it in the thread 
> >> > > for
> >> > > the Hive 4.1.x release.
> >> > >
> >> > > Finally, let me express my gratitude to everyone working on the 
> >> > > challenging
> >> > > upgrade project. It will definitely be one of the most outstanding
> >> > > achievements!
> >> > >
> >> > > Sincerely,
> >> > > Okumin
> >> > >
> >> > > On Mon, Apr 7, 2025 at 7:03 PM Ayush Saxena <ayush...@gmail.com> wrote:
> >> > >
> >> > > > Hi Okumin,
> >> > > >
> >> > > > I know I'm one of the people responsible for misjudging the timeline
> >> > > > around the JDK-17 upgrade, which has ultimately blocked the 4.1.0 
> >> > > > release.
> >> > > > I want to take a moment to share my thoughts and where things 
> >> > > > currently
> >> > > > stand.
> >> > > >
> >> > > > Q: Can we confirm the importance and ETA of the JDK upgrade and how 
> >> > > > we
> >> > > > should proceed?
> >> > > >
> >> > > > Ans: I think we all agree that moving to JDK-17 is a major 
> >> > > > milestone, and
> >> > > > we're admittedly late in adopting it. Many projects have already 
> >> > > > moved or
> >> > > > are actively working towards it. Not being on JDK-17 restricts us 
> >> > > > from
> >> > > > adopting modern third-party libraries like Iceberg and from 
> >> > > > leveraging
> >> > > > several improvements that come with it.
> >> > > >
> >> > > > As for the ETA: I realize I previously gave an inaccurate timeline, 
> >> > > > and I
> >> > > > sincerely regret that. However, I now have a much clearer picture. 
> >> > > > I'm in
> >> > > > active discussions with folks internally who are working on it. I 
> >> > > > believe
> >> > > > we should have the code ready by mid-May, at which point we can start
> >> > > > gathering feedback from the community.
> >> > > >
> >> > > > Q: We've been working on this upgrade for over half a year. When 
> >> > > > will it
> >> > > > merge? PR: https://github.com/apache/hive/pull/5404
> >> > > > In my opinion, #5404 should be divided into multiple pull requests. I
> >> > > > really understand we want to test simultaneously rather than make
> >> > > > incremental progress. However, the consolidated PR makes it 
> >> > > > impossible for
> >> > > > other developers to help the initiative, reviewers have to check huge
> >> > > > changes, and authors must occasionally resolve many conflicts. I 
> >> > > > don't
> >> > > > prefer any HOW as long as the delivery is predictable, meaning it is 
> >> > > > okay
> >> > > > for me if we are confident with the agreed ETA. If it is still 
> >> > > > uncertain,
> >> > > > we may try another project management, which is 
> >> > > > incremental/predictable and
> >> > > > where we can invest other developers' resources in it.
> >> > > >
> >> > > > Ans: I completely agree with your observations. The original PR 
> >> > > > started
> >> > > > more as an experiment to see what changes were needed. At the time, 
> >> > > > I don’t
> >> > > > think anyone anticipated how large it would become, and 
> >> > > > unfortunately, it
> >> > > > kept growing incrementally over time.
> >> > > > That said, I’ve already asked the team to start breaking down the PR 
> >> > > > into
> >> > > > smaller, more manageable parts. Dependency upgrades will be 
> >> > > > separated out
> >> > > > soon, and changes like those related to SpotBugs (which account for 
> >> > > > a large
> >> > > > number of line changes) will also be split. The goal is to make the 
> >> > > > PR
> >> > > > easier to review and more accessible for other contributors to jump 
> >> > > > in. I
> >> > > > understand this is a genuine blocker, and folks are  doing their 
> >> > > > best to
> >> > > > address it.
> >> > > >
> >> > > > Q: [1] Should we ship 4.1 without JDK-17, given it's been a long time
> >> > > > since 4.0.1?
> >> > > >
> >> > > > Ans: Honestly, I’m not in a position to oppose this. The community 
> >> > > > kindly
> >> > > > respected my request last time when I asked for more time for the 
> >> > > > JDK-17
> >> > > > upgrade, and unfortunately, I couldn’t follow through as expected. 
> >> > > > So, with
> >> > > > sincere apologies for being one of the reasons for the delay, I’m 
> >> > > > okay with
> >> > > > whatever direction the community chooses to take from here.
> >> > > >
> >> > > > -Ayush
> >> > > >
> >> > > >
> >> > > > On Sun, 6 Apr 2025 at 11:55, Shohei Okumiya <oku...@apache.org> 
> >> > > > wrote:
> >> > > >
> >> > > >> Hi,
> >> > > >>
> >> > > >> I appreciate everyone working on this big and challenging project!
> >> > > >>
> >> > > >> Can we confirm the importance and ETA of the JDK upgrade and how we
> >> > > >> should work on it?
> >> > > >>
> >> > > >> I recognize it is one of the most essential initiatives blocking the
> >> > > >> release of Hive 4.1.0[1] and the upgrade of Iceberg libraries[2]. 
> >> > > >> Users and
> >> > > >> developers would expect it to be shipped soon.
> >> > > >>
> >> > > >> We have to complete this effort to ship JDK 17. We've been working 
> >> > > >> on it
> >> > > >> for over half a year. Some of us might be interested in when we 
> >> > > >> will merge
> >> > > >> it.
> >> > > >> https://github.com/apache/hive/pull/5404
> >> > > >>
> >> > > >> In my opinion, #5404 should be divided into multiple pull requests. 
> >> > > >> I
> >> > > >> really understand we want to test simultaneously rather than make
> >> > > >> incremental progress. However, the consolidated PR makes it 
> >> > > >> impossible for
> >> > > >> other developers to help the initiative, reviewers have to check 
> >> > > >> huge
> >> > > >> changes, and authors must occasionally resolve many conflicts. I 
> >> > > >> don't
> >> > > >> prefer any HOW as long as the delivery is predictable, meaning it 
> >> > > >> is okay
> >> > > >> for me if we are confident with the agreed ETA. If it is still 
> >> > > >> uncertain,
> >> > > >> we may try another project management, which is 
> >> > > >> incremental/predictable and
> >> > > >> where we can invest other developers' resources in it.
> >> > > >>
> >> > > >> - [1] Honestly, I feel we should ship 4.1 without JDK 17, as it's 
> >> > > >> been a
> >> > > >> long time since the last release of 4.0.1
> >> > > >> - [2] Iceberg 1.7 or later doesn't support Java 8
> >> > > >>
> >> > > >> Best,
> >> > > >> Okumin
> >> > > >>
> >> > > >> On Wed, Dec 11, 2024 at 2:51 AM Ayush Saxena <ayush...@gmail.com> 
> >> > > >> wrote:
> >> > > >>
> >> > > >>> Regarding JDK-11, we didn't actively chased that, so its status 
> >> > > >>> remains
> >> > > >>> uncertain. With lower compile-time support like JDK-8, we can 
> >> > > >>> certainly
> >> > > >>> ensure runtime support for both JDK-11 and JDK-17, which I believe 
> >> > > >>> should
> >> > > >>> work for Hive 4.0.1. As far as I know, this approach works 
> >> > > >>> downstream for
> >> > > >>> us atleast.
> >> > > >>>
> >> > > >>> However, maintaining compile-time compatibility across JDK-17, 
> >> > > >>> JDK-11,
> >> > > >>> and JDK-8 simultaneously may not be feasible. JDK-17 introduces 
> >> > > >>> significant
> >> > > >>> changes, and I don’t believe it’s naive—or even possible—to 
> >> > > >>> compile Hive
> >> > > >>> using both JDK-8 and JDK-17 & expect things to work properly.
> >> > > >>>
> >> > > >>> The initial plan appeared to be supporting JDK-8 for Hive 4.0.x 
> >> > > >>> release
> >> > > >>> line, with potential tweaks if required to enable runtime support 
> >> > > >>> for
> >> > > >>> JDK-17. Starting with Hive 4.1.x, we could shift to JDK-17 for 
> >> > > >>> compile-time
> >> > > >>> support, and later evaluate the feasibility of supporting JDK-21 
> >> > > >>> or newer
> >> > > >>> versions.
> >> > > >>>
> >> > > >>> Once we see a state when nobody want JDK-8 compile time support &
> >> > > >>> everyone has moved to JDK-17 or say using 4.1.x release line, We 
> >> > > >>> can then
> >> > > >>> drop having bug fix releases for 4.0.x
> >> > > >>>
> >> > > >>> -Ayush
> >> > > >>>
> >> > > >>> On Mon, 9 Dec 2024 at 15:11, Butao Zhang <butaozha...@163.com> 
> >> > > >>> wrote:
> >> > > >>>
> >> > > >>>> IMO, I think there two challenges if we want to maintain JDK8 & 
> >> > > >>>> JDK11 &
> >> > > >>>> JDK17 at the same time.
> >> > > >>>>
> >> > > >>>>    - Different jdk syntactic sugar between multiple jdks, and 
> >> > > >>>> advanced
> >> > > >>>>    feats in jdk17&jdk21, such as jdk vector api. If we want the 
> >> > > >>>> multiple jdks,
> >> > > >>>>    we can not use the advanced feats in high jdk in the future. 
> >> > > >>>> Such as the
> >> > > >>>>    discussion in https://issues.apache.org/jira/browse/HIVE-28614
> >> > > >>>>    - We need to run the jenkins CI process multiple times to run 
> >> > > >>>> the
> >> > > >>>>    multiple jdk env. I'm not sure we have enough jenkins 
> >> > > >>>> resources to do this.
> >> > > >>>>    However, i want to say if we can use github actions to run the 
> >> > > >>>> whole CI? I
> >> > > >>>>    have seen Apache Spark have done the good job which using the 
> >> > > >>>> forked
> >> > > >>>>    repository github actions to run the CI.  FRY
> >> > > >>>>    
> >> > > >>>> https://lists.apache.org/thread/dmv1h2snqc7qtdjlzj4k8qxookclp720
> >> > > >>>>
> >> > > >>>>
> >> > > >>>> Thanks,
> >> > > >>>> Butao Zhang
> >> > > >>>> ---- Replied Message ----
> >> > > >>>> From Shohei Okumiya<oku...@apache.org> <undefined>
> >> > > >>>> Date 12/9/2024 17:06
> >> > > >>>> To <dev@hive.apache.org> <dev@hive.apache.org>
> >> > > >>>> Subject Re: Move to JDK-11
> >> > > >>>> Hi,
> >> > > >>>>
> >> > > >>>> I appreciate everyone's work on the JDK upgrade. It would be 
> >> > > >>>> ideal if
> >> > > >>>> we could migrate JDK like this. That will give us an
> >> > > >>>> incremental migration path.
> >> > > >>>>
> >> > > >>>> 1. (now) Hive 4.0 compiled by JDK 8, and run by JDK 8
> >> > > >>>> 2. Hive 4.x compiled by JDK 8, and run by JDK 17
> >> > > >>>> 3. Hive 4.x compiled by JDK 17, and run by JDK 17
> >> > > >>>>
> >> > > >>>> I guess it is technically possible that Hive supports JDK 8 and 
> >> > > >>>> 17 at
> >> > > >>>> runtime. Apache Spark supports JDK 8, 11, and 17. What is the most
> >> > > >>>> challenging point?
> >> > > >>>>
> >> > > >>>> Regards,
> >> > > >>>> Okumin
> >> > > >>>>
> >> > > >>>> On Thu, Aug 29, 2024 at 10:48 PM Butao Zhang <butaozha...@163.com>
> >> > > >>>> wrote:
> >> > > >>>>
> >> > > >>>>> Hi Akshat,
> >> > > >>>>>
> >> > > >>>>> Thanks for your reply! I agree that supporting both JDK 8 and 
> >> > > >>>>> JDK 17
> >> > > >>>>> is not easy. I think we can safely to move to JDK 17 if we can 
> >> > > >>>>> ensure *hive
> >> > > >>>>> application* can access to HiveServer2/Metastore without forcing 
> >> > > >>>>>  to
> >> > > >>>>> use the JDK 17.
> >> > > >>>>> For example, if a hive application is developed by JDK 8, it can
> >> > > >>>>> import the *hive-exec.jar & hive-jdbc.jar* which is compiled by 
> >> > > >>>>> JDK
> >> > > >>>>> 17 and can access to hs2/hms successfully.  If this case can be 
> >> > > >>>>> supported,
> >> > > >>>>> I think we can move to JDK 17 as soon as possible.
> >> > > >>>>>
> >> > > >>>>> BTW, If Apache Hive , Apache Tez and Apache Hadoop are all 
> >> > > >>>>> compiled
> >> > > >>>>> with JDK17 or higer version, i think the hive job will run more 
> >> > > >>>>> faster.
> >> > > >>>>> Users will enjoy significant performance improvements due to JDK 
> >> > > >>>>> version
> >> > > >>>>> upgrades.
> >> > > >>>>> I'm looking forward to the performance improvement. :)
> >> > > >>>>>
> >> > > >>>>> Thanks,
> >> > > >>>>> Butao Zhang
> >> > > >>>>> ---- Replied Message ----
> >> > > >>>>> From Akshat m<akshatats...@gmail.com> <undefined>
> >> > > >>>>> Date 8/29/2024 18:44
> >> > > >>>>> To <dev@hive.apache.org> <dev@hive.apache.org>
> >> > > >>>>> Subject Re: Move to JDK-11
> >> > > >>>>> Hi Butao,
> >> > > >>>>>
> >> > > >>>>> It won’t be feasible to support both JDK 8 and JDK 17 in the 
> >> > > >>>>> same Hive
> >> > > >>>>> version due to compatibility issues and behavior changes, as 
> >> > > >>>>> detailed in
> >> > > >>>>> the PR. The internal changes in JDK implementations have made 
> >> > > >>>>> this
> >> > > >>>>> challenging. Additionally, many other libraries maintain 
> >> > > >>>>> separate versions
> >> > > >>>>> for JDK 17+ and JDK 8 support.
> >> > > >>>>>
> >> > > >>>>> To align with this approach, I suggest we continue with Hive 4
> >> > > >>>>> supporting JDK 8. Once the JDK 17 PR is merged, we could plan a 
> >> > > >>>>> separate
> >> > > >>>>> release that specifically targets JDK 17+ support. This would 
> >> > > >>>>> ensure that
> >> > > >>>>> Hive stays up-to-date while maintaining stability across 
> >> > > >>>>> different
> >> > > >>>>> environments.
> >> > > >>>>>
> >> > > >>>>> Would love to hear your thoughts or any additional suggestions 
> >> > > >>>>> the
> >> > > >>>>> community might have!
> >> > > >>>>>
> >> > > >>>>> Regards
> >> > > >>>>>
> >> > > >>>>> Akshat
> >> > > >>>>>
> >> > > >>>>>
> >> > > >>>>>
> >> > > >>>>> On Wed, Aug 28, 2024 at 8:00 AM Butao Zhang <butaozha...@163.com>
> >> > > >>>>> wrote:
> >> > > >>>>>
> >> > > >>>>>> Good job! Thanks everyone who put effort into the JDK upgrading.
> >> > > >>>>>> That's a big good news for Apache Hive community!
> >> > > >>>>>> Just a question: Given that many people are still using the 
> >> > > >>>>>> lower
> >> > > >>>>>> jdk8 version. Do we consider to keeping compatibility with 
> >> > > >>>>>> lower jdk8
> >> > > >>>>>> version?
> >> > > >>>>>>
> >> > > >>>>>>
> >> > > >>>>>> Thanks,
> >> > > >>>>>> Butao Zhang
> >> > > >>>>>> ---- Replied Message ----
> >> > > >>>>>> From Akshat m<akshatats...@gmail.com> <undefined>
> >> > > >>>>>> Date 8/27/2024 23:12
> >> > > >>>>>> To dev<dev@hive.apache.org> <dev@hive.apache.org>
> >> > > >>>>>> Cc kokila narayanan<kokilanarayana...@gmail.com>,
> >> > > >>>>>> <kokilanarayana...@gmail.com>Raghav
> >> > > >>>>>> Aggarwal<raghavaggarwal03...@gmail.com>,
> >> > > >>>>>> <raghavaggarwal03...@gmail.com>Ayush Saxena<ayush...@gmail.com>,
> >> > > >>>>>> <ayush...@gmail.com>tanishq chugh<tanishqchugh4...@gmail.com>,
> >> > > >>>>>> <tanishqchugh4...@gmail.com><aturo...@cloudera.com>
> >> > > >>>>>> <aturo...@cloudera.com>
> >> > > >>>>>> Subject Re: Move to JDK-11
> >> > > >>>>>>
> >> > > >>>>>> Hi everyone,
> >> > > >>>>>>
> >> > > >>>>>> I’m excited to share that after months of dedicated work, we’ve
> >> > > >>>>>> finally achieved a passing PR[1] for running *Apache Hive on 
> >> > > >>>>>> JDK 17*!
> >> > > >>>>>>
> >> > > >>>>>> This was a challenging task, requiring to address various
> >> > > >>>>>> compatibility issues and ensure our comprehensive test suite 
> >> > > >>>>>> passed without
> >> > > >>>>>> a hitch.
> >> > > >>>>>>
> >> > > >>>>>> Thanks Kokila and Tanishq for joining me in the initiative and 
> >> > > >>>>>> Raghav
> >> > > >>>>>> for contributing fix for the errorProne issue, and thanks to 
> >> > > >>>>>> Attila, Ayush
> >> > > >>>>>> and other Hive PMC members for their guidance in the initiative
> >> > > >>>>>>
> >> > > >>>>>> It’s been a real team effort, and I’m grateful for the 
> >> > > >>>>>> collaboration
> >> > > >>>>>> and support from everyone involved.
> >> > > >>>>>>
> >> > > >>>>>> I’d appreciate it if you could take some time to review the PR.
> >> > > >>>>>> Community feedback is really important to make this happen.
> >> > > >>>>>>
> >> > > >>>>>> The PR #5404 addresses the integration of Apache Hive with JDK 
> >> > > >>>>>> 17,
> >> > > >>>>>> introducing a range of changes aimed at ensuring compatibility 
> >> > > >>>>>> and
> >> > > >>>>>> improving the codebase. Here’s a detailed breakdown:
> >> > > >>>>>>
> >> > > >>>>>> 1. Core Code Adjustments:
> >> > > >>>>>>
> >> > > >>>>>>    - Compatibility Fixes: Various parts of the Hive codebase 
> >> > > >>>>>> have
> >> > > >>>>>> been modified to ensure they work seamlessly with JDK 17. This 
> >> > > >>>>>> includes
> >> > > >>>>>> addressing language or API changes that JDK 17 introduces.
> >> > > >>>>>>
> >> > > >>>>>>    - Reflection and Modules: Adjustments related to how Hive
> >> > > >>>>>> interacts with Java's reflection API, which has stricter module 
> >> > > >>>>>> system
> >> > > >>>>>> rules in JDK 17.
> >> > > >>>>>>
> >> > > >>>>>> 2. Dependency Management:
> >> > > >>>>>>
> >> > > >>>>>>    - Version Upgrades: Several third-party dependencies have 
> >> > > >>>>>> been
> >> > > >>>>>> updated to versions that are compatible with JDK 17. This is 
> >> > > >>>>>> critical
> >> > > >>>>>> because older versions of some libraries may not work properly 
> >> > > >>>>>> or might use
> >> > > >>>>>> deprecated features removed in JDK 17.
> >> > > >>>>>>
> >> > > >>>>>>    - Data nucleus upgrade: To ensure compatibility with Java 
> >> > > >>>>>> 17, we
> >> > > >>>>>> upgraded DataNucleus to version 6.x. This upgrade, however, 
> >> > > >>>>>> introduced an
> >> > > >>>>>> issue with Hive schema creation. The issue stems from a commit 
> >> > > >>>>>> in
> >> > > >>>>>> DataNucleus[2] that alters the handling of primary key column 
> >> > > >>>>>> names by
> >> > > >>>>>> enclosing them in double quotes. This change caused conflicts 
> >> > > >>>>>> during Hive
> >> > > >>>>>> schema generation. As an interim solution, we implemented a 
> >> > > >>>>>> patched
> >> > > >>>>>> DataNucleus-rdbms module to address the issue in Hive. The 
> >> > > >>>>>> necessary fix is
> >> > > >>>>>> now included in the DataNucleus repository[3]. Once DataNucleus 
> >> > > >>>>>> 6.0.8 is
> >> > > >>>>>> officially released, we will upgrade to this version and remove 
> >> > > >>>>>> the
> >> > > >>>>>> temporary patched module.
> >> > > >>>>>>
> >> > > >>>>>>  - Dependency Cleanup: Unused or outdated dependencies were 
> >> > > >>>>>> removed,
> >> > > >>>>>> reducing potential conflicts and simplifying the build process.
> >> > > >>>>>>
> >> > > >>>>>> 3. Testing Enhancements:
> >> > > >>>>>>
> >> > > >>>>>>  - Test Suite Updates: The PR includes updates to Hive’s test 
> >> > > >>>>>> suite
> >> > > >>>>>> to ensure all tests pass under JDK 17. This likely involved 
> >> > > >>>>>> modifying tests
> >> > > >>>>>> that needed adjustments to align with JDK 17’s behaviour.
> >> > > >>>>>>
> >> > > >>>>>>  - New Tests: New tests may have been added to cover areas 
> >> > > >>>>>> where JDK
> >> > > >>>>>> 17 introduced new features or changes that could impact Hive.
> >> > > >>>>>>
> >> > > >>>>>> 4. Compatibility Checks:
> >> > > >>>>>>
> >> > > >>>>>>  - JDK Version Verification: A new compatibility check has been
> >> > > >>>>>> introduced to verify the JDK version at runtime. This ensures 
> >> > > >>>>>> that Hive is
> >> > > >>>>>> running on an appropriate JDK version, preventing unexpected 
> >> > > >>>>>> errors or
> >> > > >>>>>> behaviour due to version mismatches.
> >> > > >>>>>>
> >> > > >>>>>> 5. Build Updates:
> >> > > >>>>>>
> >> > > >>>>>>  - Build System Tweaks: The PR likely includes changes to the 
> >> > > >>>>>> Maven
> >> > > >>>>>> build configuration to accommodate JDK 17, ensuring that the 
> >> > > >>>>>> build process
> >> > > >>>>>> is smooth and compatible.
> >> > > >>>>>>
> >> > > >>>>>> Overall, this PR is a comprehensive effort to bring Apache Hive 
> >> > > >>>>>> up to
> >> > > >>>>>> date with the latest Java platform, ensuring long-term 
> >> > > >>>>>> maintainability and
> >> > > >>>>>> performance improvements.
> >> > > >>>>>>
> >> > > >>>>>> As the next steps, we’ll be focusing on further sanity checks 
> >> > > >>>>>> and
> >> > > >>>>>> extensive testing to make sure everything functions smoothly. 
> >> > > >>>>>> Community
> >> > > >>>>>> involvement in this process will be crucial, so please do share 
> >> > > >>>>>> any issues
> >> > > >>>>>> or feedback as you test the changes.
> >> > > >>>>>>
> >> > > >>>>>> I’m looking forward to seeing the benefits this will bring to 
> >> > > >>>>>> our
> >> > > >>>>>> community and users.
> >> > > >>>>>>
> >> > > >>>>>>
> >> > > >>>>>> [1] https://github.com/apache/hive/pull/5404
> >> > > >>>>>>
> >> > > >>>>>> [2]
> >> > > >>>>>> https://github.com/datanucleus/datanucleus-rdbms/commit/b99df3c803862f9e9feacc7891461bcb21160b38
> >> > > >>>>>>
> >> > > >>>>>> [3]
> >> > > >>>>>> https://github.com/datanucleus/datanucleus-rdbms/commit/b99df3c803862f9e9feacc7891461bcb21160b38
> >> > > >>>>>>
> >> > > >>>>>>
> >> > > >>>>>> Best regards,
> >> > > >>>>>>
> >> > > >>>>>> Akshat Mathur
> >> > > >>>>>>
> >> > > >>>>>>
> >> > > >>>>>> On Tue, 11 Jul 2023 at 2:25 PM, Attila Turoczy <
> >> > > >>>>>> aturo...@cloudera.com.invalid> wrote:
> >> > > >>>>>>
> >> > > >>>>>>> Returning to this topic, I kindly request those who would like 
> >> > > >>>>>>> to
> >> > > >>>>>>> advocate
> >> > > >>>>>>> for the continued support of JDK8 to please share their 
> >> > > >>>>>>> reasoning and
> >> > > >>>>>>> insights with us. Your input and perspective are greatly 
> >> > > >>>>>>> appreciated!
> >> > > >>>>>>> Thank you.
> >> > > >>>>>>>
> >> > > >>>>>>> -Attila
> >> > > >>>>>>>
> >> > > >>>>>>> On Fri, Jun 2, 2023 at 12:43 PM Attila Turoczy <
> >> > > >>>>>>> aturo...@cloudera.com>
> >> > > >>>>>>> wrote:
> >> > > >>>>>>>
> >> > > >>>>>>> > Hi All,
> >> > > >>>>>>> >
> >> > > >>>>>>> > I know my opinion might not be the most popular, but I 
> >> > > >>>>>>> > advocate
> >> > > >>>>>>> for using*
> >> > > >>>>>>> > JDK 17*. Here's why:
> >> > > >>>>>>> >
> >> > > >>>>>>> > Let's consider a scenario where a customer wants to use the 
> >> > > >>>>>>> > latest
> >> > > >>>>>>> version
> >> > > >>>>>>> > of Apache Hive. They would typically install it locally or 
> >> > > >>>>>>> > on a
> >> > > >>>>>>> small
> >> > > >>>>>>> > cluster. In 2023, is it realistic to assume that this 
> >> > > >>>>>>> > customer
> >> > > >>>>>>> won't be
> >> > > >>>>>>> > able to install JDK 17 on their cluster? Even in large
> >> > > >>>>>>> enterprises, it
> >> > > >>>>>>> > should be feasible to install an LTS JDK, especially 
> >> > > >>>>>>> > considering
> >> > > >>>>>>> the
> >> > > >>>>>>> > widespread adoption of cloud computing. Sungwoo Park's
> >> > > >>>>>>> measurements also
> >> > > >>>>>>> > support this recommendation to go with JDK 17. It 
> >> > > >>>>>>> > outperforms JDK
> >> > > >>>>>>> 11 by 8%
> >> > > >>>>>>> > in terms of runtime speed, and JDK 11 itself is 10+% faster 
> >> > > >>>>>>> > than
> >> > > >>>>>>> JDK 8.
> >> > > >>>>>>> > This is a significant value proposition. Who would be the 
> >> > > >>>>>>> > customer
> >> > > >>>>>>> that
> >> > > >>>>>>> > says, "I don't want faster query execution! I'd rather use 
> >> > > >>>>>>> > JDK 8
> >> > > >>>>>>> and pay
> >> > > >>>>>>> > more for cloud or data center resources instead of using JDK 
> >> > > >>>>>>> > 17!"
> >> > > >>>>>>> It
> >> > > >>>>>>> > doesn't make sense to me.
> >> > > >>>>>>> >
> >> > > >>>>>>> > The tech industry has been evolving at an incredible pace, 
> >> > > >>>>>>> > with
> >> > > >>>>>>> > improvements in serialization, IPC mechanisms, and 
> >> > > >>>>>>> > parallelized
> >> > > >>>>>>> frameworks
> >> > > >>>>>>> > since the release of JDK 8 ten years ago. We should leverage 
> >> > > >>>>>>> > these
> >> > > >>>>>>> > advancements! Couple years ago, we invested a lot to improve 
> >> > > >>>>>>> > 1-2%
> >> > > >>>>>>> of the
> >> > > >>>>>>> > execution. We prayed for 3 gods, sacrificed 2 ships and 
> >> > > >>>>>>> > traveled
> >> > > >>>>>>> around the
> >> > > >>>>>>> > world to make it happen. :-) Now, the JDK itself provides a
> >> > > >>>>>>> substantial
> >> > > >>>>>>> > amount of improvement. So, why would we resist progress just
> >> > > >>>>>>> because there
> >> > > >>>>>>> > are a few lazy or conservative admins who don't want to 
> >> > > >>>>>>> > spend two
> >> > > >>>>>>> minutes
> >> > > >>>>>>> > installing a JDK?
> >> > > >>>>>>> >
> >> > > >>>>>>> > A platform needs to be modern and incorporate the latest
> >> > > >>>>>>> technologies to
> >> > > >>>>>>> > attract developers and users. I understand that some may 
> >> > > >>>>>>> > prefer to
> >> > > >>>>>>> stay
> >> > > >>>>>>> > with JDK 8 as it seems like the safest position, but I 
> >> > > >>>>>>> > believe in
> >> > > >>>>>>> taking
> >> > > >>>>>>> > bold bets to achieve big wins. Even if we decide to stick 
> >> > > >>>>>>> > with JDK
> >> > > >>>>>>> 8, I
> >> > > >>>>>>> > would still be happy since we are moving forward and not 
> >> > > >>>>>>> > dwelling
> >> > > >>>>>>> on a JDK
> >> > > >>>>>>> > that is a decade old. Personally, I think focusing on one 
> >> > > >>>>>>> > thing
> >> > > >>>>>>> that brings
> >> > > >>>>>>> > more value to us and our users is the idealistic path 
> >> > > >>>>>>> > forward.
> >> > > >>>>>>> >
> >> > > >>>>>>> > -Attila
> >> > > >>>>>>> >
> >> > > >>>>>>> >
> >> > > >>>>>>> >
> >> > > >>>>>>> >
> >> > > >>>>>>> >
> >> > > >>>>>>> >
> >> > > >>>>>>> >
> >> > > >>>>>>> >
> >> > > >>>>>>> >
> >> > > >>>>>>> > On Thu, Jun 1, 2023 at 11:23 AM Stamatis Zampetakis <
> >> > > >>>>>>> zabe...@gmail.com>
> >> > > >>>>>>> > wrote:
> >> > > >>>>>>> >
> >> > > >>>>>>> >> Hey everyone,
> >> > > >>>>>>> >>
> >> > > >>>>>>> >> If we claim that Hive supports a certain JDK then we should
> >> > > >>>>>>> compile and
> >> > > >>>>>>> >> run
> >> > > >>>>>>> >> tests with it.
> >> > > >>>>>>> >>
> >> > > >>>>>>> >> The more JDKs we can support the better for everyone but 
> >> > > >>>>>>> >> this
> >> > > >>>>>>> comes at a
> >> > > >>>>>>> >> cost (resources mostly). We should have a precommit run for 
> >> > > >>>>>>> >> every
> >> > > >>>>>>> >> supported
> >> > > >>>>>>> >> JDK (frequency to be determined once per day/week) that 
> >> > > >>>>>>> >> compiles
> >> > > >>>>>>> and run
> >> > > >>>>>>> >> all tests.
> >> > > >>>>>>> >>
> >> > > >>>>>>> >> From my perspective, I would be pretty happy if we could 
> >> > > >>>>>>> >> cover
> >> > > >>>>>>> the two
> >> > > >>>>>>> >> edge
> >> > > >>>>>>> >> LTS releases at every point in time.
> >> > > >>>>>>> >>
> >> > > >>>>>>> >> Then we have to decide also which JDK shall we use for the 
> >> > > >>>>>>> >> pull
> >> > > >>>>>>> requests
> >> > > >>>>>>> >> and local dev environment. I think it makes sense to use the
> >> > > >>>>>>> latest.
> >> > > >>>>>>> >> People
> >> > > >>>>>>> >> like working on modern stuff and also it makes sense that 
> >> > > >>>>>>> >> newer
> >> > > >>>>>>> releases
> >> > > >>>>>>> >> will also use newer versions. It would be pretty awkward if
> >> > > >>>>>>> someone wants
> >> > > >>>>>>> >> to use the latest Hive version and it turns out that it can 
> >> > > >>>>>>> >> only
> >> > > >>>>>>> run on
> >> > > >>>>>>> >> JDK8.
> >> > > >>>>>>> >>
> >> > > >>>>>>> >> Best,
> >> > > >>>>>>> >> Stamatis
> >> > > >>>>>>> >>
> >> > > >>>>>>> >> On Thu, Jun 1, 2023, 3:42 AM Sungwoo Park 
> >> > > >>>>>>> >> <glap...@gmail.com>
> >> > > >>>>>>> wrote:
> >> > > >>>>>>> >>
> >> > > >>>>>>> >> > Hi, everyone.
> >> > > >>>>>>> >> >
> >> > > >>>>>>> >> > I have not tested the master branch with Java 11/17 yet, 
> >> > > >>>>>>> >> > but I
> >> > > >>>>>>> would
> >> > > >>>>>>> >> like
> >> > > >>>>>>> >> > to share my experience with testing a fork of branch-3.1 
> >> > > >>>>>>> >> > with
> >> > > >>>>>>> Java 11/17
> >> > > >>>>>>> >> > (as part of developing Hive-MR3), in case that it can be 
> >> > > >>>>>>> >> > useful
> >> > > >>>>>>> for the
> >> > > >>>>>>> >> > discussion. I merged the patches listed in [1] HIVE-22415 
> >> > > >>>>>>> >> > and
> >> > > >>>>>>> updated
> >> > > >>>>>>> >> the
> >> > > >>>>>>> >> > Maven configuration for Java 11.
> >> > > >>>>>>> >> >
> >> > > >>>>>>> >> > 1. Building Hive was fine and I was able to run it with 
> >> > > >>>>>>> >> > Java 11
> >> > > >>>>>>> as well
> >> > > >>>>>>> >> as
> >> > > >>>>>>> >> > Java 17. So, it seems that the work reported in [1] is 
> >> > > >>>>>>> >> > indeed
> >> > > >>>>>>> complete
> >> > > >>>>>>> >> for
> >> > > >>>>>>> >> > upgrading to Java 11 (and Java 17) and getting Hive to 
> >> > > >>>>>>> >> > work.
> >> > > >>>>>>> >> >
> >> > > >>>>>>> >> > 2. However, there was a problem with running tests, so 
> >> > > >>>>>>> >> > this can
> >> > > >>>>>>> be
> >> > > >>>>>>> >> > additional work for upgrading to Java 11.
> >> > > >>>>>>> >> >
> >> > > >>>>>>> >> > 3. For performance, Java 17 gives about 8 percent of 
> >> > > >>>>>>> >> > (free)
> >> > > >>>>>>> performance
> >> > > >>>>>>> >> > improvement. When tested with 10TB TPC-DS, Java 8 takes 
> >> > > >>>>>>> >> > 8074
> >> > > >>>>>>> seconds,
> >> > > >>>>>>> >> > whereas Java 17 takes 7415 seconds. Considering the 
> >> > > >>>>>>> >> > maturity of
> >> > > >>>>>>> Hive, I
> >> > > >>>>>>> >> > think this is not a small improvement because almost every
> >> > > >>>>>>> query gets
> >> > > >>>>>>> >> some
> >> > > >>>>>>> >> > speedup.
> >> > > >>>>>>> >> >
> >> > > >>>>>>> >> > Thanks,
> >> > > >>>>>>> >> >
> >> > > >>>>>>> >> > --- Sungwoo
> >> > > >>>>>>> >> >
> >> > > >>>>>>> >> > [1] https://issues.apache.org/jira/browse/HIVE-22415
> >> > > >>>>>>> >> >
> >> > > >>>>>>> >> >
> >> > > >>>>>>> >> > On Thu, Jun 1, 2023 at 3:53 AM Sai Hemanth Gantasala
> >> > > >>>>>>> >> > <saihema...@cloudera.com.invalid> wrote:
> >> > > >>>>>>> >> >
> >> > > >>>>>>> >> > > Hi All,
> >> > > >>>>>>> >> > >
> >> > > >>>>>>> >> > > I would strongly advocate keeping support for JDK8.
> >> > > >>>>>>> >> > > Between JDK11 and JDK17, Depending on the amount of 
> >> > > >>>>>>> >> > > effort on
> >> > > >>>>>>> the
> >> > > >>>>>>> >> upgrade
> >> > > >>>>>>> >> > > I'm inclined towards JDK17 (JDK21 LTS will be released 
> >> > > >>>>>>> >> > > in Sep
> >> > > >>>>>>> 2023).
> >> > > >>>>>>> >> > >
> >> > > >>>>>>> >> > > Thanks,
> >> > > >>>>>>> >> > > Sai.
> >> > > >>>>>>> >> > >
> >> > > >>>>>>> >> > > On Wed, May 31, 2023 at 5:39 AM László Bodor <
> >> > > >>>>>>> >> bodorlaszlo0...@gmail.com>
> >> > > >>>>>>> >> > > wrote:
> >> > > >>>>>>> >> > >
> >> > > >>>>>>> >> > > > *Hi!*
> >> > > >>>>>>> >> > > >
> >> > > >>>>>>> >> > > >
> >> > > >>>>>>> >> > > > *Should we support both JDK-11 & JDK-8?*
> >> > > >>>>>>> >> > > > IMO absolutely yes, let's not break up with JDK-8:
> >> > > >>>>>>> according to its
> >> > > >>>>>>> >> > > > lifecycle, it's going to stay with us for a long time.
> >> > > >>>>>>> >> > > >
> >> > > >>>>>>> >> > > > I believe
> >> > > >>>>>>> >> > > > a) we should be able to compile on JDK8, JDK11, and 
> >> > > >>>>>>> >> > > > JDK17
> >> > > >>>>>>> (github
> >> > > >>>>>>> >> > actions
> >> > > >>>>>>> >> > > > can cover this conveniently in precommit time, like 
> >> > > >>>>>>> >> > > > tez
> >> > > >>>>>>> >> > > > <
> >> > > >>>>>>> >>
> >> > > >>>>>>> https://github.com/apache/tez/blob/master/.github/workflows/build.yml
> >> > > >>>>>>> >> > >)
> >> > > >>>>>>> >> > > > b) the release artifacts should be compatible with 
> >> > > >>>>>>> >> > > > JDK8 as
> >> > > >>>>>>> long as
> >> > > >>>>>>> >> it
> >> > > >>>>>>> >> > is
> >> > > >>>>>>> >> > > > with us.
> >> > > >>>>>>> >> > > >
> >> > > >>>>>>> >> > > > Regards,
> >> > > >>>>>>> >> > > > Laszlo Bodor
> >> > > >>>>>>> >> > > >
> >> > > >>>>>>> >> > > >
> >> > > >>>>>>> >> > > > Butao Zhang <butaozha...@163.com> ezt írta (időpont: 
> >> > > >>>>>>> >> > > > 2023.
> >> > > >>>>>>> máj.
> >> > > >>>>>>> >> 31.,
> >> > > >>>>>>> >> > > Sze,
> >> > > >>>>>>> >> > > > 14:33):
> >> > > >>>>>>> >> > > >
> >> > > >>>>>>> >> > > > > Thanks Ayush for driving this! Good to know that 
> >> > > >>>>>>> >> > > > > Hive is
> >> > > >>>>>>> getting
> >> > > >>>>>>> >> > ready
> >> > > >>>>>>> >> > > > for
> >> > > >>>>>>> >> > > > > newer JDK.
> >> > > >>>>>>> >> > > > > From my opinon, if we have more community energy to 
> >> > > >>>>>>> >> > > > > put
> >> > > >>>>>>> into it,
> >> > > >>>>>>> >> we
> >> > > >>>>>>> >> > can
> >> > > >>>>>>> >> > > > > support both JDK-11 and JDK-17 like Spark[1]. If we 
> >> > > >>>>>>> >> > > > > have
> >> > > >>>>>>> to  make
> >> > > >>>>>>> >> a
> >> > > >>>>>>> >> > > > choice
> >> > > >>>>>>> >> > > > > between a JDK-11 and JDK-17, i would like to choose 
> >> > > >>>>>>> >> > > > > the
> >> > > >>>>>>> relatively
> >> > > >>>>>>> >> > new
> >> > > >>>>>>> >> > > > > version JDK-17, meanwhile, we should maintain
> >> > > >>>>>>> compatibility with
> >> > > >>>>>>> >> > jdk8,
> >> > > >>>>>>> >> > > as
> >> > > >>>>>>> >> > > > > JDK-8 is still widely used in most big data 
> >> > > >>>>>>> >> > > > > platforms.
> >> > > >>>>>>> >> > > > >
> >> > > >>>>>>> >> > > > >
> >> > > >>>>>>> >> > > > > Thanks,
> >> > > >>>>>>> >> > > > > Butao Zhang
> >> > > >>>>>>> >> > > > >
> >> > > >>>>>>> >> > > > >
> >> > > >>>>>>> >> > > > > [1]https://issues.apache.org/jira/browse/SPARK-33772
> >> > > >>>>>>> >> > > > > ---- Replied Message ----
> >> > > >>>>>>> >> > > > > | From | Ayush Saxena<ayush...@gmail.com> |
> >> > > >>>>>>> >> > > > > | Date | 5/31/2023 18:39 |
> >> > > >>>>>>> >> > > > > | To | dev<dev@hive.apache.org> |
> >> > > >>>>>>> >> > > > > | Subject | Move to JDK-11 |
> >> > > >>>>>>> >> > > > > Hi Everyone,
> >> > > >>>>>>> >> > > > > Want to pull in the attention of folks towards 
> >> > > >>>>>>> >> > > > > moving to
> >> > > >>>>>>> JDK-11
> >> > > >>>>>>> >> > compile
> >> > > >>>>>>> >> > > > > time support in Hive. There was a ticket in the 
> >> > > >>>>>>> >> > > > > past [1]
> >> > > >>>>>>> which
> >> > > >>>>>>> >> talks
> >> > > >>>>>>> >> > > > about
> >> > > >>>>>>> >> > > > > it and If I could decode it right, it was blocked 
> >> > > >>>>>>> >> > > > > because
> >> > > >>>>>>> the
> >> > > >>>>>>> >> Hadoop
> >> > > >>>>>>> >> > > > > version used by Hive didn't had JDK-11 runtime 
> >> > > >>>>>>> >> > > > > support,
> >> > > >>>>>>> But with
> >> > > >>>>>>> >> [2]
> >> > > >>>>>>> >> > in
> >> > > >>>>>>> >> > > > we
> >> > > >>>>>>> >> > > > > have upgraded the Hadoop version, so that problem is
> >> > > >>>>>>> sorted out. I
> >> > > >>>>>>> >> > > > couldn't
> >> > > >>>>>>> >> > > > > even see any unresolved tickets in the blocked state
> >> > > >>>>>>> either.
> >> > > >>>>>>> >> > > > >
> >> > > >>>>>>> >> > > > > I quickly tried* a  mvn clean install -DskipTests
> >> > > >>>>>>> -Piceberg
> >> > > >>>>>>> >> -Pitests
> >> > > >>>>>>> >> > > > > -Dmaven.javadoc.skip=true
> >> > > >>>>>>> >> > > > >
> >> > > >>>>>>> >> > > > > And no surprises it failed with some weird 
> >> > > >>>>>>> >> > > > > exceptions
> >> > > >>>>>>> towards the
> >> > > >>>>>>> >> > end.
> >> > > >>>>>>> >> > > > But
> >> > > >>>>>>> >> > > > > I think that should be solvable.
> >> > > >>>>>>> >> > > > >
> >> > > >>>>>>> >> > > > > So, Questions?
> >> > > >>>>>>> >> > > > >
> >> > > >>>>>>> >> > > > > - What do folks think about this? Should we put in 
> >> > > >>>>>>> >> > > > > some
> >> > > >>>>>>> effort
> >> > > >>>>>>> >> > towards
> >> > > >>>>>>> >> > > > > JDK-11
> >> > > >>>>>>> >> > > > > - Should we support both JDK-11 & JDK-8?
> >> > > >>>>>>> >> > > > > - Ditch JDK-11 and directly shoot for JDK-17?
> >> > > >>>>>>> >> > > > >
> >> > > >>>>>>> >> > > > > Let me know your thoughts, In case anyone has some
> >> > > >>>>>>> experience in
> >> > > >>>>>>> >> this
> >> > > >>>>>>> >> > > > area
> >> > > >>>>>>> >> > > > > and have tried something in the context, feel free 
> >> > > >>>>>>> >> > > > > to
> >> > > >>>>>>> share or
> >> > > >>>>>>> >> may be
> >> > > >>>>>>> >> > > if
> >> > > >>>>>>> >> > > > > someone has any potential action plan or so
> >> > > >>>>>>> >> > > > >
> >> > > >>>>>>> >> > > > > -Ayush
> >> > > >>>>>>> >> > > > >
> >> > > >>>>>>> >> > > > > [1] https://issues.apache.org/jira/browse/HIVE-22415
> >> > > >>>>>>> >> > > > > [2] https://issues.apache.org/jira/browse/HIVE-24484
> >> > > >>>>>>> >> > > > >
> >> > > >>>>>>> >> > > > > * changed the maven.compiler.source &
> >> > > >>>>>>> maven.compiler.target to 11
> >> > > >>>>>>> >> > > > >
> >> > > >>>>>>> >> > > >
> >> > > >>>>>>> >> > >
> >> > > >>>>>>> >> >
> >> > > >>>>>>> >>
> >> > > >>>>>>> >
> >> > > >>>>>>>
> >> > > >>>>>>
> >> > >
> >> >

Reply via email to