[
https://issues.apache.org/jira/browse/HIVE-29015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17978625#comment-17978625
]
yongzhi.shao edited comment on HIVE-29015 at 6/16/25 8:39 AM:
--------------------------------------------------------------
[~zhangbutao] [~ayushtkn] :
Sir. You can refer to this document for configuration. [Apache Hive : Manual
Installation|https://hive.apache.org/docs/latest/manual-installation_283118363/#installing-with-old-version-hadoopgreater-than-or-equal-310]
But admittedly, running Hive (jdk17) + Hadoop (jdk8) is a bit tricky.
Can you specify TEZ's JAVA_HOME separately first?
was (Author: lisoda):
[~zhangbutao] [~ayushtkn] :
Sir. You can refer to this document for configuration. [Apache Hive : Manual
Installation|https://hive.apache.org/docs/latest/manual-installation_283118363/#installing-with-old-version-hadoopgreater-than-or-equal-310]
But admittedly, running Hive (jdk17) + Hadoop (jdk8) is a bit tricky.
> Hive(jdk17) + Hadoop(jdk8) can not submit Tez job
> -------------------------------------------------
>
> Key: HIVE-29015
> URL: https://issues.apache.org/jira/browse/HIVE-29015
> Project: Hive
> Issue Type: Test
> Reporter: Butao Zhang
> Priority: Major
>
> I'm not sure if this is a problem. So creating this issue will facilitate our
> discussion.
>
> {*}This is how I conducted the test{*}:
> * Hive was deployed on JDK 17, while Hadoop was deployed on JDK 8.
> However, when I submitted the Tez task, the Tez am would fail to start with
> exception: {color:#de350b}Unrecognized option:
> --add-opens=java.base/java.net=ALL-UNNAMED{color}
>
> {code:java}
> INFO : Tez session hasn't been created yet. Opening session
> ERROR : Failed to execute tez graph.
> org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown.
> Application application_1749800465429_0005 failed 2 times due to AM Container
> for appattempt_1749800465429_0005_000002 exited with exitCode: 1
> Failing this attempt.Diagnostics: [2025-06-13 15:46:17.468]Exception from
> container-launch.
> Container id: container_1749800465429_0005_02_000001
> Exit code: 1[2025-06-13 15:46:17.471]Container exited with a non-zero exit
> code 1. Error file: prelaunch.err.
> Last 4096 bytes of prelaunch.err :
> Last 4096 bytes of stderr :
> Unrecognized option: --add-opens=java.base/java.net=ALL-UNNAMED
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> [2025-06-13 15:46:17.472]Container exited with a non-zero exit code 1. Error
> file: prelaunch.err.
> Last 4096 bytes of prelaunch.err :
> Last 4096 bytes of stderr :
> Unrecognized option: --add-opens=java.base/java.net=ALL-UNNAMED
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> For more detailed output, check the application tracking page:
> http://127.0.0.1:8188/applicationhistory/app/application_1749800465429_0005
> Then click on links to logs of each attempt.
> . Failing the application.
> at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:995)
> ~[tez-api-0.10.5.jar:0.10.5]
> at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:964)
> ~[tez-api-0.10.5.jar:0.10.5]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:484)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:384)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolSession.openInternal(TezSessionPoolSession.java:124)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:250)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezTask.ensureSessionHasResources(TezTask.java:481)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:232)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:214)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Executor.launchTask(Executor.java:354)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Executor.launchTasks(Executor.java:327)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Executor.runTasks(Executor.java:244)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:105)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:345)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:189)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:142)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:137)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:190)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:234)
> ~[hive-service-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:334)
> ~[hive-service-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at
> java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
> ~[?:?]
> at java.base/javax.security.auth.Subject.doAs(Subject.java:439) ~[?:?]
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1953)
> ~[hadoop-common-3.4.1.jar:?]
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:354)
> ~[hive-service-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
> ~[?:?]
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> ~[?:?]
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> ~[?:?]
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> ~[?:?]
> at java.base/java.lang.Thread.run(Thread.java:833) [?:?]
> ERROR : FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask. TezSession has already shutdown.
> Application application_1749800465429_0005 failed 2 times due to AM Container
> for appattempt_1749800465429_0005_000002 exited with exitCode: 1
> {code}
>
>
> * Hive was deployed on JDK 17, and Hadoop 3.4.1 was also deployed on JDK 17.
> {color:#de350b}But If I switch the JDK8 used by Hadoop to JDK17, then
> the Tez tasks can run normally.{color}
>
>
> Due to the actual production environment, users may not readily change the
> JDK of Hadoop. Users may prefer to have Hadoop continue to use JDK 8, while
> Hive uses JDK 17.
> {color:#de350b}So my question is: If Hive is deployed on JDK 17, can we
> submit Tez tasks in the Hadoop cluster (with JDK 8)? {color}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
