Hi team,
I'm new to Apache Linkis. I've successfully set up Apache Linkis version
1.4.1 and I'm currently attempting to integrate it with Hive version 3.1.3.
However, I've encountered an issue while executing the following command:
sh ./bin/linkis-cli -engineType hive-3.1.3 -codeType hql -code "select *
from load1" -submitUser techuser -proxyUser techuser
I am getting below error
2023-08-14 11:53:20.053 INFO Background is starting a new engine for
you,execId TaskID_10_otJobId_astJob_9_codeExec_9 mark id is mark_12, it may
take several seconds, please wait
2023-08-14 11:53:42.053 ERROR jobRequest(LINKISCLI_techuser_hive_7)
execute failed,21304, Task is Failed,errorMsg: errCode: 12003 ,desc:
linkis-instance:9101_16 Failed to async get EngineNode AMErrorException:
errCode: 30002 ,desc: ServiceInstance(linkis-cg-engineconn,
linkis-instance:41969) ticketID: e04ac7a2-fc39-43fb-87a6-d88d4e084f13
Failed to initialize engine, reason: ServiceInstance(linkis-cg-engineconn,
linkis-instance:41969): log dir:
/appcom/tmp/techuser/20230814/hive/e04ac7a2-fc39-43fb-87a6-d88d4e084f13/logs,java.lang.NoClassDefFoundError:
org/apache/tez/dag/api/TezConfiguration
at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:661)
at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:586)
at
org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.getSessionState(HiveEngineConnFactory.scala:135)
at
org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.doCreateHiveSession(HiveEngineConnFactory.scala:120)
at
org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.createEngineConnSession(HiveEngineConnFactory.scala:105)
at
org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.createEngineConnSession(HiveEngineConnFactory.scala:60)
at
org.apache.linkis.manager.engineplugin.common.creation.AbstractEngineConnFactory.createEngineConn(EngineConnFactory.scala:47)
at
org.apache.linkis.manager.engineplugin.common.creation.AbstractEngineConnFactory.createEngineConn$(EngineConnFactory.scala:45)
at
org.apache.linkis.engineplugin.hive.creation.HiveEngineConnFactory.createEngineConn(HiveEngineConnFactory.scala:60)
at
org.apache.linkis.engineconn.core.engineconn.DefaultEngineConnManager.createEngineConn(EngineConnManager.scala:47)
at
org.apache.linkis.engineconn.launch.EngineConnServer$.main(EngineConnServer.scala:88)
at
org.apache.linkis.engineconn.launch.EngineConnServer.main(EngineConnServer.scala)
Caused by: java.lang.ClassNotFoundException:
org.apache.tez.dag.api.TezConfiguration
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 12 more
,ip: linkis-instance ,port: 9101 ,serviceKind: linkis-cg-linkismanager
,ip: linkis-instance ,port: 9104 ,serviceKind: linkis-cg-entrance
To provide context, I'm using Hive on Tez, and I've copied all the Tez JAR
files to {LINKIS_HOME}/lib/linkis-engineconn-plugins/hive/plugin/3.1.3.
However, despite this, the error persists. My concern is whether the new
version of Linkis supports Hive on Tez, or if there are any additional
steps I need to follow to enable Hive on Tez integration.
Thanks in Advance
