[
https://issues.apache.org/jira/browse/HIVE-23604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuanhao Lu updated HIVE-23604:
------------------------------
Description:
This JIRA https://issues.apache.org/jira/browse/HIVE-22126 shaded guava in
hive-exec. There is a issue that LLAP is also using this guava, so after
shading, it cannot use hive-exec guava and may turn to `./tez/guava-11.0.2.jar`
which will cause following error
{code:java}
Status: Running (Executing on YARN cluster with App id
application_1591081923777_0005)
Map 1: -/- Reducer 2: 0/1
Map 1: 0/11 Reducer 2: 0/1
Map 1: 0(+11,-11)/11 Reducer 2: 0/1
Map 1: 0(+0,-32)/11 Reducer 2: 0/1
Status: Failed
Vertex failed, vertexName=Map 1, vertexId=vertex_1591081923777_0005_1_00,
diagnostics=[Task failed, taskId=task_1591081923777_0005_1_00_000002,
diagnostics=[TaskAttempt 0 failed,
info=[org.apache.hadoop.ipc.RemoteException(java.lang.NoSuchMethodError):
com.google.common.base.Stopwatch.createUnstarted()Lcom/google/common/base/Stopwatch;
at
org.apache.hadoop.hive.llap.daemon.impl.TaskRunnerCallable.<init>(TaskRunnerCallable.java:122)
at
org.apache.hadoop.hive.llap.daemon.impl.ContainerRunnerImpl.submitWork(ContainerRunnerImpl.java:274)
at
org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.submitWork(LlapDaemon.java:558)
at
org.apache.hadoop.hive.llap.daemon.impl.LlapProtocolServerImpl.submitWork(LlapProtocolServerImpl.java:104)
at
org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$LlapDaemonProtocol$2.callBlockingMethod(LlapDaemonProtocolProtos.java:19020)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:528)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1070)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:999)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:927)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2915)
{code}
This can be solved by adding `com.google.common.base.Stopwatch.class` to this
section in hive 3.1.2 release
[https://github.com/apache/hive/blob/release-3.1.2-rc0/llap-server/src/java/org/apache/hadoop/hive/llap/cli/LlapServiceDriver.java#L385-L409]
and probably to this file for master branch
https://github.com/apache/hive/blob/2385361ef33ff81760115cfee4de31ec18afe2f3/llap-server/src/java/org/apache/hadoop/hive/llap/cli/service/AsyncTaskCopyLocalJars.java
was:
This JIRA https://issues.apache.org/jira/browse/HIVE-22126 shaded guava in
hive-exec. There is a issue that LLAP is also using this guava, so after
shading, it cannot use hive-exec guava and may turn to `./tez/guava-11.0.2.jar`
which will cause following error
{code:java}
Status: Running (Executing on YARN cluster with App id
application_1591081923777_0005)
Map 1: -/- Reducer 2: 0/1
Map 1: 0/11 Reducer 2: 0/1
Map 1: 0(+11,-11)/11 Reducer 2: 0/1
Map 1: 0(+0,-32)/11 Reducer 2: 0/1
Status: Failed
Vertex failed, vertexName=Map 1, vertexId=vertex_1591081923777_0005_1_00,
diagnostics=[Task failed, taskId=task_1591081923777_0005_1_00_000002,
diagnostics=[TaskAttempt 0 failed,
info=[org.apache.hadoop.ipc.RemoteException(java.lang.NoSuchMethodError):
com.google.common.base.Stopwatch.createUnstarted()Lcom/google/common/base/Stopwatch;
at
org.apache.hadoop.hive.llap.daemon.impl.TaskRunnerCallable.<init>(TaskRunnerCallable.java:122)
at
org.apache.hadoop.hive.llap.daemon.impl.ContainerRunnerImpl.submitWork(ContainerRunnerImpl.java:274)
at
org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.submitWork(LlapDaemon.java:558)
at
org.apache.hadoop.hive.llap.daemon.impl.LlapProtocolServerImpl.submitWork(LlapProtocolServerImpl.java:104)
at
org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$LlapDaemonProtocol$2.callBlockingMethod(LlapDaemonProtocolProtos.java:19020)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:528)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1070)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:999)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:927)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2915)
{code}
This can be solved by adding `com.google.common.base.Stopwatch.class` to this
section
[https://github.com/apache/hive/blob/release-3.1.2-rc0/llap-server/src/java/org/apache/hadoop/hive/llap/cli/LlapServiceDriver.java#L385-L409]
> LLAP does not have correct version of guava after HIVE-22126
> ------------------------------------------------------------
>
> Key: HIVE-23604
> URL: https://issues.apache.org/jira/browse/HIVE-23604
> Project: Hive
> Issue Type: Bug
> Components: llap
> Reporter: Yuanhao Lu
> Priority: Major
>
> This JIRA https://issues.apache.org/jira/browse/HIVE-22126 shaded guava in
> hive-exec. There is a issue that LLAP is also using this guava, so after
> shading, it cannot use hive-exec guava and may turn to
> `./tez/guava-11.0.2.jar` which will cause following error
> {code:java}
> Status: Running (Executing on YARN cluster with App id
> application_1591081923777_0005)
>
> Map 1: -/- Reducer 2: 0/1
> Map 1: 0/11 Reducer 2: 0/1
> Map 1: 0(+11,-11)/11 Reducer 2: 0/1
> Map 1: 0(+0,-32)/11 Reducer 2: 0/1
> Status: Failed
> Vertex failed, vertexName=Map 1, vertexId=vertex_1591081923777_0005_1_00,
> diagnostics=[Task failed, taskId=task_1591081923777_0005_1_00_000002,
> diagnostics=[TaskAttempt 0 failed,
> info=[org.apache.hadoop.ipc.RemoteException(java.lang.NoSuchMethodError):
> com.google.common.base.Stopwatch.createUnstarted()Lcom/google/common/base/Stopwatch;
> at
> org.apache.hadoop.hive.llap.daemon.impl.TaskRunnerCallable.<init>(TaskRunnerCallable.java:122)
> at
> org.apache.hadoop.hive.llap.daemon.impl.ContainerRunnerImpl.submitWork(ContainerRunnerImpl.java:274)
> at
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.submitWork(LlapDaemon.java:558)
> at
> org.apache.hadoop.hive.llap.daemon.impl.LlapProtocolServerImpl.submitWork(LlapProtocolServerImpl.java:104)
> at
> org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$LlapDaemonProtocol$2.callBlockingMethod(LlapDaemonProtocolProtos.java:19020)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:528)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1070)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:999)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:927)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2915)
> {code}
> This can be solved by adding `com.google.common.base.Stopwatch.class` to this
> section in hive 3.1.2 release
> [https://github.com/apache/hive/blob/release-3.1.2-rc0/llap-server/src/java/org/apache/hadoop/hive/llap/cli/LlapServiceDriver.java#L385-L409]
> and probably to this file for master branch
> https://github.com/apache/hive/blob/2385361ef33ff81760115cfee4de31ec18afe2f3/llap-server/src/java/org/apache/hadoop/hive/llap/cli/service/AsyncTaskCopyLocalJars.java
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
