[jira] [Commented] (TEZ-4074) Tez does not run with Hadoop Trunk (3.3.0-snapshot)
[
https://issues.apache.org/jira/browse/TEZ-4074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309211#comment-17309211
]
Dhirendra Pandit commented on TEZ-4074:
---
Thanks for the hint [~vinaypatil18] ,
I have checked the compatible versions of Hadoop 3.1.3 and Tez 0.10.0 from
[https://debian.netcologne.de/apache.org/tez/0.10.0/] and trying to setup the
cluster. It gives me an error shown below while starting hive CLI.
{code:java}
Logging initialized using configuration in
jar:file:/opt/bigdata/hadoop3/apache-hive-3.1.0-bin/lib/hive-common-3.1.0.jar!/hive-log4j2.properties
Async: true
2021-03-26 07:54:52,153 INFO [main] SessionState
(SessionState.java:printInfo(1224)) -
Logging initialized using configuration in
jar:file:/opt/bigdata/hadoop3/apache-hive-3.1.0-bin/lib/hive-common-3.1.0.jar!/hive-log4j2.properties
Async: true
Exception in thread "main" java.lang.ClassCastException:
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$GetFileInfoRequestProto
cannot be cast to com.google.protobuf.Message
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118)
at com.sun.proxy.$Proxy28.getFileInfo(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:904)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
at com.sun.proxy.$Proxy29.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1661)
at
org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1577)
at
org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1574)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1589)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1683)
at
org.apache.hadoop.hive.ql.exec.Utilities.ensurePathIsWritable(Utilities.java:4486)
at
org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:757)
at
org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:698)
at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:624)
at
org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:588)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:747)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
{code}
> Tez does not run with Hadoop Trunk (3.3.0-snapshot)
> ---
>
> Key: TEZ-4074
> URL: https://issues.apache.org/jira/browse/TEZ-4074
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Ahmed Hussein
>Priority: Major
>
> Tez throws a runtime exception when compiled against Hadoop-3.3.0.
> With Tez running Guava (11.0.2) and Hadoop eunning Guava 27.0-jre (see
> HADOOP-16210), there is an incompatibility of Guava library.
> {code:java}
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.136
> s <<< FAILURE! - in org.apache.tez.dag.app.TestSpeculation
> [ERROR] org.apache.tez.dag.app.TestSpeculation Time elapsed: 0.136 s <<<
> ERROR!
> java.lang.NoSuchMethodError:
> com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
> at
>
[jira] [Commented] (TEZ-4074) Tez does not run with Hadoop Trunk (3.3.0-snapshot)
[
https://issues.apache.org/jira/browse/TEZ-4074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17308801#comment-17308801
]
Vinay commented on TEZ-4074:
Even set this config, also we have set `tez.use.cluster.hadoop-libs` to false.
We had to build tez-0.10.0 from the source code with hadoop version 3.2.1 and
placed the fat jar to hdfs
{code:java}
tez.task.launch.cluster-default.cmd-opts
-cp
/usr/hadoop/share/hadoop/hdfs/lib/guava-27.0-jre.jar:$$CLASSPATH -server
-Djava.net.preferIPv4Stack=true
{code}
> Tez does not run with Hadoop Trunk (3.3.0-snapshot)
> ---
>
> Key: TEZ-4074
> URL: https://issues.apache.org/jira/browse/TEZ-4074
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Ahmed Hussein
>Priority: Major
>
> Tez throws a runtime exception when compiled against Hadoop-3.3.0.
> With Tez running Guava (11.0.2) and Hadoop eunning Guava 27.0-jre (see
> HADOOP-16210), there is an incompatibility of Guava library.
> {code:java}
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.136
> s <<< FAILURE! - in org.apache.tez.dag.app.TestSpeculation
> [ERROR] org.apache.tez.dag.app.TestSpeculation Time elapsed: 0.136 s <<<
> ERROR!
> java.lang.NoSuchMethodError:
> com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
> at
> org.apache.tez.dag.app.TestSpeculation.setupSpeculation(TestSpeculation.java:86)
> {code}
> It looks like guava added single parameter optimizations which breaks
> compatibility with {{VAR_ARGS}}. So, even though it shows source
> compatibility it is throwing a runtime error due to binary incompatibility.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TEZ-4074) Tez does not run with Hadoop Trunk (3.3.0-snapshot)
[
https://issues.apache.org/jira/browse/TEZ-4074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17308568#comment-17308568
]
Dhirendra Pandit commented on TEZ-4074:
---
Hi [~vinaypatil18],
Thanks for the hint, I have tried that option but still getting below error. My
Tez configuration file looks like below.
{code:java}
// code placeholder
tez.lib.uris
${fs.defaultFS}/apps/tez-0.10.0/tez-0.10.0-minimal.tar.gz
tez.queue.name default
tez.am.resource.memory.mb 4096
tez.am.java.opts -server -Xmx3276m
-Djava.net.preferIPv4Stack=true -XX:+UseNUMA -XX:+UseParallelGC
tez.runtime.io.sort.mb 1024
tez.use.cluster.hadoop-libs
true
tez.am.launch.cluster-default.cmd-opts -cp
$HADOOP_HOME/share/hadoop/hdfs/lib/guava-27.0-jre.jar:$CLASSPATH -server
-Djava.net.preferIPv4Stack=true
{code}
Kindly let me know if you can help me to resolve this problem.
Regards,
Dhirendra
> Tez does not run with Hadoop Trunk (3.3.0-snapshot)
> ---
>
> Key: TEZ-4074
> URL: https://issues.apache.org/jira/browse/TEZ-4074
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Ahmed Hussein
>Priority: Major
>
> Tez throws a runtime exception when compiled against Hadoop-3.3.0.
> With Tez running Guava (11.0.2) and Hadoop eunning Guava 27.0-jre (see
> HADOOP-16210), there is an incompatibility of Guava library.
> {code:java}
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.136
> s <<< FAILURE! - in org.apache.tez.dag.app.TestSpeculation
> [ERROR] org.apache.tez.dag.app.TestSpeculation Time elapsed: 0.136 s <<<
> ERROR!
> java.lang.NoSuchMethodError:
> com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
> at
> org.apache.tez.dag.app.TestSpeculation.setupSpeculation(TestSpeculation.java:86)
> {code}
> It looks like guava added single parameter optimizations which breaks
> compatibility with {{VAR_ARGS}}. So, even though it shows source
> compatibility it is throwing a runtime error due to binary incompatibility.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TEZ-4074) Tez does not run with Hadoop Trunk (3.3.0-snapshot)
[
https://issues.apache.org/jira/browse/TEZ-4074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301422#comment-17301422
]
Vinay commented on TEZ-4074:
We were able to solve this issue by copying guava jar to the class path
{code:java}
tez.am.launch.cluster-default.cmd-opts
-cp /usr/hadoop/share/hadoop/hdfs/lib/guava-27.0-jre.jar:$CLASSPATH
-server -Djava.net.preferIPv4Stack=true
{code}
IMO, the above is a workaround, instead TEZ should adhere to the Hadoop class
path first which has all the latest jar's (incase of Hadoop 3.x)
The main issue is the hive-exec jar which uses guava 11, the fix for which is
being done here - https://issues.apache.org/jira/browse/HIVE-22126
> Tez does not run with Hadoop Trunk (3.3.0-snapshot)
> ---
>
> Key: TEZ-4074
> URL: https://issues.apache.org/jira/browse/TEZ-4074
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Ahmed Hussein
>Priority: Major
>
> Tez throws a runtime exception when compiled against Hadoop-3.3.0.
> With Tez running Guava (11.0.2) and Hadoop eunning Guava 27.0-jre (see
> HADOOP-16210), there is an incompatibility of Guava library.
> {code:java}
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.136
> s <<< FAILURE! - in org.apache.tez.dag.app.TestSpeculation
> [ERROR] org.apache.tez.dag.app.TestSpeculation Time elapsed: 0.136 s <<<
> ERROR!
> java.lang.NoSuchMethodError:
> com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
> at
> org.apache.tez.dag.app.TestSpeculation.setupSpeculation(TestSpeculation.java:86)
> {code}
> It looks like guava added single parameter optimizations which breaks
> compatibility with {{VAR_ARGS}}. So, even though it shows source
> compatibility it is throwing a runtime error due to binary incompatibility.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TEZ-4074) Tez does not run with Hadoop Trunk (3.3.0-snapshot)
[
https://issues.apache.org/jira/browse/TEZ-4074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301386#comment-17301386
]
Dhirendra Pandit commented on TEZ-4074:
---
I had the same problem. I tried compiling the snapshot version of Tez 010.1 but
still, it does not work. I have tried changing the classpath of it along with
MapReduce libraries but still no success. Can anybody help me to rectify this
problem?
> Tez does not run with Hadoop Trunk (3.3.0-snapshot)
> ---
>
> Key: TEZ-4074
> URL: https://issues.apache.org/jira/browse/TEZ-4074
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Ahmed Hussein
>Priority: Major
>
> Tez throws a runtime exception when compiled against Hadoop-3.3.0.
> With Tez running Guava (11.0.2) and Hadoop eunning Guava 27.0-jre (see
> HADOOP-16210), there is an incompatibility of Guava library.
> {code:java}
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.136
> s <<< FAILURE! - in org.apache.tez.dag.app.TestSpeculation
> [ERROR] org.apache.tez.dag.app.TestSpeculation Time elapsed: 0.136 s <<<
> ERROR!
> java.lang.NoSuchMethodError:
> com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
> at
> org.apache.tez.dag.app.TestSpeculation.setupSpeculation(TestSpeculation.java:86)
> {code}
> It looks like guava added single parameter optimizations which breaks
> compatibility with {{VAR_ARGS}}. So, even though it shows source
> compatibility it is throwing a runtime error due to binary incompatibility.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TEZ-4074) Tez does not run with Hadoop Trunk (3.3.0-snapshot)
[
https://issues.apache.org/jira/browse/TEZ-4074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301384#comment-17301384
]
suryansh agnihotri commented on TEZ-4074:
-
[~abstractdog] I faced an similar issue while running test of 0.10.0 with
hadoop 3.1.2.
{code}[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
2.016 s <<< FAILURE! - in org.apache.tez.common.TestTezCommonUtils[ERROR] Tests
run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.016 s <<< FAILURE!
- in org.apache.tez.common.TestTezCommonUtils[ERROR]
org.apache.tez.common.TestTezCommonUtils Time elapsed: 2.013 s <<<
ERROR!java.lang.NoSuchMethodError:
com.google.common.util.concurrent.Futures.addCallback(Lcom/google/common/util/concurrent/ListenableFuture;Lcom/google/common/util/concurrent/FutureCallback;)V
at
org.apache.tez.common.TestTezCommonUtils.setup(TestTezCommonUtils.java:65)\{code}
I verified that tez 0.10 uses guava "27.0-jre" and hadoop 3.1.2 also uses the
same version and that particular method is removed from guava
[https://guava.dev/releases/27.0-jre/api/docs/com/google/common/util/concurrent/Futures.html#addCallback-com.google.common.util.concurrent.ListenableFuture-com.google.common.util.concurrent.FutureCallback-java.util.concurrent.Executor-|guava-27.0]
I see the same issue in master too.
> Tez does not run with Hadoop Trunk (3.3.0-snapshot)
> ---
>
> Key: TEZ-4074
> URL: https://issues.apache.org/jira/browse/TEZ-4074
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Ahmed Hussein
>Priority: Major
>
> Tez throws a runtime exception when compiled against Hadoop-3.3.0.
> With Tez running Guava (11.0.2) and Hadoop eunning Guava 27.0-jre (see
> HADOOP-16210), there is an incompatibility of Guava library.
> {code:java}
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.136
> s <<< FAILURE! - in org.apache.tez.dag.app.TestSpeculation
> [ERROR] org.apache.tez.dag.app.TestSpeculation Time elapsed: 0.136 s <<<
> ERROR!
> java.lang.NoSuchMethodError:
> com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
> at
> org.apache.tez.dag.app.TestSpeculation.setupSpeculation(TestSpeculation.java:86)
> {code}
> It looks like guava added single parameter optimizations which breaks
> compatibility with {{VAR_ARGS}}. So, even though it shows source
> compatibility it is throwing a runtime error due to binary incompatibility.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TEZ-4074) Tez does not run with Hadoop Trunk (3.3.0-snapshot)
[
https://issues.apache.org/jira/browse/TEZ-4074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17299122#comment-17299122
]
László Bodor commented on TEZ-4074:
---
this could be tricky, if you faced this during runtime, you should double-check
the classpath...actually, this could happen very easily on a cluster, if an
"unexpected" guava jar appears on the classpath somehow
> Tez does not run with Hadoop Trunk (3.3.0-snapshot)
> ---
>
> Key: TEZ-4074
> URL: https://issues.apache.org/jira/browse/TEZ-4074
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Ahmed Hussein
>Priority: Major
>
> Tez throws a runtime exception when compiled against Hadoop-3.3.0.
> With Tez running Guava (11.0.2) and Hadoop eunning Guava 27.0-jre (see
> HADOOP-16210), there is an incompatibility of Guava library.
> {code:java}
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.136
> s <<< FAILURE! - in org.apache.tez.dag.app.TestSpeculation
> [ERROR] org.apache.tez.dag.app.TestSpeculation Time elapsed: 0.136 s <<<
> ERROR!
> java.lang.NoSuchMethodError:
> com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
> at
> org.apache.tez.dag.app.TestSpeculation.setupSpeculation(TestSpeculation.java:86)
> {code}
> It looks like guava added single parameter optimizations which breaks
> compatibility with {{VAR_ARGS}}. So, even though it shows source
> compatibility it is throwing a runtime error due to binary incompatibility.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TEZ-4074) Tez does not run with Hadoop Trunk (3.3.0-snapshot)
[
https://issues.apache.org/jira/browse/TEZ-4074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17298848#comment-17298848
]
Vinay commented on TEZ-4074:
We are facing the same issue, Our cluster configuration:
Hadoop - 3.2.1
TEZ - 0.10.0
(We have copied tez tar from the shared folder to hdfs)
{code:java}
java.lang.NoSuchMethodError:
com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338)
at
org.apache.tez.common.TezUtilsInternal.addUserSpecifiedTezConfiguration(TezUtilsInternal.java:84)
at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2396)
{code}
[~abstractdog] any suggestions ?
> Tez does not run with Hadoop Trunk (3.3.0-snapshot)
> ---
>
> Key: TEZ-4074
> URL: https://issues.apache.org/jira/browse/TEZ-4074
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Ahmed Hussein
>Priority: Major
>
> Tez throws a runtime exception when compiled against Hadoop-3.3.0.
> With Tez running Guava (11.0.2) and Hadoop eunning Guava 27.0-jre (see
> HADOOP-16210), there is an incompatibility of Guava library.
> {code:java}
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.136
> s <<< FAILURE! - in org.apache.tez.dag.app.TestSpeculation
> [ERROR] org.apache.tez.dag.app.TestSpeculation Time elapsed: 0.136 s <<<
> ERROR!
> java.lang.NoSuchMethodError:
> com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
> at
> org.apache.tez.dag.app.TestSpeculation.setupSpeculation(TestSpeculation.java:86)
> {code}
> It looks like guava added single parameter optimizations which breaks
> compatibility with {{VAR_ARGS}}. So, even though it shows source
> compatibility it is throwing a runtime error due to binary incompatibility.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TEZ-4074) Tez does not run with Hadoop Trunk (3.3.0-snapshot)
[
https://issues.apache.org/jira/browse/TEZ-4074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17280073#comment-17280073
]
Dhirendra Pandit commented on TEZ-4074:
---
Does anyone has solved this version conflict problem?
> Tez does not run with Hadoop Trunk (3.3.0-snapshot)
> ---
>
> Key: TEZ-4074
> URL: https://issues.apache.org/jira/browse/TEZ-4074
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Ahmed Hussein
>Priority: Major
>
> Tez throws a runtime exception when compiled against Hadoop-3.3.0.
> With Tez running Guava (11.0.2) and Hadoop eunning Guava 27.0-jre (see
> HADOOP-16210), there is an incompatibility of Guava library.
> {code:java}
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.136
> s <<< FAILURE! - in org.apache.tez.dag.app.TestSpeculation
> [ERROR] org.apache.tez.dag.app.TestSpeculation Time elapsed: 0.136 s <<<
> ERROR!
> java.lang.NoSuchMethodError:
> com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
> at
> org.apache.tez.dag.app.TestSpeculation.setupSpeculation(TestSpeculation.java:86)
> {code}
> It looks like guava added single parameter optimizations which breaks
> compatibility with {{VAR_ARGS}}. So, even though it shows source
> compatibility it is throwing a runtime error due to binary incompatibility.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TEZ-4074) Tez does not run with Hadoop Trunk (3.3.0-snapshot)
[
https://issues.apache.org/jira/browse/TEZ-4074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043476#comment-17043476
]
László Bodor commented on TEZ-4074:
---
for the record, guava related issues are eliminated by: TEZ-4101 and TEZ-4124,
so this should not be the problem anymore
however, as this jira is about 3.3.0-SNAPSHOT in general, I would keep it open
as I saw another compile time issues (related to @Private annotations as far as
I remember)
> Tez does not run with Hadoop Trunk (3.3.0-snapshot)
> ---
>
> Key: TEZ-4074
> URL: https://issues.apache.org/jira/browse/TEZ-4074
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Ahmed Hussein
>Priority: Major
>
> Tez throws a runtime exception when compiled against Hadoop-3.3.0.
> With Tez running Guava (11.0.2) and Hadoop eunning Guava 27.0-jre (see
> HADOOP-16210), there is an incompatibility of Guava library.
> {code:java}
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.136
> s <<< FAILURE! - in org.apache.tez.dag.app.TestSpeculation
> [ERROR] org.apache.tez.dag.app.TestSpeculation Time elapsed: 0.136 s <<<
> ERROR!
> java.lang.NoSuchMethodError:
> com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
> at
> org.apache.tez.dag.app.TestSpeculation.setupSpeculation(TestSpeculation.java:86)
> {code}
> It looks like guava added single parameter optimizations which breaks
> compatibility with {{VAR_ARGS}}. So, even though it shows source
> compatibility it is throwing a runtime error due to binary incompatibility.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TEZ-4074) Tez does not run with Hadoop Trunk (3.3.0-snapshot)
[
https://issues.apache.org/jira/browse/TEZ-4074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16853032#comment-16853032
]
Ahmed Hussein commented on TEZ-4074:
Guava27 and 11.0.2 are not source compatible.
For example, Guava27 removed API methods such as
{{Futures.addCallback(ListenableFuture future, FutureCallback
callback)}}
* Guava11.0.2:
[FutureCallback|https://google.github.io/guava/releases/11.0.2/api/docs/com/google/common/util/concurrent/Futures.html#addCallback(com.google.common.util.concurrent.ListenableFuture,%20com.google.common.util.concurrent.FutureCallback)]
* Guava27:
[FutureCallback|https://static.javadoc.io/com.google.guava/guava/27.0.1-jre/com/google/common/util/concurrent/Futures.html#addCallback-com.google.common.util.concurrent.ListenableFuture-com.google.common.util.concurrent.FutureCallback-java.util.concurrent.Executor-]
> Tez does not run with Hadoop Trunk (3.3.0-snapshot)
> ---
>
> Key: TEZ-4074
> URL: https://issues.apache.org/jira/browse/TEZ-4074
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Ahmed Hussein
>Priority: Major
>
> Tez throws a runtime exception when compiled against Hadoop-3.3.0.
> With Tez running Guava (11.0.2) and Hadoop eunning Guava 27.0-jre (see
> HADOOP-16210), there is an incompatibility of Guava library.
> {code:java}
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.136
> s <<< FAILURE! - in org.apache.tez.dag.app.TestSpeculation
> [ERROR] org.apache.tez.dag.app.TestSpeculation Time elapsed: 0.136 s <<<
> ERROR!
> java.lang.NoSuchMethodError:
> com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
> at
> org.apache.tez.dag.app.TestSpeculation.setupSpeculation(TestSpeculation.java:86)
> {code}
> It looks like guava added single parameter optimizations which breaks
> compatibility with {{VAR_ARGS}}. So, even though it shows source
> compatibility it is throwing a runtime error due to binary incompatibility.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
