[
https://issues.apache.org/jira/browse/TEZ-3670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15938916#comment-15938916
]
Zhiyuan Yang edited comment on TEZ-3670 at 3/23/17 6:05 PM:
------------------------------------------------------------
[~txhsj] How did you find out federation path was converted? Stacktrace shows
viewfs was correctly used.
{code}
java.io.FileNotFoundException: /hive
at org.apache.hadoop.fs.viewfs.InodeTree.resolve(InodeTree.java:402)
at
org.apache.hadoop.fs.viewfs.ViewFileSystem.getFileStatus(ViewFileSystem.java:377)
{code}
Also, I suggest to ask this kind of question in tez user mailing list There you
are more likely to get help. Once it's confirmed that there is a Tez bug, you
can file jira for it then.
was (Author: aplusplus):
[~txhsj] How did you find out federation path was converted? Stacktrace shows
viewfs was correctly used.
{code}
java.io.FileNotFoundException: /hive
at org.apache.hadoop.fs.viewfs.InodeTree.resolve(InodeTree.java:402)
at
org.apache.hadoop.fs.viewfs.ViewFileSystem.getFileStatus(ViewFileSystem.java:377)
{code}
> hive on tez throw FileNotFoundException,when enable hdfs federation
> --------------------------------------------------------------------
>
> Key: TEZ-3670
> URL: https://issues.apache.org/jira/browse/TEZ-3670
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.8.4
> Reporter: Saijin Huang
>
> hive version is apache hive-2.1.0 tez version is apache tez-0.8.4 i set
> tez.lib.uris=viewfs://nsX/nameservice/hive/lib/tez-0.8.4.tar.gz
> when i run a tez job on hive ,it always appears the problem
> {code}
> 2017-03-23T15:26:24,255 INFO client.TezClientUtils (:()) - Using
> tez.lib.uris value from configuration:
> viewfs://nsX/nameservice/hive/lib/tez-0.8.4.tar.gz
> 2017-03-23T15:26:24,255 INFO client.TezClientUtils (:()) - Using
> tez.lib.uris.classpath value from configuration:
> $HADOOP_CONF_DIR,/opt/ZDH/parcels/lib/hadoop/*,/opt/ZDH/parcels/lib/hadoop/lib/*,/opt/ZDH/parcels/lib/hadoop-hdfs/*,/opt/ZDH/parcels/lib/hadoop-hdfs/lib/*,/opt/ZDH/parcels/lib/hadoop-yarn/*,/opt/ZDH/parcels/lib/hadoop-yarn/lib/*,/opt/ZDH/parcels/lib/hadoop-mapreduce/*,/opt/ZDH/parcels/lib/hadoop-mapreduce/lib/*,/opt/ZDH/parcels/lib/tez/*,/opt/ZDH/parcels/lib/tez/lib/*
> 2017-03-23T15:26:24,260 ERROR exec.Task (:()) - Failed to execute tez graph.
> java.io.FileNotFoundException: /hive
> at org.apache.hadoop.fs.viewfs.InodeTree.resolve(InodeTree.java:402)
> at
> org.apache.hadoop.fs.viewfs.ViewFileSystem.getFileStatus(ViewFileSystem.java:377)
> at
> org.apache.tez.client.TezClientUtils.checkAncestorPermissionsForAllUsers(TezClientUtils.java:1031)
> at
> org.apache.tez.client.TezClientUtils.addLocalResources(TezClientUtils.java:275)
> at
> org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183)
> at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:1057)
> at org.apache.tez.client.TezClient.start(TezClient.java:447)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:390)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:320)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager$TezSessionPoolSession.openInternal(TezSessionPoolManager.java:611)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:203)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezTask.updateSession(TezTask.java:277)
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:155)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1917)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1586)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1331)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1092)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1080)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:399)
> at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:776)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:714)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> {code}
> I found the federation path
> "viewfs://nsX/nameservice/hive/lib/tez-0.8.4.tar.gz" will be convert to
> "hdfs://nameservice/hive/lib/tez-0.8.4.tar.gz" ,which leads to the path is
> not correct.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
