[
https://issues.apache.org/jira/browse/TEZ-1348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16809292#comment-16809292
]
Todd Lipcon commented on TEZ-1348:
----------------------------------
It seems that by changing the code to ensure that the tez working directory is
created on Local FS, then tez local-mode can work even if defaultFS is a remote
cluster. This is actually useful, for example when testing Hive against a
pseudo-distributed HDFS, if you don't want to also start a pseudo-distributed
YARN. I'll work on a patch for this.
> Setup configs required for local mode automatically, instead of relying on
> changes to tez-site
> ----------------------------------------------------------------------------------------------
>
> Key: TEZ-1348
> URL: https://issues.apache.org/jira/browse/TEZ-1348
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Siddharth Seth
> Priority: Critical
>
> In TEZ-717, I incorrect thought setting fs.defaultFS programmatically in
> tez-site would work for local mode.
> Currently the requirement is that tez-site.xml must have fs.defaultFS set to
> file:///.
> While that works, it doesn't allow for seamless execution in either
> local-mode or on a cluster.
> The main issue here is that when Inputs / Outputs are configured - they use a
> version of configuration which reads tez-site, and do not use the
> configuration from the client itself (which is correct behaviour).
> Not sure what a good way to fix this is
> 1) It may be possible to override this value each time an instance of
> Configuration/TezConfiguration is created. One possible way would be to
> statically add a default resource to Configuration the moment a local client
> is created.
> 2) Provide information in the contexts on whether this is local or not. This
> is fairly ugly, and would get in the way of running mixed mode tasks.
> Anyone have other suggestions ?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
