[
https://issues.apache.org/jira/browse/TEZ-1106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13997348#comment-13997348
]
Mohammad Kamrul Islam commented on TEZ-1106:
--------------------------------------------
>is there an existing file with utility functions that could be re-used?
Yes. there is TezUtils.java in tea-common module. I tried to add those in that
file. But I believe TezClientUtils.java which is in tez-api needs this methods.
I believe tea-common depends on tez-api. That's why i have put it into tez-api
module.
Open for any suggestion.
>are any of its functions meant to be used by user code? or only framework code?
Both. end user could call getTezBaseStagingPath().
>Instead of throwing an error for a staging dir not being configured. How about
>have the function fall back to using something along the lines of
>"/user/<user>/.tez_staging/" as the base path?
So basically use this as defined in
TezConfiguration.TEZ_AM_STAGING_DIR_DEFAULT. In TEZ-692, there was some
discussion to
not to use any default.
>this does not seem right. For every function call, a new TezConf is being
>created.
I can reduce it to one TezConf object creation instead of two, Are you
suggesting to remove the last one too?
> Tez framework should use a unique subdir when creating new files in staging
> -----------------------------------------------------------------------------
>
> Key: TEZ-1106
> URL: https://issues.apache.org/jira/browse/TEZ-1106
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Mohammad Kamrul Islam
> Assignee: Mohammad Kamrul Islam
> Attachments: TEZ-1106.1.patch
>
>
> Currently the files are created in different sub-directories. It is hard to
> manage and cleanup at the end.
> The proposal is to create a new subdir : $STAGE_DIR/<APP_ID>/
> All recovery files will go under : $STAGE_DIR/<APP_ID>/recovery/<attemp_num>/
> All confs will go under: $STAGE_DIR/<APP_ID>/conf/
> All dagplans will go: $STAGE_DIR/<APP_ID>/dag_id/plan/
--
This message was sent by Atlassian JIRA
(v6.2#6252)
