[
https://issues.apache.org/jira/browse/TEZ-1106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14028697#comment-14028697
]
Hitesh Shah commented on TEZ-1106:
----------------------------------
Comments:
Taking a look at the staging dir contents after a job ran:
{code}
drwx------ - hitesh supergroup 0 2014-06-11 17:19
.tez/application_1402532236805_0002
drwx------ - hitesh supergroup 0 2014-06-11 17:19
.tez/application_1402532236805_0002/recovery
drwx------ - hitesh supergroup 0 2014-06-11 17:19
.tez/application_1402532236805_0002/recovery/1
-rw-r--r-- 1 hitesh supergroup 330 2014-06-11 17:19
.tez/application_1402532236805_0002/recovery/1/.summary
-rw-r--r-- 1 hitesh supergroup 133063 2014-06-11 17:19
.tez/application_1402532236805_0002/recovery/1/dag_1402532236805_0002_1.recovery
-rw-r--r-- 1 hitesh supergroup 1025 2014-06-11 17:19
.tez/application_1402532236805_0002/tez-conf.pb
-rw-r--r-- 1 hitesh supergroup 3123 2014-06-11 17:19
.tez/application_1402532236805_0002/tez.session.local-resources.pb.file-name
{code}
For .summary, I think it should be just renamed to "summary" with the prefix
dot removed.
Minor nit: Might be good to rename "tez.session.local-resources.pb.file-name"
to "tez.session.local-resources.pb"
Feel free to go ahead and commit once the above 2 minor nits are addressed.
> Tez framework should use a unique subdir when creating new files in staging
> -----------------------------------------------------------------------------
>
> Key: TEZ-1106
> URL: https://issues.apache.org/jira/browse/TEZ-1106
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Mohammad Kamrul Islam
> Assignee: Mohammad Kamrul Islam
> Attachments: TEZ-1106.1.patch, TEZ-1106.2.patch, TEZ-1106.3.patch,
> TEZ-1106.4.patch, TEZ-1106.5.patch, TEZ-1106.6.patch
>
>
> Currently the files are created in different sub-directories. It is hard to
> manage and cleanup at the end.
> The proposal is to create a new subdir : $STAGE_DIR/<APP_ID>/
> All recovery files will go under : $STAGE_DIR/<APP_ID>/recovery/<attemp_num>/
> All confs will go under: $STAGE_DIR/<APP_ID>/conf/
> All dagplans will go: $STAGE_DIR/<APP_ID>/dag_id/plan/
--
This message was sent by Atlassian JIRA
(v6.2#6252)
