[jira] [Commented] (TEZ-4547) Add Tez AM JobID to the JobConf
[ https://issues.apache.org/jira/browse/TEZ-4547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17826815#comment-17826815 ] Venkatasubrahmanian Narayanan commented on TEZ-4547: Past discussion in: https://issues.apache.org/jira/browse/HADOOP-19091 > Add Tez AM JobID to the JobConf > --- > > Key: TEZ-4547 > URL: https://issues.apache.org/jira/browse/TEZ-4547 > Project: Apache Tez > Issue Type: Improvement >Affects Versions: 0.10.2 >Reporter: Venkatasubrahmanian Narayanan >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Tez creates JobIDs for tasks by appending the vertex index to the cluster > timestamp to avoid multiple jobs run in a single Tez session sharing a JobID. > Hadoop's MagicS3GuardCommitter needs a job-wide UUID to ensure that the task > committers and the job committer write to/read from the same paths and can > hence actually commit data. Adding the AM's JobID to the Configuration > objects allows applications like Hive to pass that as the UUID to the > committer. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (TEZ-4547) Add Tez AM JobID to the JobConf
[ https://issues.apache.org/jira/browse/TEZ-4547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17826814#comment-17826814 ] Venkatasubrahmanian Narayanan commented on TEZ-4547: Github PR: https://github.com/apache/tez/pull/339 > Add Tez AM JobID to the JobConf > --- > > Key: TEZ-4547 > URL: https://issues.apache.org/jira/browse/TEZ-4547 > Project: Apache Tez > Issue Type: Improvement >Affects Versions: 0.10.2 >Reporter: Venkatasubrahmanian Narayanan >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Tez creates JobIDs for tasks by appending the vertex index to the cluster > timestamp to avoid multiple jobs run in a single Tez session sharing a JobID. > Hadoop's MagicS3GuardCommitter needs a job-wide UUID to ensure that the task > committers and the job committer write to/read from the same paths and can > hence actually commit data. Adding the AM's JobID to the Configuration > objects allows applications like Hive to pass that as the UUID to the > committer. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (TEZ-4547) Add Tez AM JobID to the JobConf
[ https://issues.apache.org/jira/browse/TEZ-4547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkatasubrahmanian Narayanan updated TEZ-4547: --- Summary: Add Tez AM JobID to the JobConf (was: Add Tez AM JobID to the Configuration object) > Add Tez AM JobID to the JobConf > --- > > Key: TEZ-4547 > URL: https://issues.apache.org/jira/browse/TEZ-4547 > Project: Apache Tez > Issue Type: Improvement >Affects Versions: 0.10.2 >Reporter: Venkatasubrahmanian Narayanan >Priority: Major > > Tez creates JobIDs for tasks by appending the vertex index to the cluster > timestamp to avoid multiple jobs run in a single Tez session sharing a JobID. > Hadoop's MagicS3GuardCommitter needs a job-wide UUID to ensure that the task > committers and the job committer write to/read from the same paths and can > hence actually commit data. Adding the AM's JobID to the Configuration > objects allows applications like Hive to pass that as the UUID to the > committer. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (TEZ-4547) Add Tez AM JobID to the Configuration object
Venkatasubrahmanian Narayanan created TEZ-4547: -- Summary: Add Tez AM JobID to the Configuration object Key: TEZ-4547 URL: https://issues.apache.org/jira/browse/TEZ-4547 Project: Apache Tez Issue Type: Improvement Affects Versions: 0.10.2 Reporter: Venkatasubrahmanian Narayanan Tez creates JobIDs for tasks by appending the vertex index to the cluster timestamp to avoid multiple jobs run in a single Tez session sharing a JobID. Hadoop's MagicS3GuardCommitter needs a job-wide UUID to ensure that the task committers and the job committer write to/read from the same paths and can hence actually commit data. Adding the AM's JobID to the Configuration objects allows applications like Hive to pass that as the UUID to the committer. -- This message was sent by Atlassian Jira (v8.20.10#820010)
