[ https://issues.apache.org/jira/browse/TEZ-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512249#comment-14512249 ]
Siddharth Seth commented on TEZ-2368: ------------------------------------- This could be renamed to something like getDagIdentifier (short unique integer identifying a dag within an app) instead of getDagNumber - which makes it cleaner. > Make the dag number available in Context classes > ------------------------------------------------ > > Key: TEZ-2368 > URL: https://issues.apache.org/jira/browse/TEZ-2368 > Project: Apache Tez > Issue Type: Improvement > Reporter: Siddharth Seth > Assignee: Siddharth Seth > Attachments: TEZ-2368.1.txt > > > Provide the dag number, which is a unique number, for each dag running within > an application in the TezInputContext, TezOutputContext, TezProcessorContext. > When containers are re-used, or for external services, this can be used to > generate intermediate data to a dag specific directory instead of an > application specific directory, where it becomes difficult to differentiate > between different dags. > The DAG name does provide this - but is not suitable for use in a directory > name. Hashing the name is an option, but can lead to collisions. > Generating data into a dag specific directory will eventually only be usable > when we move away from the default MR handler, or enhance it to support an > additional parameter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)