[ https://issues.apache.org/jira/browse/PIG-3048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13752934#comment-13752934 ]
Daniel Dai commented on PIG-3048: --------------------------------- The patch looks fine for me. Yes we can have different ways to represent track a job as Bill indicated, which might already exist in Pig. However, the key here is we also need a way to capture this information consistently across multiple projects (such as Hive). Since Hive already adopted HIVE-3708, it is meaningful to check in this patch, so upper layer application such as Ambari can make of this in a consistent manner. I would like to check it in if no objection. > Add mapreduce workflow information to job configuration > ------------------------------------------------------- > > Key: PIG-3048 > URL: https://issues.apache.org/jira/browse/PIG-3048 > Project: Pig > Issue Type: Improvement > Reporter: Billie Rinaldi > Assignee: Billie Rinaldi > Fix For: 0.11.2 > > Attachments: PIG-3048.patch, PIG-3048.patch, PIG-3048.patch > > > Adding workflow properties to the job configuration would enable logging and > analysis of workflows in addition to individual MapReduce jobs. Suggested > properties include a workflow ID, workflow name, adjacency list connecting > nodes in the workflow, and the name of the current node in the workflow. > mapreduce.workflow.id - a unique ID for the workflow, ideally prepended with > the application name > e.g. pig_<pigScriptId> > mapreduce.workflow.name - a name for the workflow, to distinguish this > workflow from other workflows and to group different runs of the same workflow > e.g. pig command line > mapreduce.workflow.adjacency - an adjacency list for the workflow graph, > encoded as mapreduce.workflow.adjacency.<source node> = <comma-separated list > of target nodes> > mapreduce.workflow.node.name - the name of the node corresponding to this > MapReduce job in the workflow adjacency list -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira