[ https://issues.apache.org/jira/browse/PIG-908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich updated PIG-908: ------------------------------- With Pig 0.8.0 we print a summary of the execution that contains (among other things) how aliases mapped to jobs. Example: JobId Maps Reduces MaxMapTime MinMapTIme AvgMapTime MaxReduceTime MinReduceTime AvgReduceTime Alias Feature Outputs job_201004271216_12712 1 1 3 3 3 12 12 12 B,C GROUP_BY,COMBINER job_201004271216_12713 1 1 3 3 3 12 12 12 D SAMPLER job_201004271216_12714 1 1 3 3 3 12 12 12 D ORDER_BY,COMBINER hdfs://wilbur20.labs.corp.sp1.yahoo.com:9020/tmp/temp743703298/tmp-2019944040, > Need a way to correlate MR jobs with Pig statements > --------------------------------------------------- > > Key: PIG-908 > URL: https://issues.apache.org/jira/browse/PIG-908 > Project: Pig > Issue Type: Wish > Reporter: Dmitriy V. Ryaboy > Assignee: Richard Ding > Fix For: 0.8.0 > > > Complex Pig Scripts often generate many Map-Reduce jobs, especially with the > recent introduction of multi-store capabilities. > For example, the first script in the Pig tutorial produces 5 MR jobs. > There is currently very little support for debugging resulting jobs; if one > of the MR jobs fails, it is hard to figure out which part of the script it > was responsible for. Explain plans help, but even with the explain plan, a > fair amount of effort (and sometimes, experimentation) is required to > correlate the failing MR job with the corresponding PigLatin statements. > This ticket is created to discuss approaches to alleviating this problem. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.