[jira] [Commented] (PIG-4677) Display failure information on stop on failure
[
https://issues.apache.org/jira/browse/PIG-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951726#comment-15951726
]
Rohini Palaniswamy commented on PIG-4677:
-
Committed PIG-4677-fixflakytest.patch to trunk. Thanks Daniel for the review.
> Display failure information on stop on failure
> --
>
> Key: PIG-4677
> URL: https://issues.apache.org/jira/browse/PIG-4677
> Project: Pig
> Issue Type: Bug
>Affects Versions: 0.11.1
>Reporter: Mit Desai
>Assignee: Rohini Palaniswamy
> Fix For: 0.17.0
>
> Attachments: PIG-4677.2.patch, PIG-4677.3.patch, PIG-4677.4.patch,
> PIG-4677-5.patch, PIG-4677-fixflakytest.patch, PIG-4677.patch
>
>
> When stop on failure option is specified, pig abruptly exits without
> displaying any job stats or failed job information which it usually does in
> case of failures.
> {code}
> 2015-06-04 20:35:38,170 [uber-SubtaskRunner] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 9% complete
> 2015-06-04 20:35:38,171 [uber-SubtaskRunner] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Running jobs are
> [job_1428329756093_3741748,job_1428329756093_3741752,job_1428329756093_3741753,job_1428329756093_3741754,job_1428329756093_3741756]
> 2015-06-04 20:35:40,201 [uber-SubtaskRunner] ERROR
> org.apache.pig.tools.grunt.Grunt - ERROR 6017: Job failed!
> Hadoop Job IDs executed by Pig:
> job_1428329756093_3739816,job_1428329756093_3741752,job_1428329756093_3739814,job_1428329756093_3741748,job_1428329756093_3741756,job_1428329756093_3741753,job_1428329756093_3741754
> <<< Invocation of Main class completed <<<
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (PIG-4677) Display failure information on stop on failure
[
https://issues.apache.org/jira/browse/PIG-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951709#comment-15951709
]
Daniel Dai commented on PIG-4677:
-
+1 on PIG-4677-fixflakytest.patch.
> Display failure information on stop on failure
> --
>
> Key: PIG-4677
> URL: https://issues.apache.org/jira/browse/PIG-4677
> Project: Pig
> Issue Type: Bug
>Affects Versions: 0.11.1
>Reporter: Mit Desai
>Assignee: Rohini Palaniswamy
> Fix For: 0.17.0
>
> Attachments: PIG-4677.2.patch, PIG-4677.3.patch, PIG-4677.4.patch,
> PIG-4677-5.patch, PIG-4677-fixflakytest.patch, PIG-4677.patch
>
>
> When stop on failure option is specified, pig abruptly exits without
> displaying any job stats or failed job information which it usually does in
> case of failures.
> {code}
> 2015-06-04 20:35:38,170 [uber-SubtaskRunner] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 9% complete
> 2015-06-04 20:35:38,171 [uber-SubtaskRunner] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Running jobs are
> [job_1428329756093_3741748,job_1428329756093_3741752,job_1428329756093_3741753,job_1428329756093_3741754,job_1428329756093_3741756]
> 2015-06-04 20:35:40,201 [uber-SubtaskRunner] ERROR
> org.apache.pig.tools.grunt.Grunt - ERROR 6017: Job failed!
> Hadoop Job IDs executed by Pig:
> job_1428329756093_3739816,job_1428329756093_3741752,job_1428329756093_3739814,job_1428329756093_3741748,job_1428329756093_3741756,job_1428329756093_3741753,job_1428329756093_3741754
> <<< Invocation of Main class completed <<<
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (PIG-4677) Display failure information on stop on failure
[
https://issues.apache.org/jira/browse/PIG-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15948477#comment-15948477
]
Daniel Dai commented on PIG-4677:
-
+1
> Display failure information on stop on failure
> --
>
> Key: PIG-4677
> URL: https://issues.apache.org/jira/browse/PIG-4677
> Project: Pig
> Issue Type: Bug
>Affects Versions: 0.11.1
>Reporter: Mit Desai
>Assignee: Rohini Palaniswamy
> Fix For: 0.17.0
>
> Attachments: PIG-4677.2.patch, PIG-4677.3.patch, PIG-4677.4.patch,
> PIG-4677-5.patch, PIG-4677.patch
>
>
> When stop on failure option is specified, pig abruptly exits without
> displaying any job stats or failed job information which it usually does in
> case of failures.
> {code}
> 2015-06-04 20:35:38,170 [uber-SubtaskRunner] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 9% complete
> 2015-06-04 20:35:38,171 [uber-SubtaskRunner] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Running jobs are
> [job_1428329756093_3741748,job_1428329756093_3741752,job_1428329756093_3741753,job_1428329756093_3741754,job_1428329756093_3741756]
> 2015-06-04 20:35:40,201 [uber-SubtaskRunner] ERROR
> org.apache.pig.tools.grunt.Grunt - ERROR 6017: Job failed!
> Hadoop Job IDs executed by Pig:
> job_1428329756093_3739816,job_1428329756093_3741752,job_1428329756093_3739814,job_1428329756093_3741748,job_1428329756093_3741756,job_1428329756093_3741753,job_1428329756093_3741754
> <<< Invocation of Main class completed <<<
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (PIG-4677) Display failure information on stop on failure
[
https://issues.apache.org/jira/browse/PIG-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900703#comment-14900703
]
Mit Desai commented on PIG-4677:
The ExecException message is different in the log here. I forgot to rerun the
test after modifying the error message.
> Display failure information on stop on failure
> --
>
> Key: PIG-4677
> URL: https://issues.apache.org/jira/browse/PIG-4677
> Project: Pig
> Issue Type: Bug
>Affects Versions: 0.11.1
>Reporter: Mit Desai
>Assignee: Mit Desai
> Attachments: PIG-4677.2.patch, PIG-4677.3.patch, PIG-4677.patch
>
>
> When stop on failure option is specified, pig abruptly exits without
> displaying any job stats or failed job information which it usually does in
> case of failures.
> {code}
> 2015-06-04 20:35:38,170 [uber-SubtaskRunner] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 9% complete
> 2015-06-04 20:35:38,171 [uber-SubtaskRunner] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Running jobs are
> [job_1428329756093_3741748,job_1428329756093_3741752,job_1428329756093_3741753,job_1428329756093_3741754,job_1428329756093_3741756]
> 2015-06-04 20:35:40,201 [uber-SubtaskRunner] ERROR
> org.apache.pig.tools.grunt.Grunt - ERROR 6017: Job failed!
> Hadoop Job IDs executed by Pig:
> job_1428329756093_3739816,job_1428329756093_3741752,job_1428329756093_3739814,job_1428329756093_3741748,job_1428329756093_3741756,job_1428329756093_3741753,job_1428329756093_3741754
> <<< Invocation of Main class completed <<<
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PIG-4677) Display failure information on stop on failure
[
https://issues.apache.org/jira/browse/PIG-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14876652#comment-14876652
]
Rohini Palaniswamy commented on PIG-4677:
-
It should still be assertFalse(server.existsFile("done")); . With this change
it is still not stopping execution of the script when it is compiled in two
phases due to fs statements. Will have to make checkStopOnFailure return true
instead of void, and throw new ExecException(msg.toString(), errCode,
PigException.REMOTE_ENVIRONMENT); instead of return pigStats at the end in
launchPig if checkStopOnFailure returned true.
> Display failure information on stop on failure
> --
>
> Key: PIG-4677
> URL: https://issues.apache.org/jira/browse/PIG-4677
> Project: Pig
> Issue Type: Bug
>Affects Versions: 0.11.1
>Reporter: Mit Desai
>Assignee: Mit Desai
> Attachments: PIG-4677.2.patch, PIG-4677.patch
>
>
> When stop on failure option is specified, pig abruptly exits without
> displaying any job stats or failed job information which it usually does in
> case of failures.
> {code}
> 2015-06-04 20:35:38,170 [uber-SubtaskRunner] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 9% complete
> 2015-06-04 20:35:38,171 [uber-SubtaskRunner] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Running jobs are
> [job_1428329756093_3741748,job_1428329756093_3741752,job_1428329756093_3741753,job_1428329756093_3741754,job_1428329756093_3741756]
> 2015-06-04 20:35:40,201 [uber-SubtaskRunner] ERROR
> org.apache.pig.tools.grunt.Grunt - ERROR 6017: Job failed!
> Hadoop Job IDs executed by Pig:
> job_1428329756093_3739816,job_1428329756093_3741752,job_1428329756093_3739814,job_1428329756093_3741748,job_1428329756093_3741756,job_1428329756093_3741753,job_1428329756093_3741754
> <<< Invocation of Main class completed <<<
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PIG-4677) Display failure information on stop on failure
[
https://issues.apache.org/jira/browse/PIG-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791270#comment-14791270
]
Rohini Palaniswamy commented on PIG-4677:
-
[~mitdesai],
checkStopOnFailure currently throws an exception which makes it exit early.
So it ends up skipping cleanupOnFailure, printing "Job Stats" information, etc.
i.e all code starting from
MRScriptState.get().emitProgressUpdatedNotification(100);.
So instead of throwing exception and exiting early and relying on system
shutdown hook to kill the remaining jobs, you will have to call
failJob(message) on JobControl.getReadyJobsList() and
JobControl.getRunningJobList() in checkStopOnFailure. That will kill all jobs
and will then follow the regular code path.
> Display failure information on stop on failure
> --
>
> Key: PIG-4677
> URL: https://issues.apache.org/jira/browse/PIG-4677
> Project: Pig
> Issue Type: Bug
>Affects Versions: 0.11.1
>Reporter: Mit Desai
>Assignee: Mit Desai
> Attachments: PIG-4677.patch
>
>
> When stop on failure option is specified, pig abruptly exits without
> displaying any job stats or failed job information which it usually does in
> case of failures.
> {code}
> 2015-06-04 20:35:38,170 [uber-SubtaskRunner] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 9% complete
> 2015-06-04 20:35:38,171 [uber-SubtaskRunner] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Running jobs are
> [job_1428329756093_3741748,job_1428329756093_3741752,job_1428329756093_3741753,job_1428329756093_3741754,job_1428329756093_3741756]
> 2015-06-04 20:35:40,201 [uber-SubtaskRunner] ERROR
> org.apache.pig.tools.grunt.Grunt - ERROR 6017: Job failed!
> Hadoop Job IDs executed by Pig:
> job_1428329756093_3739816,job_1428329756093_3741752,job_1428329756093_3739814,job_1428329756093_3741748,job_1428329756093_3741756,job_1428329756093_3741753,job_1428329756093_3741754
> <<< Invocation of Main class completed <<<
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
