[
https://issues.apache.org/jira/browse/PIG-1854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Richard Ding updated PIG-1854:
------------------------------
Attachment: PIG-1854.patch
The problem is that Pig doesn't check the return code while calling method
FsShell.run. This works fine in interactive mode since the error message is
written to the stdout. The fix checks the return code and throws an IOException
in batch mode.
> Pig returns exit code 0 for the failed Pig script .
> ---------------------------------------------------
>
> Key: PIG-1854
> URL: https://issues.apache.org/jira/browse/PIG-1854
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.7.0, 0.8.0
> Reporter: Peeyush Bishnoi
> Assignee: Richard Ding
> Fix For: 0.8.0, 0.9.0
>
> Attachments: PIG-1854.patch
>
>
> Pig returns exit code 0 for the some of the failed Pig scripts. Due to this
> workflow system like Oozie showing the "succeeded" status instead of "killed"
> status.
> Illustrative example
> ---------------------
> Suppose in Pig script we have following statement:
> $ cat move.pig
> fs -mv /user/jerry/a.txt /user/tom/
> On executing the pig script, script failed with following exception
> [tom@abc ]$ pig move.pig
> 2011-02-14 06:48:08,354 [main] INFO org.apache.pig.Main - Logging error
> messages to: /home/tom/pig_1297666088350.log
> 2011-02-14 06:48:08,652 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
> to hadoop file system at: hdfs://abc-nn:8020
> 2011-02-14 06:48:09,542 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
> to map-reduce job tracker at: abc-jt:50300
> mv: org.apache.hadoop.security.AccessControlException: Permission denied:
> user=tom, access=WRITE, inode="jobs":jerry:users:rwxr-xr-x
> [tom@abc ]$
> But when executed the similar Pig script through Oozie workflow, Pig script
> failed but Oozie show the "succeeded" status message instead of "killed"
> status as return exit code is 0.
> From logs of workflow launcher job:
> stdout logs
> -----------
> >>> Invoking Pig command line now >>>
> Apache Pig version 0.7.0.20.100.1.1007220309 (r966485)
> compiled Jul 22 2010, 03:09:21
> 807 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
> to hadoop file system at: hdfs://abc-nn:8020/
> 853 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
> to map-reduce job tracker at: abc-jt:50300
> <<< Invocation of Pig command completed <<<
> Hadoop Job IDs executed by Pig:
> <<< Invocation of Main class completed <<<
> Oozie Launcher, capturing output data:
> =======================
> #
> #Mon Feb 14 06:46:00 UTC 2011
> hadoopJobs=
> =======================
> Oozie Launcher ends
> stderr logs
> -----------
> mv: org.apache.hadoop.security.AccessControlException: Permission denied:
> user=tom, access=WRITE, inode="jobs":jerry:users:rwxr-xr-x
> One more scenario where Pig script failed but exit code is 0 and no exception
> occurred for this failure.
> stdout logs
> -----------
> >>> Invoking Pig command line now >>>
> Apache Pig version 0.7.0.20.100.1.1007220309 (r966485)
> compiled Jul 22 2010, 03:09:21
> ...
> ...
> 48247 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 100% complete
> 48248 [main] ERROR
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 1 map reduce job(s) failed!
> 48306 [main] ERROR
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Failed to produce result in: "hdfs://abc-nn/user/jerry/feature-keys"
> 48307 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Failed!
> <<< Invocation of Pig command completed <<<
> Hadoop Job IDs executed by Pig: job_201102080811_28852
> <<< Invocation of Main class completed <<<
> Oozie Launcher, capturing output data:
> =======================
> #
> #Sat Feb 12 04:13:53 UTC 2011
> hadoopJobs=job_201102080811_28852
> =======================
> Oozie Launcher ends
> ---
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
