[
https://issues.apache.org/jira/browse/PIG-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172519#comment-13172519
]
xuting zhao commented on PIG-2429:
----------------------------------
Hi Anitha,
Thanks for the help. I have run the tests in MapReduce mode in Pig 0.9 and
trunk, it all seems give the correct error message. I think this should be a
Hadoop problem or communication problem between Hadoop and Pig.
The mapRep retrieves the running reports of the Map tasks that are running
on Hadoop and then Pig will use the information stored in the reports to create
different types of exception and error messages. If the mapRep is null, no
exception information can be retrieved and then the wrong error is thrown.
> Better error message in pig_*.log
> ---------------------------------
>
> Key: PIG-2429
> URL: https://issues.apache.org/jira/browse/PIG-2429
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Anitha Raju
> Assignee: xuting zhao
> Priority: Minor
>
> Hi,
> The below script sometimes fails giving null error message and some times
> with correct error message in the pig_*.log
> Script
> {code}
> register 'a.py' using jython as test;
> A = load 'test.txt' as (x:chararray);
> B = foreach A generate
> test.a(x) as y;
> C = group B by (y);
> store C into 'C';
> {code}
> where a.py is
> {code}
> @outputSchema("n:chararray")
> def a(n):
> n = n.lower();
> return n;
> {code}
> input- test.txt
> =======
> a
> =======
> I have not put null check in a.py so that the script would error out.
> This script when run with 0.9 version, fails printing in the pig_*.log either
> null error message or the correct error message
> Null Error message
> {code}
> Pig Stack Trace
> ---------------
> ERROR 2244: Job failed, hadoop does not return any error message
> org.apache.pig.backend.executionengine.ExecException: ERROR 2244: Job failed,
> hadoop does not return any error message
> at
> org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:139)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:192)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
> at org.apache.pig.Main.run(Main.java:561)
> at org.apache.pig.Main.main(Main.java:111)
> {code}
> Correct Error message
> {code}
> Backend error message
> ---------------------
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Error
> executing function
> at
> org.apache.pig.scripting.jython.JythonFunction.exec(JythonFunction.java:106)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:216)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:305)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:322)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:332)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:284)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:290)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:256)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:267)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:262)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:261)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> at org.apache.hadoop.mapred.Child.main(Child.java:255)
> Caused by: Traceback (most recent call last):
> File "a.py", line 5, in n
> AttributeError: 'NoneType' object has no attribute 'lower'
> at org.python.core.PyException.fillInStackTrace(PyException.java:70)
> at java.lang.Throwable.<init>(Throwable.java:181)
> at java.lang.Exception.<init>(Exception.java:29)
> at java.lang.RuntimeException.<init>(RuntimeException.java:32)
> at org.python.core.PyException.<init>(PyException.java:46)
> at org.python.core.PyException.<init>(PyException.java:43)
> at org.python.core.PyException.<init>(PyException.java:61)
> at org.python.core.Py.AttributeError(Py.java:145)
> at org.python.core.PyObject.noAttributeError(PyObject.java:936)
> at org.python.core.PyObject.__getattr__(PyObject.java:931)
> at org.python.pycode._pyx3.n$1(a.py:6)
> at org.python.pycode._pyx3.call_function(a.py)
> at org.python.core.PyTableCode.call(PyTableCode.java:165)
> at org.python.core.PyBaseCode.call(PyBaseCode.java:297)
> at org.python.core.PyFunction.__call__(PyFunction.java:370)
> at org.python.core.PyFunction.__call__(PyFunction.java:360)
> at org.python.core.PyFunction.__call__(PyFunction.java:355)
> at
> org.apache.pig.scripting.jython.JythonFunction.exec(JythonFunction.java:103)
> ... 18 more
>
> {code}
> Putting a couple of sysout in the code, i found it was happening when in
> Launcher.getStats
> {code}
> TaskReport[] mapRep = jobClient.getMapTaskReports(MRJobID);
> getErrorMessages(mapRep, "map", errNotDbg, pigContext);
> totalHadoopTimeSpent += computeTimeSpent(mapRep);
> {code}
> whenever mapRed becomes null, it gives null error message.
> Can it be made better?
> Regards,
> Anitha
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
