[
https://issues.apache.org/jira/browse/PIG-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anitha Raju updated PIG-2429:
-----------------------------
Description:
Hi,
The below script sometimes fails giving null error message and some times with
correct error message in the pig_*.log
Script
{code}
register 'a.py' using jython as test;
A = load 'test.txt' as (x:chararray);
B = foreach A generate
test.a(x) as y;
C = group B by (y);
store C into 'C';
{code}
where a.py is
{code}
@outputSchema("n:chararray")
def a(n):
n = n.lower();
return n;
{code}
input- test.txt
=======
a
=======
I have not put null check in a.py so that the script would error out.
This script when run with 0.9 version, fails printing in the pig_*.log either
null error message or the correct error message
Null Error message
{code}
Pig Stack Trace
---------------
ERROR 2244: Job failed, hadoop does not return any error message
org.apache.pig.backend.executionengine.ExecException: ERROR 2244: Job failed,
hadoop does not return any error message
at
org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:139)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:192)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
at org.apache.pig.Main.run(Main.java:561)
at org.apache.pig.Main.main(Main.java:111)
{code}
Correct Error message
{code}
Backend error message
---------------------
org.apache.pig.backend.executionengine.ExecException: ERROR 0: Error executing
function
at
org.apache.pig.scripting.jython.JythonFunction.exec(JythonFunction.java:106)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:216)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:305)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:322)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:332)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:284)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:290)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:256)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:267)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:262)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:261)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.mapred.Child.main(Child.java:255)
Caused by: Traceback (most recent call last):
File "a.py", line 5, in n
AttributeError: 'NoneType' object has no attribute 'lower'
at org.python.core.PyException.fillInStackTrace(PyException.java:70)
at java.lang.Throwable.<init>(Throwable.java:181)
at java.lang.Exception.<init>(Exception.java:29)
at java.lang.RuntimeException.<init>(RuntimeException.java:32)
at org.python.core.PyException.<init>(PyException.java:46)
at org.python.core.PyException.<init>(PyException.java:43)
at org.python.core.PyException.<init>(PyException.java:61)
at org.python.core.Py.AttributeError(Py.java:145)
at org.python.core.PyObject.noAttributeError(PyObject.java:936)
at org.python.core.PyObject.__getattr__(PyObject.java:931)
at org.python.pycode._pyx3.n$1(a.py:6)
at org.python.pycode._pyx3.call_function(a.py)
at org.python.core.PyTableCode.call(PyTableCode.java:165)
at org.python.core.PyBaseCode.call(PyBaseCode.java:297)
at org.python.core.PyFunction.__call__(PyFunction.java:370)
at org.python.core.PyFunction.__call__(PyFunction.java:360)
at org.python.core.PyFunction.__call__(PyFunction.java:355)
at
org.apache.pig.scripting.jython.JythonFunction.exec(JythonFunction.java:103)
... 18 more
{code}
Putting a couple of sysout in the code, i found it was happening when in
Launcher.getStats
{code}
TaskReport[] mapRep = jobClient.getMapTaskReports(MRJobID);
getErrorMessages(mapRep, "map", errNotDbg, pigContext);
totalHadoopTimeSpent += computeTimeSpent(mapRep);
{code}
whenever mapRed becomes null, it gives null error message.
Can it be made better?
Regards,
Anitha
was:
Hi,
The below script sometimes fails giving null error message and some times with
correct error message in the pig_*.log
Script
{code}
register 'a.py' using jython as test;
A = load 'test.txt' as (x:chararray);
B = foreach A generate
test.a(x) as y;
C = group B by (y);
store C into 'C';
{code}
where a.py is
{code}
@outputSchema("n:chararray")
def a(n):
n = n.lower();
return n;
{code}
input- test.txt
{code}
a
{code}
I have not put null check in a.py so that the script would error out.
This script when run with 0.9 version, fails printing in the pig_*.log either
null error message or the correct error message
Null Error message
===============================================================================
Pig Stack Trace
---------------
ERROR 2244: Job failed, hadoop does not return any error message
org.apache.pig.backend.executionengine.ExecException: ERROR 2244: Job failed,
hadoop does not return any error message
at
org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:139)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:192)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
at org.apache.pig.Main.run(Main.java:561)
at org.apache.pig.Main.main(Main.java:111)
================================================================================
Correct Error message
===================
Backend error message
---------------------
org.apache.pig.backend.executionengine.ExecException: ERROR 0: Error executing
function
at
org.apache.pig.scripting.jython.JythonFunction.exec(JythonFunction.java:106)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:216)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:305)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:322)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:332)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:284)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:290)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:256)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:267)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:262)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:261)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.mapred.Child.main(Child.java:255)
Caused by: Traceback (most recent call last):
File "a.py", line 5, in n
AttributeError: 'NoneType' object has no attribute 'lower'
at org.python.core.PyException.fillInStackTrace(PyException.java:70)
at java.lang.Throwable.<init>(Throwable.java:181)
at java.lang.Exception.<init>(Exception.java:29)
at java.lang.RuntimeException.<init>(RuntimeException.java:32)
at org.python.core.PyException.<init>(PyException.java:46)
at org.python.core.PyException.<init>(PyException.java:43)
at org.python.core.PyException.<init>(PyException.java:61)
at org.python.core.Py.AttributeError(Py.java:145)
at org.python.core.PyObject.noAttributeError(PyObject.java:936)
at org.python.core.PyObject.__getattr__(PyObject.java:931)
at org.python.pycode._pyx3.n$1(a.py:6)
at org.python.pycode._pyx3.call_function(a.py)
at org.python.core.PyTableCode.call(PyTableCode.java:165)
at org.python.core.PyBaseCode.call(PyBaseCode.java:297)
at org.python.core.PyFunction.__call__(PyFunction.java:370)
at org.python.core.PyFunction.__call__(PyFunction.java:360)
at org.python.core.PyFunction.__call__(PyFunction.java:355)
at
org.apache.pig.scripting.jython.JythonFunction.exec(JythonFunction.java:103)
... 18 more
=======================
Putting a couple of sysout in the code, i found it was happening when in
Launcher.getStats
------------------------------
TaskReport[] mapRep = jobClient.getMapTaskReports(MRJobID);
System.out.println("going to get error message");
getErrorMessages(mapRep, "map", errNotDbg, pigContext);
System.out.println("got the error message");
totalHadoopTimeSpent += computeTimeSpent(mapRep);
------------------------------
whenever mapRed becomes null, it gives null error message.
Can it be made better?
Regards,
Anitha
> Better error message in pig_*.log
> ---------------------------------
>
> Key: PIG-2429
> URL: https://issues.apache.org/jira/browse/PIG-2429
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Anitha Raju
> Priority: Minor
>
> Hi,
> The below script sometimes fails giving null error message and some times
> with correct error message in the pig_*.log
> Script
> {code}
> register 'a.py' using jython as test;
> A = load 'test.txt' as (x:chararray);
> B = foreach A generate
> test.a(x) as y;
> C = group B by (y);
> store C into 'C';
> {code}
> where a.py is
> {code}
> @outputSchema("n:chararray")
> def a(n):
> n = n.lower();
> return n;
> {code}
> input- test.txt
> =======
> a
> =======
> I have not put null check in a.py so that the script would error out.
> This script when run with 0.9 version, fails printing in the pig_*.log either
> null error message or the correct error message
> Null Error message
> {code}
> Pig Stack Trace
> ---------------
> ERROR 2244: Job failed, hadoop does not return any error message
> org.apache.pig.backend.executionengine.ExecException: ERROR 2244: Job failed,
> hadoop does not return any error message
> at
> org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:139)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:192)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
> at org.apache.pig.Main.run(Main.java:561)
> at org.apache.pig.Main.main(Main.java:111)
> {code}
> Correct Error message
> {code}
> Backend error message
> ---------------------
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Error
> executing function
> at
> org.apache.pig.scripting.jython.JythonFunction.exec(JythonFunction.java:106)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:216)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:305)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:322)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:332)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:284)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:290)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:256)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:267)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:262)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:261)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> at org.apache.hadoop.mapred.Child.main(Child.java:255)
> Caused by: Traceback (most recent call last):
> File "a.py", line 5, in n
> AttributeError: 'NoneType' object has no attribute 'lower'
> at org.python.core.PyException.fillInStackTrace(PyException.java:70)
> at java.lang.Throwable.<init>(Throwable.java:181)
> at java.lang.Exception.<init>(Exception.java:29)
> at java.lang.RuntimeException.<init>(RuntimeException.java:32)
> at org.python.core.PyException.<init>(PyException.java:46)
> at org.python.core.PyException.<init>(PyException.java:43)
> at org.python.core.PyException.<init>(PyException.java:61)
> at org.python.core.Py.AttributeError(Py.java:145)
> at org.python.core.PyObject.noAttributeError(PyObject.java:936)
> at org.python.core.PyObject.__getattr__(PyObject.java:931)
> at org.python.pycode._pyx3.n$1(a.py:6)
> at org.python.pycode._pyx3.call_function(a.py)
> at org.python.core.PyTableCode.call(PyTableCode.java:165)
> at org.python.core.PyBaseCode.call(PyBaseCode.java:297)
> at org.python.core.PyFunction.__call__(PyFunction.java:370)
> at org.python.core.PyFunction.__call__(PyFunction.java:360)
> at org.python.core.PyFunction.__call__(PyFunction.java:355)
> at
> org.apache.pig.scripting.jython.JythonFunction.exec(JythonFunction.java:103)
> ... 18 more
>
> {code}
> Putting a couple of sysout in the code, i found it was happening when in
> Launcher.getStats
> {code}
> TaskReport[] mapRep = jobClient.getMapTaskReports(MRJobID);
> getErrorMessages(mapRep, "map", errNotDbg, pigContext);
> totalHadoopTimeSpent += computeTimeSpent(mapRep);
> {code}
> whenever mapRed becomes null, it gives null error message.
> Can it be made better?
> Regards,
> Anitha
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
