-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/439/
-----------------------------------------------------------
(Updated 2011-02-22 09:58:48.484463)
Review request for pig and Richard Ding.
Summary (updated)
-------
The below is my script :
register myanotherudf.jar;
A = load 'myinput' using PigStorage() as (
date:chararray,bcookie:chararray,count:int,avg:double,pvs:int);
B = foreach A generate (int)(avg / 100.0) * 100 as avg, pvs;
C = group B by ( avg );
D = foreach C {
Pvs = order B by pvs;
Const = org.vivek.MyAnotherUDF(Pvs.pvs).(count,sum);
generate Const.sum as sum;
};
store D into 'out_D';
The script is failing during compilation of the plan. The usage of the udf
inside the foreach is causing the problem. The udf implements algebraic and the
output schema is also defined.
The below is the exception that I get :
ERROR 2042: Error in new logical plan. Try -Dpig.usenewlogicalplan=false.
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2042: Error in new
logical plan. Try -Dpig.usenewlogicalplan=false.
at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:309)
at org.apache.pig.PigServer.compilePp(PigServer.java:1364)
at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1206)
at org.apache.pig.PigServer.execute(PigServer.java:1200)
at org.apache.pig.PigServer.access$100(PigServer.java:128)
at org.apache.pig.PigServer$Graph.execute(PigServer.java:1527)
at org.apache.pig.PigServer.executeBatchEx(PigServer.java:372)
at org.apache.pig.PigServer.executeBatch(PigServer.java:339)
at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:112)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:169)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
at org.apache.pig.Main.run(Main.java:500)
at org.apache.pig.Main.main(Main.java:107)
Caused by: java.lang.NullPointerException
at
org.apache.pig.newplan.ReverseDependencyOrderWalker.walk(ReverseDependencyOrderWalker.java:70)
at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
at
org.apache.pig.newplan.logical.optimizer.SchemaResetter.visit(SchemaResetter.java:105)
at
org.apache.pig.newplan.logical.relational.LOGenerate.accept(LOGenerate.java:229)
at
org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
at
org.apache.pig.newplan.logical.optimizer.SchemaResetter.visit(SchemaResetter.java:94)
at org.apache.pig.newplan.logical.relational.LOForEach.accept(LOForEach.java:71)
at
org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:261)
... 13 more
This addresses bug PIG-1858.
https://issues.apache.org/jira/browse/PIG-1858
Diffs
-----
http://svn.apache.org/repos/asf/pig/branches/branch-0.8/src/org/apache/pig/impl/logicalLayer/parser/QueryParser.jjt
1072544
http://svn.apache.org/repos/asf/pig/branches/branch-0.8/test/org/apache/pig/test/TestEvalPipeline2.java
1072544
Diff: https://reviews.apache.org/r/439/diff
Testing
-------
test-patch:
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or
modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning
messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number
of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs
warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the
total number of release audit warnings.
unit test:
all pass
end to end test:
all pass
Thanks,
Daniel
