[
https://issues.apache.org/jira/browse/PIG-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated PIG-2170:
-------------------------------
Attachment: PIG-2170.2.patch
PIG-2170.2.patch fixes a findbugs warning in PIG-2170.1.patch.
Changes -
- BinCond is not a boolean expression (ie it does not return boolean value), so
equivalence classes are not created for it.
- Boolean expressions can appear in foreach (within bincond), and every where
else after boolean type is introduced. But adding equivalence classes for each
boolean expression in all these places will generate too many examples,
affecting the conciseness. So I have chosen not to add equivalence classes in
those cases.
- When type conversion errors happened, pig tries to increment counters for it,
and was throwing NPE because the local mode simulation did not supply the
object that returns counters. That has also been fixed.
> NPE thrown during illustrate
> ----------------------------
>
> Key: PIG-2170
> URL: https://issues.apache.org/jira/browse/PIG-2170
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.10
> Reporter: Mat Kelcey
> Assignee: Thejas M Nair
> Attachments: PIG-2170.1.patch, PIG-2170.2.patch
>
>
> working with version
> https://svn.apache.org/repos/asf/pig/trunk@1146777
> fetched from git
> git://git.apache.org/pig.git a7e1228a0fdfe76c3cff0e749e252dba8d387052
> using file /tmp/data.tsv
> id1 123
> id1 234
> id2 345
> id2 456
> this is the most cutdown/simplest script i can make that illustrates (no pun
> intended) the problem
> grunt> data = load '/tmp/data.tsv' as (id:chararray, value:long);
> grunt> cogrouped = cogroup data by id;
> grunt> exists = foreach cogrouped generate (IsEmpty(data.value) ? 0 : 1) as
> exists;
> grunt> dump exists
> is ok
> but
> grunt> illustrate exists
> throws
> java.lang.NullPointerException
> at
> org.apache.pig.pen.IllustratorAttacher.visitBinCond(IllustratorAttacher.java:360)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POBinCond.visit(POBinCond.java:145)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POBinCond.visit(POBinCond.java:36)
> at
> org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
> at
> org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
> at
> org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
> at
> org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
> at
> org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:52)
> at
> org.apache.pig.pen.IllustratorAttacher.innerPlanAttach(IllustratorAttacher.java:417)
> at
> org.apache.pig.pen.IllustratorAttacher.visitPOForEach(IllustratorAttacher.java:229)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.visit(POForEach.java:117)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.visit(POForEach.java:47)
> at
> org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
> at
> org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
> at
> org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
> at
> org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
> at
> org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
> at
> org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
> at
> org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:52)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at
> org.apache.pig.pen.ExampleGenerator.getData(ExampleGenerator.java:246)
> at
> org.apache.pig.pen.ExampleGenerator.getData(ExampleGenerator.java:238)
> at
> org.apache.pig.pen.LineageTrimmingVisitor.init(LineageTrimmingVisitor.java:103)
> at
> org.apache.pig.pen.LineageTrimmingVisitor.<init>(LineageTrimmingVisitor.java:98)
> at
> org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:166)
> at org.apache.pig.PigServer.getExamples(PigServer.java:1201)
> at
> org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:698)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.Illustrate(PigScriptParser.java:591)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:306)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:67)
> at org.apache.pig.Main.run(Main.java:487)
> at org.apache.pig.Main.main(Main.java:108)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> with pig.log containing
> java.io.IOException: Exception : null
> at org.apache.pig.PigServer.getExamples(PigServer.java:1207)
> at
> org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:698)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.Illustrate(PigScriptParser.java:591)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:306)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:67)
> at org.apache.pig.Main.run(Main.java:487)
> at org.apache.pig.Main.main(Main.java:108)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> something to do with the IsEmpty since...
> grunt> data = load '/tmp/data.tsv' as (id:chararray, value:long);
> grunt> cogrouped = cogroup data by id;
> grunt> exists = foreach cogrouped generate data.value as exists;
> illustrates ok
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
