[
https://issues.apache.org/jira/browse/PIG-2481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Martin updated PIG-2481:
-----------------------------
Attachment: illustrate_bug.pig
Cleaning up the sample script a little bit. The previous version had some
extra "REGISTER ..." statements which I had added in order to test the script
with Pig 0.8.1.
> Illustrate throws NullPointerException when custom load function uses counters
> ------------------------------------------------------------------------------
>
> Key: PIG-2481
> URL: https://issues.apache.org/jira/browse/PIG-2481
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.1
> Environment: Apache Pig 0.9.1 and JSON Loader from ElephantBird 2.1.1
> (i.e. all changes through the following commit:
> https://github.com/kevinweil/elephant-bird/commit/13dda2a1de31844fe07bcf4a58e94fffcb9aa374)
> Reporter: Matt Martin
> Priority: Minor
> Attachments: illustrate_bug.pig, illustrate_bug.pig,
> illustrate_bug.txt, pig_1326905442471.log
>
>
> Pig throws the following NullPointerException while trying to use
> "illustrate" in conjunction with ElephantBird's JSON loader:
> java.lang.NullPointerException
> at
> org.apache.hadoop.mapreduce.TaskInputOutputContext.getCounter(TaskInputOutputContext.java:84)
> at
> org.apache.pig.tools.pigstats.PigStatusReporter.getCounter(PigStatusReporter.java:55)
> at
> com.twitter.elephantbird.pig.util.PigCounterHelper.incrCounter(PigCounterHelper.java:54)
> at
> com.twitter.elephantbird.pig.load.JsonLoader.incrCounter(JsonLoader.java:91)
> at
> com.twitter.elephantbird.pig.load.JsonLoader.getNext(JsonLoader.java:55)
> at
> org.apache.pig.impl.io.ReadToEndLoader.getNextHelper(ReadToEndLoader.java:210)
> at
> org.apache.pig.impl.io.ReadToEndLoader.getNext(ReadToEndLoader.java:190)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad.getNext(POLoad.java:129)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:267)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:262)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at
> org.apache.pig.pen.LocalMapReduceSimulator.launchPig(LocalMapReduceSimulator.java:194)
> at
> org.apache.pig.pen.ExampleGenerator.getData(ExampleGenerator.java:257)
> at
> org.apache.pig.pen.ExampleGenerator.readBaseData(ExampleGenerator.java:222)
> at
> org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:154)
> at org.apache.pig.PigServer.getExamples(PigServer.java:1245)
> at
> org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:698)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.Illustrate(PigScriptParser.java:591)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:306)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
> at org.apache.pig.Main.run(Main.java:553)
> at org.apache.pig.Main.main(Main.java:108)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> I have verified that this issue does not occur if the ElephantBird JSON
> loader is not used so the issue seems to be related to the interaction
> between Pig and a Custom Loader (in this case, it seems to be related to the
> way in which Pig's "illustrate" functionality handles custom counters). I've
> also verified that this issue does not occur with Pig 0.8.1 regardless of
> whether the ElephantBird JSON loader is used or not, so this seems to be a
> Pig-specific issue (as opposed to ElephantBird-related).
> I'll attach sample input and script for reference.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
