[ https://issues.apache.org/jira/browse/PIG-2313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Daniels updated PIG-2313: ------------------------------ Attachment: PIG-2313-first_shot.patch PIG-2313-example.tar.gz > NPE in ILLUSTRATE trying to get StatusReporter in STORE > ------------------------------------------------------- > > Key: PIG-2313 > URL: https://issues.apache.org/jira/browse/PIG-2313 > Project: Pig > Issue Type: Bug > Affects Versions: 0.9.0, 0.10 > Reporter: Doug Daniels > Attachments: PIG-2313-example.tar.gz, PIG-2313-first_shot.patch > > > I'm seeing an NPE trying to do an illustrate on a script. So far the > simplest version of the script that exhibits the issue is: > {code} > raw = LOAD 'data.txt' USING PigStorage() AS (x:int, y:int); > filtered = FILTER raw BY x < 5; > grouped = GROUP filtered BY x; > counted = FOREACH grouped > GENERATE group AS x, > COUNT(filtered) AS the_count; > rmf output; > STORE counted INTO 'output'; > {code} > I had to pass a few nested Exceptions along to get it, but the bottom stack > trace looks like: > {code} > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.mapreduce.TaskInputOutputContext.getCounter(TaskInputOutputContext.java:88) > at > org.apache.pig.tools.pigstats.PigStatusReporter.getCounter(PigStatusReporter.java:60) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReducePOStoreImpl.createRecordCounter(MapReducePOStoreImpl.java:121) > at > org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.setUp(POStore.java:108) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.cleanup(PigGenericMapReduce.java:525) > at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:178) > at > org.apache.pig.pen.LocalMapReduceSimulator.launchPig(LocalMapReduceSimulator.java:222) > at > org.apache.pig.pen.ExampleGenerator.getData(ExampleGenerator.java:257) > at > org.apache.pig.pen.ExampleGenerator.getData(ExampleGenerator.java:275) > at > org.apache.pig.pen.LineageTrimmingVisitor.checkNewBaseData(LineageTrimmingVisitor.java:418) > ... 21 more > {code} > It looks like the IllustratorContext in the hadoop20 PigMapReduce.java shim > (line 73) is getting setup with a null StatusReporter. This seems to be for > the Reduce phase. On the other hand, the PigMapBase.java sets up the > IllustratorContext with an IllustratorDummyReporter for the Map phase. > Eventually when the code in MapReducePOStoreImpl line 121 tries to get the > reporter, it fails with the NPE. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira