[
https://issues.apache.org/jira/browse/PIG-2152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13061891#comment-13061891
]
Vivek Padmanabhan commented on PIG-2152:
----------------------------------------
Even though not reproducible , the exception is happening randomly and quite
frequently.
>From most of the failed jobs it looks like this is happening towards end of
>the task execution. Till now, it is seen only in Map Tasks.
The below is one NullPointer from progess() with a different call heirachy;
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.ProgressableReporter.progress(ProgressableReporter.java:37)
at
org.apache.pig.data.DefaultAbstractBag.reportProgress(DefaultAbstractBag.java:369)
at
org.apache.pig.data.DefaultDataBag$DefaultDataBagIterator.next(DefaultDataBag.java:165)
at
org.apache.pig.data.DefaultDataBag$DefaultDataBagIterator.hasNext(DefaultDataBag.java:157)
at org.apache.pig.data.BinInterSedes.writeBag(BinInterSedes.java:522)
at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:361)
at org.apache.pig.data.BinInterSedes.writeTuple(BinInterSedes.java:542)
at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:357)
at org.apache.pig.data.BinInterSedes.writeTuple(BinInterSedes.java:542)
at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:357)
at org.apache.pig.data.BinSedesTuple.write(BinSedesTuple.java:57)
at
org.apache.pig.impl.io.PigNullableWritable.write(PigNullableWritable.java:123)
at
org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:90)
at
org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:77)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1069)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:124)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:263)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:256)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:58)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:261)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.mapred.Child.main(Child.java:255)
> Null pointer exception while reporting progress
> -----------------------------------------------
>
> Key: PIG-2152
> URL: https://issues.apache.org/jira/browse/PIG-2152
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Olga Natkovich
> Fix For: 0.9.0
>
> Attachments: null_pointer_traces (copy)
>
>
> We have observed the following issues with code built from Pig 0.9 branch. We
> have not seen this with earlier versions; however, since this happens once in
> a while and is not reproducible at will it is not clear whether the issue is
> specific to 0.9 or not.
> Here is the stack:
> java.lang.NullPointerException at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.ProgressableReporter.progress(ProgressableReporter.java:37)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:399)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:284)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:290)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:256)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:261)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:256)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:58)
> at
> org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) at
> org.apache.hadoop.mapred.Child$4.run(Child.java:261) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:396) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> at
> org.apache.hadoop.mapred.Child.main(Child.java:255)
> Note that the code in progress function looks as follows:
> public void progress() {
> if(rep!=null)
> rep.progress();
> }
> This points to some sort of synchronization issue
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
