[
https://issues.apache.org/jira/browse/PIG-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768352#action_12768352
]
Ashutosh Chauhan commented on PIG-1012:
---------------------------------------
Marking "log" in POFRJoin transient causes FRJoin to fail. Because at the
backend it can't be deserialized and log.debug is used while building
hashtables resulting in NPE. Either it shouldn't be marked transient or it
should be instantiated in "readObject()" method.
Stack Trace:
Pig Stack Trace
---------------
ERROR 2999: Unexpected internal error. null
java.lang.NullPointerException
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFRJoin.setUpHashMap(POFRJoin.java:293)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFRJoin.getNext(POFRJoin.java:197)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:249)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:240)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.map(PigMapOnly.java:65)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
Thanks to Tejal to pointing this out.
> FINDBUGS: SE_BAD_FIELD: Non-transient non-serializable instance field in
> serializable class
> -------------------------------------------------------------------------------------------
>
> Key: PIG-1012
> URL: https://issues.apache.org/jira/browse/PIG-1012
> Project: Pig
> Issue Type: Bug
> Reporter: Olga Natkovich
> Attachments: PIG-1012.patch
>
>
> Se Class org.apache.pig.backend.executionengine.PigSlice defines
> non-transient non-serializable instance field is
> Se Class org.apache.pig.backend.executionengine.PigSlice defines
> non-transient non-serializable instance field loader
> Se java.util.zip.GZIPInputStream stored into non-transient field
> PigSlice.is
> Se org.apache.pig.backend.datastorage.SeekableInputStream stored into
> non-transient field PigSlice.is
> Se org.apache.tools.bzip2r.CBZip2InputStream stored into non-transient
> field PigSlice.is
> Se org.apache.pig.builtin.PigStorage stored into non-transient field
> PigSlice.loader
> Se org.apache.pig.backend.hadoop.DoubleWritable$Comparator implements
> Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigBagWritableComparator
> implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigCharArrayWritableComparator
> implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigDBAWritableComparator
> implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigDoubleWritableComparator
> implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigFloatWritableComparator
> implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigIntWritableComparator
> implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigLongWritableComparator
> implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigTupleWritableComparator
> implements Comparator but not Serializable
> Se
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler$PigWritableComparator
> implements Comparator but not Serializable
> Se Class
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper
> defines non-transient non-serializable instance field nig
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.EqualToExpr
> defines non-transient non-serializable instance field log
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GreaterThanExpr
> defines non-transient non-serializable instance field log
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GTOrEqualToExpr
> defines non-transient non-serializable instance field log
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.LessThanExpr
> defines non-transient non-serializable instance field log
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.LTOrEqualToExpr
> defines non-transient non-serializable instance field log
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.NotEqualToExpr
> defines non-transient non-serializable instance field log
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast
> defines non-transient non-serializable instance field log
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject
> defines non-transient non-serializable instance field bagIterator
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserComparisonFunc
> defines non-transient non-serializable instance field log
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc
> defines non-transient non-serializable instance field log
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POCombinerPackage
> defines non-transient non-serializable instance field log
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.PODemux
> defines non-transient non-serializable instance field log
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.PODistinct
> defines non-transient non-serializable instance field log
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach
> defines non-transient non-serializable instance field its
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach
> defines non-transient non-serializable instance field log
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFRJoin
> defines non-transient non-serializable instance field log
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLimit
> defines non-transient non-serializable instance field log
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad
> defines non-transient non-serializable instance field is
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad
> defines non-transient non-serializable instance field loader
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad
> defines non-transient non-serializable instance field log
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange
> defines non-transient non-serializable instance field log
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POMultiQueryPackage
> defines non-transient non-serializable instance field log
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POMultiQueryPackage
> defines non-transient non-serializable instance field myKey
>
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POPackage
> defines non-transient non-serializable instance field log
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POPackage
> defines non-transient non-serializable instance field tupIter
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POSort
> defines non-transient non-serializable instance field log
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POSort
> defines non-transient non-serializable instance field mComparator
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POSplit
> defines non-transient non-serializable instance field log
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore
> defines non-transient non-serializable instance field impl
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore
> defines non-transient non-serializable instance field log
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore
> defines non-transient non-serializable instance field storer
> Se Class
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStream
> defines non-transient non-serializable instance field executableManager
> Se Class
> org.apache.pig.backend.local.executionengine.physicalLayer.relationalOperators.POCogroup
> defines non-transient non-serializable instance field its
> Se
> org.apache.pig.backend.local.executionengine.physicalLayer.relationalOperators.POCogroup$groupComparator
> implements Comparator but not Serializable
> Se Class
> org.apache.pig.backend.local.executionengine.physicalLayer.relationalOperators.POCross
> defines non-transient non-serializable instance field its
> Se Class
> org.apache.pig.backend.local.executionengine.physicalLayer.relationalOperators.POSplitOutput
> defines non-transient non-serializable instance field it
> Se Class org.apache.pig.data.InternalCachedBag defines non-transient
> non-serializable instance field factory
> Se Class org.apache.pig.data.InternalCachedBag defines non-transient
> non-serializable instance field out
> Se Class org.apache.pig.data.SortedDataBag defines non-transient
> non-serializable instance field mComp
> Se org.apache.pig.data.SortedDataBag$DefaultComparator implements
> Comparator but not Serializable
> Se The field org.apache.pig.impl.logicalLayer.LOLoad.mLoadFunc is
> transient but isn't set by deserialization
> Se The field org.apache.pig.impl.logicalLayer.LOStore.mStoreFunc is
> transient but isn't set by deserialization
> Se Class org.apache.pig.impl.logicalLayer.LOStream defines non-transient
> non-serializable instance field executableManager
> Se The field org.apache.pig.impl.PigContext.extraJars is transient but
> isn't set by deserialization
> Se The field org.apache.pig.impl.PigContext.skipJars is transient but
> isn't set by deserialization
> Se The field org.apache.pig.impl.PigContext.log is transient but isn't set
> by deserialization
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
