[
https://issues.apache.org/jira/browse/PIG-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115582#comment-16115582
]
Rohini Palaniswamy commented on PIG-5283:
-----------------------------------------
bq. HADOOP-8588 was committed in Hadoop 3, supposed to fix an NPE, but
actually causes an other one, if the required key is not set in Configuration
You should comment on the hadoop jira and have them fix it properly instead
of working around in Pig.
> Configuration is not passed to SparkPigSplits on the backend
> ------------------------------------------------------------
>
> Key: PIG-5283
> URL: https://issues.apache.org/jira/browse/PIG-5283
> Project: Pig
> Issue Type: Bug
> Components: spark
> Reporter: Adam Szita
> Assignee: Adam Szita
> Attachments: PIG-5283.0.patch
>
>
> When a Hadoop ObjectWritable is created during a Spark job, the instantiated
> PigSplit (wrapped into a SparkPigSplit) is given an empty Configuration
> instance.
> This happens
> [here|https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SerializableWritable.scala#L44]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
