[
https://issues.apache.org/jira/browse/PIG-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13965362#comment-13965362
]
Rohini Palaniswamy commented on PIG-2872:
-----------------------------------------
Can you open a separate jira for this? If it works with multiquery off /adding
a DUMP statement then the problem is with multiquery mode. multiquery off mode
stops parsing when a STORE is encountered and executes the script till then.
DUMP also essentially does the same executive the script till DUMP for both
multiquery on and off modes. So basically it fails with the multiquery mode.
> StoreFuncInterface.setStoreLocation get's a copy of a Configuration object
> --------------------------------------------------------------------------
>
> Key: PIG-2872
> URL: https://issues.apache.org/jira/browse/PIG-2872
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.11
> Environment: Pig trunk, Hadoop 0.20.205 with Kerberos, ElasticSearch
> trunk, Wonderdog trunk
> Reporter: Evert Lammerts
>
> When an implementation of StoreFuncInterface.setStoreLocation is called from
> JobControlCompiler.getJob, it is passed a copy of the Configuration that will
> be used for the Job that will be submitted:
> {code:title=JobControlCompiler.java}
> sFunc.setStoreLocation(st.getSFile().getFileName(), new
> org.apache.hadoop.mapreduce.Job(nwJob.getConfiguration()));
> {code}
> When a new org.apache.hadoop.mapreduce.Job is created it creates a copy of
> the Configuration object, as far as I know. Thus anything added to the
> Configuration object in the implementation of setStoreLocation will not be
> included in the Configuration of nwJob in JobControlCompiler.getJob.
> I notice this goes wrong in Wonderdog, which needs to include the
> Elasticsearch configuration file in the DistributedCache. It is added to
> mapred.cache.files through setStoreLocation, but this setting doesn't make it
> back into the Job returned by JobControlCompiler.getJob, and is therefore
> never localized.
> This might be intentional semantics within Pig, but I'm not familiar enough
> with StoreFuncs to know whether it is.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
