[
https://issues.apache.org/jira/browse/PIG-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13433199#comment-13433199
]
Bill Graham commented on PIG-2872:
----------------------------------
I ran into issues with this as well. See PIG-2870 and the attached patch. Does
Wonderdog work if you apply this patch?
> StoreFuncInterface.setStoreLocation get's a copy of a Configuration object
> --------------------------------------------------------------------------
>
> Key: PIG-2872
> URL: https://issues.apache.org/jira/browse/PIG-2872
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.11
> Environment: Pig trunk, Hadoop 0.20.205 with Kerberos, ElasticSearch
> trunk, Wonderdog trunk
> Reporter: Evert Lammerts
>
> When an implementation of StoreFuncInterface.setStoreLocation is called from
> JobControlCompiler.getJob, it is passed a copy of the Configuration that will
> be used for the Job that will be submitted:
> {code:title=JobControlCompiler.java}
> sFunc.setStoreLocation(st.getSFile().getFileName(), new
> org.apache.hadoop.mapreduce.Job(nwJob.getConfiguration()));
> {code}
> When a new org.apache.hadoop.mapreduce.Job is created it creates a copy of
> the Configuration object, as far as I know. Thus anything added to the
> Configuration object in the implementation of setStoreLocation will not be
> included in the Configuration of nwJob in JobControlCompiler.getJob.
> I notice this goes wrong in Wonderdog, which needs to include the
> Elasticsearch configuration file in the DistributedCache. It is added to
> mapred.cache.files through setStoreLocation, but this setting doesn't make it
> back into the Job returned by JobControlCompiler.getJob, and is therefore
> never localized.
> This might be intentional semantics within Pig, but I'm not familiar enough
> with StoreFuncs to know whether it is.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
