[
https://issues.apache.org/jira/browse/PIG-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13963165#comment-13963165
]
William Watson commented on PIG-2872:
-------------------------------------
We're seeing this issue (from PIG-2578) in pig 0.12 with DBStorage. Is there a
new thread for a fix or is this the current one?
> StoreFuncInterface.setStoreLocation get's a copy of a Configuration object
> --------------------------------------------------------------------------
>
> Key: PIG-2872
> URL: https://issues.apache.org/jira/browse/PIG-2872
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.11
> Environment: Pig trunk, Hadoop 0.20.205 with Kerberos, ElasticSearch
> trunk, Wonderdog trunk
> Reporter: Evert Lammerts
>
> When an implementation of StoreFuncInterface.setStoreLocation is called from
> JobControlCompiler.getJob, it is passed a copy of the Configuration that will
> be used for the Job that will be submitted:
> {code:title=JobControlCompiler.java}
> sFunc.setStoreLocation(st.getSFile().getFileName(), new
> org.apache.hadoop.mapreduce.Job(nwJob.getConfiguration()));
> {code}
> When a new org.apache.hadoop.mapreduce.Job is created it creates a copy of
> the Configuration object, as far as I know. Thus anything added to the
> Configuration object in the implementation of setStoreLocation will not be
> included in the Configuration of nwJob in JobControlCompiler.getJob.
> I notice this goes wrong in Wonderdog, which needs to include the
> Elasticsearch configuration file in the DistributedCache. It is added to
> mapred.cache.files through setStoreLocation, but this setting doesn't make it
> back into the Job returned by JobControlCompiler.getJob, and is therefore
> never localized.
> This might be intentional semantics within Pig, but I'm not familiar enough
> with StoreFuncs to know whether it is.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
