[ 
https://issues.apache.org/jira/browse/PIG-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13433199#comment-13433199
 ] 

Bill Graham commented on PIG-2872:
----------------------------------

I ran into issues with this as well. See PIG-2870 and the attached patch. Does 
Wonderdog work if you apply this patch?
                
> StoreFuncInterface.setStoreLocation get's a copy of a Configuration object
> --------------------------------------------------------------------------
>
>                 Key: PIG-2872
>                 URL: https://issues.apache.org/jira/browse/PIG-2872
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.11
>         Environment: Pig trunk, Hadoop 0.20.205 with Kerberos, ElasticSearch 
> trunk, Wonderdog trunk
>            Reporter: Evert Lammerts
>
> When an implementation of StoreFuncInterface.setStoreLocation is called from 
> JobControlCompiler.getJob, it is passed a copy of the Configuration that will 
> be used for the Job that will be submitted:
> {code:title=JobControlCompiler.java}
> sFunc.setStoreLocation(st.getSFile().getFileName(), new 
> org.apache.hadoop.mapreduce.Job(nwJob.getConfiguration()));
> {code}
> When a new org.apache.hadoop.mapreduce.Job is created it creates a copy of 
> the Configuration object, as far as I know. Thus anything added to the 
> Configuration object in the implementation of setStoreLocation will not be 
> included in the Configuration of nwJob in JobControlCompiler.getJob.
> I notice this goes wrong in Wonderdog, which needs to include the 
> Elasticsearch configuration file in the DistributedCache. It is added to 
> mapred.cache.files through setStoreLocation, but this setting doesn't make it 
> back into the Job returned by JobControlCompiler.getJob, and is therefore 
> never localized.
> This might be intentional semantics within Pig, but I'm not familiar enough 
> with StoreFuncs to know whether it is.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Reply via email to