[
https://issues.apache.org/jira/browse/PIG-1727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12932322#action_12932322
]
V.V.Chaitanya Krishna commented on PIG-1727:
--------------------------------------------
Richard,
I see that this has more to do with the order in which properties are loaded.
If we make sure that hadoop-specific default properties are loaded first,
followed by the pig-specific ones, this problem should not arise. Looking at
the following code snippet in Main.java :
{noformat}
PropertiesUtil.loadDefaultProperties(properties);
properties.putAll(ConfigurationUtil.toProperties(conf));
{noformat}
The order of loading the properties object in this case seems to be such that
pig-specific default properties (i.e., properties in pig-default.properties,
.pigrc and pig.properties) get loaded first, followed by the ones in
hadoop-default configuration. Due to this order, some of the pig properties
might get overriden. Therefore, a simple swap of the above two lines to change
the order of loading of properties should solve the problem.
Thoughts?
> Hadoop default config override pig.properties
> ---------------------------------------------
>
> Key: PIG-1727
> URL: https://issues.apache.org/jira/browse/PIG-1727
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.8.0
> Reporter: Daniel Dai
> Assignee: Richard Ding
> Fix For: 0.8.0
>
> Attachments: PIG-1727.patch
>
>
> In GenericOptionsParser work, we have these lines:
> {code}
> GenericOptionsParser parser = new GenericOptionsParser(args);
> Configuration conf = parser.getConfiguration();
> Properties properties = new Properties();
> properties.putAll(ConfigurationUtil.toProperties(conf));
> PropertiesUtil.loadDefaultProperties(properties);
> {code}
> conf contains two sources: one from hadoop default config files
> (core-site.xml, hdfs-site.xml, mapred-site.xml), the other is from hadoop
> related command line options. Override conf over pig property files is wrong
> for the default hadoop configuration part. We shall not bring default hadoop
> configuration in. The following code illustrate how to do that:
> {code}
> Configuration conf = new Configuration(false);
> GenericOptionsParser parser = new GenericOptionsParser(conf, args);
> conf = parser.getConfiguration();
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
