[
https://issues.apache.org/jira/browse/PIG-1727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12932550#action_12932550
]
Richard Ding commented on PIG-1727:
-----------------------------------
Thanks for the comment. The purpose of using GenericOptionsParser is to parse
command line arguments generic to the Hadoop framework so that users can
override system properties (including pig properties using -D option) from the
command line. This is the reason for the current load ordering. As Daniel
pointed out, Pig doesn't need to load the hadoop-default properties at this
point. The patch now only add the command line arguments to the properties.
> Hadoop default config override pig.properties
> ---------------------------------------------
>
> Key: PIG-1727
> URL: https://issues.apache.org/jira/browse/PIG-1727
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.8.0
> Reporter: Daniel Dai
> Assignee: Richard Ding
> Fix For: 0.8.0
>
> Attachments: PIG-1727.patch
>
>
> In GenericOptionsParser work, we have these lines:
> {code}
> GenericOptionsParser parser = new GenericOptionsParser(args);
> Configuration conf = parser.getConfiguration();
> Properties properties = new Properties();
> properties.putAll(ConfigurationUtil.toProperties(conf));
> PropertiesUtil.loadDefaultProperties(properties);
> {code}
> conf contains two sources: one from hadoop default config files
> (core-site.xml, hdfs-site.xml, mapred-site.xml), the other is from hadoop
> related command line options. Override conf over pig property files is wrong
> for the default hadoop configuration part. We shall not bring default hadoop
> configuration in. The following code illustrate how to do that:
> {code}
> Configuration conf = new Configuration(false);
> GenericOptionsParser parser = new GenericOptionsParser(conf, args);
> conf = parser.getConfiguration();
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
