Hadoop default config override pig.properties
---------------------------------------------
Key: PIG-1727
URL: https://issues.apache.org/jira/browse/PIG-1727
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.8.0
Reporter: Daniel Dai
Assignee: Richard Ding
Fix For: 0.8.0
In GenericOptionsParser work, we have these lines:
{code}
GenericOptionsParser parser = new GenericOptionsParser(args);
Configuration conf = parser.getConfiguration();
Properties properties = new Properties();
properties.putAll(ConfigurationUtil.toProperties(conf));
PropertiesUtil.loadDefaultProperties(properties);
{code}
conf contains two sources: one from hadoop default config files (core-site.xml,
hdfs-site.xml, mapred-site.xml), the other is from hadoop related command line
options. Override conf over pig property files is wrong for the default hadoop
configuration part. We shall not bring default hadoop configuration in. The
following code illustrate how to do that:
{code}
Configuration conf = new Configuration(false);
GenericOptionsParser parser = new GenericOptionsParser(conf, args);
conf = parser.getConfiguration();
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
