Pankit Thapar created HIVE-7201:
-----------------------------------
Summary: Fix TestHiveConf#testConfProperties test case
Key: HIVE-7201
URL: https://issues.apache.org/jira/browse/HIVE-7201
Project: Hive
Issue Type: Bug
Components: Tests
Affects Versions: 0.13.0
Reporter: Pankit Thapar
Priority: Minor
CHANGE 1:
TEST CASE :
The intention of TestHiveConf#testConfProperties() is to test the HiveConf
properties being set in the priority as expected.
Each HiveConf object is initialized as follows:
1) Hadoop configuration properties are applied.
2) ConfVar properties with non-null values are overlayed.
3) hive-site.xml properties are overlayed.
ISSUE :
The mapreduce related configurations are loaded by JobConf and not
Configuration.
The current test tries to get the configuration properties like :
HADOOPNUMREDUCERS ("mapred.job.reduces")
from Configuration class. But these mapreduce related properties are loaded by
JobConf class from mapred-default.xml.
DETAILS :
LINE 63 : checkHadoopConf(ConfVars.HADOOPNUMREDUCERS.varname, "1"); -->fails
Because,
private void checkHadoopConf(String name, String expectedHadoopVal) {
Assert.assertEquals(expectedHadoopVal, new Configuration().get(name));
----> Second parameter is null, since its the JobConf class and not the
Configuration class that initializes mapred-default values.
}
Code that loads mapreduce resources is in ConfigUtil and JobConf makes a call
like this (in static block):
public class JobConf extends Configuration {
private static final Log LOG = LogFactory.getLog(JobConf.class);
static{
ConfigUtil.loadResources(); --> loads mapreduce related resources
(mapreduce-default.xml)
}
.....
}
Please note, the test case assertion works fine if HiveConf() constructor is
called before this assertion since, HiveConf() triggers JobConf()
which basically sets the default values of the properties pertaining to
mapreduce.
This is why, there won't be any failures if testHiveSitePath() was run before
testConfProperties() as that would load mapreduce
properties into config properties.
FIX:
Instead of using a Configuration object, we can use the JobConf object to get
the default values used by hadoop/mapreduce.
CHANGE 2:
In TestHiveConf#testHiveSitePath(), a call to static method
getHiveSiteLocation() should be called statically instead of using an object.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
