[
https://issues.apache.org/jira/browse/HADOOP-15708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16626331#comment-16626331
]
Zoltan Siegl commented on HADOOP-15708:
---------------------------------------
[~jlowe] [~daryn] could someone have a look at this issue please?
> Reading values from Configuration before adding deprecations make it
> impossible to read value with deprecated key
> -----------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-15708
> URL: https://issues.apache.org/jira/browse/HADOOP-15708
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 3.2.0
> Reporter: Szilard Nemeth
> Assignee: Zoltan Siegl
> Priority: Minor
> Attachments: HADOOP-15708-testcase.patch, HADOOP-15708.001.patch,
> HADOOP-15708.002.patch, HADOOP-15708.003.patch
>
>
> Hadoop Common contains a widely used Configuration class.
> This class can handle deprecations of properties, e.g. if property 'A' gets
> deprecated with an alternative property key 'B', users can access property
> values with keys 'A' and 'B'.
> Unfortunately, this does not work in one case.
> When a config file is specified (for instance, XML) and a property is read
> with the config.get() method, the config is loaded from the file at this
> time.
> If the deprecation mapping is not yet specified by the time any config value
> is retrieved and the XML config refers to a deprecated key, then the
> deprecation mapping specified, the config value cannot be retrieved neither
> with the deprecated nor with the new key.
> The attached patch contains a testcase that reproduces this wrong behavior.
> Here are the steps outlined what the testcase does:
> 1. Creates an XML config file with a deprecated property
> 2. Adds the config to the Configuration object
> 3. Retrieves the config with its deprecated key (it does not really matter
> which property the user gets, could be any)
> 4. Specifies the deprecation rules including the one defined in the config
> 5. Prints and asserts the property retrieved from the config with both the
> deprecated and the new property keys.
> For reference, here is the log of one execution that actually shows what the
> issue is:
> {noformat}
> Loaded items: 1
> Looked up property value with name hadoop.zk.address: null
> Looked up property value with name yarn.resourcemanager.zk-address:
> dummyZkAddress
> Contents of config file: [<?xml version="1.0"?>, <configuration>,
> <property><name>yarn.resourcemanager.zk-address</name><value>dummyZkAddress</value></property>,
> </configuration>]
> Looked up property value with name hadoop.zk.address: null
> 2018-08-31 10:10:06,484 INFO Configuration.deprecation
> (Configuration.java:logDeprecation(1397)) - yarn.resourcemanager.zk-address
> is deprecated. Instead, use hadoop.zk.address
> Looked up property value with name hadoop.zk.address: null
> Looked up property value with name hadoop.zk.address: null
> java.lang.AssertionError:
> Expected :dummyZkAddress
> Actual :null
> {noformat}
> *As it's visible from the output and the code, the issue is really that if
> the config is retrieved either with the deprecated or the new value,
> Configuration both wants to serve the value with the new key.*
> *If the mapping is not specified before any retrieval happened, the value is
> only stored under the deprecated key but not the new key.*
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
