jay vyas created MAPREDUCE-5894:
-----------------------------------
Summary: Make critical YARN properties first class citizens in the
build.
Key: MAPREDUCE-5894
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5894
Project: Hadoop Map/Reduce
Issue Type: Improvement
Reporter: jay vyas
We recently found that when deploy hadoop 2.2 with hadoop 2.0 values
{noformat} mapreduce_shuffle {noformat} changed to {noformat}
mapreduce.shuffle {noformat} .
There are likewise many similar examples of parameters which become deprecated
over time. See
http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-common/DeprecatedProperties.html
I suggest we:
1) Use the *set of parameters which are deprecated* over time into java class
which ships directly with the code, maybe even as a static list inside of
Configuration() itself, with *optional extended parameters read from a
configurable parameter *, so that ecosystem users (i.e. like Hbase, or
alternative file systems) can add their own deprecation info.
2) have this list *checked on yarn daemon startup*. so that unused parameters
which are *obviously artifacts are flagged immediately* by the daemon failing
immediately.
3)Have a list of all mandatory *current* parameters stored in the code, and
also, a list of deprecated ones. Then, have the build * automatically fail * a
parameter in the "madatory" list is NOT accessed. this would (a) make it so
that unit testing of parameters does not regress and (b) force all updates to
the code which change a parameter name, to also include update to deprecated
parameter list, before build passes.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
