[
https://issues.apache.org/jira/browse/HADOOP-2866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12571593#action_12571593
]
Doug Cutting commented on HADOOP-2866:
--------------------------------------
We should not draw a hard-line between "system" parameters which would be
well-checked and "user" parameters, where all bets are off. Hadoop is included
as a component in larger systems, so any boundary must be extensible. So if
there's a namespace boundary, one should be able to move it to include new
namespaces, not hard-code it. And if hadoop-default,xml defines a boundary,
then we'd need to permit folks who layer systems to somehow add things to the
defaults, e.g., by resurrecting Configuration#addDefaultResource().
Renaming everything would break lots of folks, and I don't see that namespace
boundaries are required. We can cover cases where things are set by code by
adding accessor methods, and we can cover cases where things are set in files
by checking against the (extensible) defaults. That covers all the cases w/o
breaking every existing config file.
So my recipe would be:
- fix all code to use accessor methods and constants, never literal property
names
- enforce this in Hudson
- resurrect Configuration#addDefaultResource()
- when loading non-default config files, warn unless overriding a default.
Adding accessors is a big job, but it's long overdue & entirely back-compatible.
> JobConf should validate key names in well-defined namespaces and warn on
> misspelling
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-2866
> URL: https://issues.apache.org/jira/browse/HADOOP-2866
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.16.0
> Reporter: Aaron Kimball
> Priority: Minor
> Original Estimate: 72h
> Remaining Estimate: 72h
>
> A discussion on the mailing list reveals that some configuration strings in
> the JobConf are deprecated over time and new configuration names replace them:
> e.g., "mapred.output.compression.type" is now replaced with
> "mapred.map.output.compression.type"
> Programmers who have been manually specifying the former string, however,
> receive no diagnostic output during testing to suggest that their compression
> type is being silently ignored.
> It would be desirable to notify developers of this change by printing a
> warning message when deprecated configuration names are used in a newer
> version of Hadoop. More generally, when any configuration string in the
> mapred.\*, fs.\*, dfs.\*, etc namespaces are provided by a user and are not
> recognized by Hadoop, it is desirable to print a warning, to indicate
> malformed configurations. No warnings should be printed when configuration
> keys are in user-defined namespaces (e.g., "myprogram.mytask.myvalue").
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
