Stamatis Zampetakis created HADOOP-19597:
--------------------------------------------
Summary: Log warning message on every set/get of a deprecated
configuration property
Key: HADOOP-19597
URL: https://issues.apache.org/jira/browse/HADOOP-19597
Project: Hadoop Common
Issue Type: Improvement
Components: hadoop-common
Reporter: Stamatis Zampetakis
Assignee: Stamatis Zampetakis
Currently, the warning message is logged once (or at most twice after
HADOOP-8865) when we first use (set/get) a deprecated configuration key and
most of the time this happens early on during system startup.
Users tend to set/get properties on their job scripts/applications that keep
running on a hourly/daily/etc basis. When a problem comes up the user will
package the latest logs and send them to us (developers/support) for further
analysis and troubleshooting. However, it's very likely that these logs will
not contain information about the deprecated usages which might be crucial for
advancing the investigation.
On the other end, a warning that appears just once is not that worrisome so
even if the users/customers/developers see it, they can easily ignore it, and
move on, thinking that there is no action needed on their end.
The above scenarios are based on applications such as the Hivemetastore,
HiveServer2, which use the Hadoop Configuration, and usually run for
weeks/months without a restart.
I propose to change the existing logic to always log a message when a
deprecated configuration key is in use. This will minimize the risk of losing
important deprecation logs and will also simplify the implementation.
Moreover, since there is a dedicated logger (HADOOP-8197) for these warning
messages applications/users can suppress/limit the log content by changing
their logging configuration. It may not be as precise as logging each
deprecation once but it can still address the verbosity concern that was raised
in the past.
+History+
In HADOOP-6105, where the logging was first introduced the initial intention
was to log on every usage as proposed here. Just before merging there were some
concerns that this may clutter the logs and a more conservative approach was
adopted:
bq. The warning message will be printed for every set and also for first get of
deprecated key after every reload of configuration.
Then HADOOP-8197 came in, claiming that it is a bug to log warning on every
user and restricted the logging even more.
Even, with HADOOP-8197 in place, some users argued that there are use-cases
where it makes sense to completely suppress these warnings thus HADOOP-9487
introduced a dedicated logger just for the warning messages.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
