[
https://issues.apache.org/jira/browse/HADOOP-15726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16615480#comment-16615480
]
Chen Liang commented on HADOOP-15726:
-------------------------------------
Thanks [~xkrogen]!. Some comments:
1. Please correct me if I'm wrong. It looks like {{LogThrottlingHelper}} is
specifically for logging of numeric info? Because the {{log}} method always
takes {{double}} type {{values}}, backed by {{SummaryStatistics}}. I think it
is pretty cool to only log aggregated statistics (e.g. max, count) when there
are too many of the logs about numbers. But it may be helpful to make it clear
that this may not be the best fit for throttling/aggregating of text only
loggings, probably in naming/comments. (a thought: even for logs without a
numeric value to aggregate, it may still be useful to log how many lines of
them are omitted/throttled, I wonder would that a good/possible addition to
this).
2. Since {{log}} method does not really log, but only to tell caller whether a
log should be made, maybe rename it to something else?
3. I guess I can see use cases of primary and dependent loggers. But seems it
may also be an extra burden to the caller. For example in {{FSNamesystemLock}},
although the object is already named {{writeLockReportLogger}}, an explicitly
given string "write" still needs to be passed to it. If someone later wants to
add more logging with this logger, he/she better be sure not only using
{{writeLockReportLogger}}, but also calling with/without the static string
"write". I imagine having dependent loggers is a more rare use case, maybe a
more common case would be to just set the name when initializing, and always
using the only (primary) logger the rest of time. e.g. maybe add a constructor
with only (name, interval) and use this to initialize in {{FSNamesystemLock}}
instead.
> Create utility to limit frequency of log statements
> ---------------------------------------------------
>
> Key: HADOOP-15726
> URL: https://issues.apache.org/jira/browse/HADOOP-15726
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common, util
> Reporter: Erik Krogen
> Assignee: Erik Krogen
> Priority: Major
> Attachments: HADOOP-15726.000.patch, HADOOP-15726.001.patch
>
>
> There is a common pattern of logging a behavior that is normally extraneous.
> Under some circumstances, such a behavior becomes common, flooding the logs
> and making it difficult to see what else is going on in the system. Under
> such situations it is beneficial to limit how frequently the extraneous
> behavior is logged, while capturing some summary information about the
> suppressed log statements.
> This is currently implemented in {{FSNamesystemLock}} (in HDFS-10713). We
> have additional use cases for this in HDFS-13791, so this is a good time to
> create a common utility for different sites to share this logic.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
