[ https://issues.apache.org/jira/browse/HADOOP-15726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16615493#comment-16615493 ]
Erik Krogen commented on HADOOP-15726: -------------------------------------- Thanks for taking a look Chen! # It can definitely be used for text-only info; you would just call log() with no arguments (no values specified). Then, you can use {{getCount()}} to indicate how many lines were throttled. I believe this addresses your parenthetical thought as well. Take a look at the [most recent patch|https://issues.apache.org/jira/secure/attachment/12938732/HDFS-13791-HDFS-12943.003.patch] in HDFS-13791 for examples of this. # I considered this as well... Especially while writing the Javadoc, I felt the confusion between "logging" to the throttler, and actually logging. I have a few ideas, though I'm not thrilled about any of them. Can you let me know your thoughts? (cc [~csun]): ** checkLog() - indicate that you are *checking* whether you should log rather than actually logging. This doesn't indicate to me that you are updating the values stored in the helper, though. ** store() - indicate that you store values. This may be too value-centric and not be accommodating to the text-only situations. ** update() - fairly generic, indicate that you are updating the state of the helper both in terms of values and whether or not logging should occur # The name is only necessary because the API which accepts a user-specified time value also accepts a logger name. I wanted to avoid too many method overloads with various argument combinations so I did one short version ({{log(double...)}}) and one long version which accepts both options (this has the additional benefit of avoiding confusion between {{log(long, double...)}} and {{log(double...)}}, since something intended as a long could also be interpreted as a double). If you look at other examples in HDFS-13791, most of the time, only the short version of the API is used, so there is none of this confusion. So far we have only one use case which requires multiple logs on a single helper, and that was a primary/dependent situation, so we made the dependent case easy. I see your reasoning that it may be less common, but with only one use case for multiple loggers so far, I think I would prefer to wait and see which ends up being more common? Let me know if this makes sense to you. > Create utility to limit frequency of log statements > --------------------------------------------------- > > Key: HADOOP-15726 > URL: https://issues.apache.org/jira/browse/HADOOP-15726 > Project: Hadoop Common > Issue Type: Improvement > Components: common, util > Reporter: Erik Krogen > Assignee: Erik Krogen > Priority: Major > Attachments: HADOOP-15726.000.patch, HADOOP-15726.001.patch > > > There is a common pattern of logging a behavior that is normally extraneous. > Under some circumstances, such a behavior becomes common, flooding the logs > and making it difficult to see what else is going on in the system. Under > such situations it is beneficial to limit how frequently the extraneous > behavior is logged, while capturing some summary information about the > suppressed log statements. > This is currently implemented in {{FSNamesystemLock}} (in HDFS-10713). We > have additional use cases for this in HDFS-13791, so this is a good time to > create a common utility for different sites to share this logic. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org