[
https://issues.apache.org/jira/browse/HDFS-6982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14213177#comment-14213177
]
Andrew Wang commented on HDFS-6982:
-----------------------------------
Hi Maysam, took a look at the latest patch, I think we're almost there :) Just
minor comments. Hopefully Jenkins behaves with the next rev too, I agree it
looks unrelated or garbled.
DFSConfigKeys / TopConf:
* Need to rename the DFSConfigKeys variable names to reflect new config names
* Seems like I gave bad advice about getInts, since it doesn't have a way of
taking a default, so right now if we try to turn it off, it'll set the default.
Reverting to what you had is cool, though adding a getInts that takes a default
would be appreciated.
RWManager:
* Could we add explanatory text to the Precondition checks?
AuditLogger:
* Rather than injecting it into the conf (kinda brittle), what I had in mind
was in FSNamesystem#initAuditLoggers, we could tack it on the end after adding
the ones from the conf. No need for reflection :)
* Related to this, it'd be good to have a unit test that disables nntop and
then checks that the audit logger isn't added and that metrics aren't
published. Feel free to add a @VisibleForTesting getter if it helps.
Nits:
* Unused import in NameNode
This is just minor stuff though, I'm +1 pending the above review comments.
> nntop: top-like tool for name node users
> -----------------------------------------
>
> Key: HDFS-6982
> URL: https://issues.apache.org/jira/browse/HDFS-6982
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Maysam Yabandeh
> Assignee: Maysam Yabandeh
> Attachments: HDFS-6982.patch, HDFS-6982.v2.patch, HDFS-6982.v3.patch,
> HDFS-6982.v4.patch, HDFS-6982.v5.patch, HDFS-6982.v6.patch,
> HDFS-6982.v7.patch, HDFS-6982.v8.patch, nntop-design-v1.pdf
>
>
> In this jira we motivate the need for nntop, a tool that, similarly to what
> top does in Linux, gives the list of top users of the HDFS name node and
> gives insight about which users are sending majority of each traffic type to
> the name node. This information turns out to be the most critical when the
> name node is under pressure and the HDFS admin needs to know which user is
> hammering the name node and with what kind of requests. Here we present the
> design of nntop which has been in production at Twitter in the past 10
> months. nntop proved to have low cpu overhead (< 2% in a cluster of 4K
> nodes), low memory footprint (less than a few MB), and quite efficient for
> the write path (only two hash lookup for updating a metric).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
