[
https://issues.apache.org/jira/browse/HDFS-6982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208866#comment-14208866
]
Maysam Yabandeh commented on HDFS-6982:
---------------------------------------
Thanks for well-detailed review. Some questions before I submit the new patch:
bq. Do you mind writing out the key strings in DFSConfigKeys? It's how we do it
in the rest of the file, and more readable.
The configuration keys are already there. I have some constants in TopConf
though. Did you mean to move those top-specific constants to DFSConfigKeys?
bq. The jmx page right now only exposes the shortest window, I see the
smallestWindow hardcode. Since the goal is to use jmx to populate the HTML
view, we need to somehow expose all of the configured windows.
We actually use jmx for plotting the data exported by nntop, and that is why
one reporting period is sufficient. For html view, the web component directly
contacts the TopMetrics and retrieves the top users for all reporting periods.
Also about the counters never go back to 0 and reported users never being
removed the plan was to submit a separate patch for it as the fix was kind of
orthogonal to what nntop does: refer to #3 in this comment:
https://issues.apache.org/jira/browse/HDFS-6982?focusedCommentId=14122097&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14122097
If you think that change should be made before this patch gets committed, I can
open the jira for that change and we have it committed before.
> nntop: top-like tool for name node users
> -----------------------------------------
>
> Key: HDFS-6982
> URL: https://issues.apache.org/jira/browse/HDFS-6982
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Maysam Yabandeh
> Assignee: Maysam Yabandeh
> Attachments: HDFS-6982.patch, HDFS-6982.v2.patch, HDFS-6982.v3.patch,
> HDFS-6982.v4.patch, HDFS-6982.v5.patch, HDFS-6982.v6.patch,
> nntop-design-v1.pdf
>
>
> In this jira we motivate the need for nntop, a tool that, similarly to what
> top does in Linux, gives the list of top users of the HDFS name node and
> gives insight about which users are sending majority of each traffic type to
> the name node. This information turns out to be the most critical when the
> name node is under pressure and the HDFS admin needs to know which user is
> hammering the name node and with what kind of requests. Here we present the
> design of nntop which has been in production at Twitter in the past 10
> months. nntop proved to have low cpu overhead (< 2% in a cluster of 4K
> nodes), low memory footprint (less than a few MB), and quite efficient for
> the write path (only two hash lookup for updating a metric).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
