[jira] [Updated] (HDFS-3647) Backport HDFS-2868 (Add number of active transfer threads to the DataNode status) to branch-1
[ https://issues.apache.org/jira/browse/HDFS-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3647: -- Component/s: (was: performance) > Backport HDFS-2868 (Add number of active transfer threads to the DataNode > status) to branch-1 > - > > Key: HDFS-3647 > URL: https://issues.apache.org/jira/browse/HDFS-3647 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.20.2 >Reporter: Steve Hoffman >Assignee: Harsh J > Fix For: 1.2.0 > > Attachments: HDFS-3647.patch, Screen Shot 2012-07-14 at 12.41.07 > AM.png > > > Not sure if this is in a newer version of Hadoop, but in CDH3u3 it isn't > there. > There is a lot of mystery surrounding how large to set > dfs.datanode.max.xcievers. Most people say to just up it to 4096, but given > that exceeding this will cause an HBase RegionServer shutdown (see Lars' blog > post here: http://www.larsgeorge.com/2012/03/hadoop-hbase-and-xceivers.html), > it would be nice if we could expose the current count via the built-in > metrics framework (most likely under dfs). In this way we could watch it to > see if we have it set too high, too low, time to bump it up, etc. > Thoughts? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3647) Backport HDFS-2868 (Add number of active transfer threads to the DataNode status) to branch-1
[
https://issues.apache.org/jira/browse/HDFS-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated HDFS-3647:
--
Attachment: Screen Shot 2012-07-14 at 12.41.07 AM.png
Screenshot of jconsole mxbean browser showing the metric, after application to
branch-1 and some few {{fs -cat}} commands.
> Backport HDFS-2868 (Add number of active transfer threads to the DataNode
> status) to branch-1
> -
>
> Key: HDFS-3647
> URL: https://issues.apache.org/jira/browse/HDFS-3647
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: data-node, performance
>Affects Versions: 0.20.2
>Reporter: Steve Hoffman
>Assignee: Harsh J
> Attachments: HDFS-3647.patch, Screen Shot 2012-07-14 at 12.41.07
> AM.png
>
>
> Not sure if this is in a newer version of Hadoop, but in CDH3u3 it isn't
> there.
> There is a lot of mystery surrounding how large to set
> dfs.datanode.max.xcievers. Most people say to just up it to 4096, but given
> that exceeding this will cause an HBase RegionServer shutdown (see Lars' blog
> post here: http://www.larsgeorge.com/2012/03/hadoop-hbase-and-xceivers.html),
> it would be nice if we could expose the current count via the built-in
> metrics framework (most likely under dfs). In this way we could watch it to
> see if we have it set too high, too low, time to bump it up, etc.
> Thoughts?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3647) Backport HDFS-2868 (Add number of active transfer threads to the DataNode status) to branch-1
[ https://issues.apache.org/jira/browse/HDFS-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated HDFS-3647: -- Attachment: HDFS-3647.patch Straight forward backport with fixed paths and conflicts. Test-patch results will arrive shortly. > Backport HDFS-2868 (Add number of active transfer threads to the DataNode > status) to branch-1 > - > > Key: HDFS-3647 > URL: https://issues.apache.org/jira/browse/HDFS-3647 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node, performance >Affects Versions: 0.20.2 >Reporter: Steve Hoffman >Assignee: Harsh J > Attachments: HDFS-3647.patch > > > Not sure if this is in a newer version of Hadoop, but in CDH3u3 it isn't > there. > There is a lot of mystery surrounding how large to set > dfs.datanode.max.xcievers. Most people say to just up it to 4096, but given > that exceeding this will cause an HBase RegionServer shutdown (see Lars' blog > post here: http://www.larsgeorge.com/2012/03/hadoop-hbase-and-xceivers.html), > it would be nice if we could expose the current count via the built-in > metrics framework (most likely under dfs). In this way we could watch it to > see if we have it set too high, too low, time to bump it up, etc. > Thoughts? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3647) Backport HDFS-2868 (Add number of active transfer threads to the DataNode status) to branch-1
[ https://issues.apache.org/jira/browse/HDFS-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-3647: -- Target Version/s: 1.2.0 Summary: Backport HDFS-2868 (Add number of active transfer threads to the DataNode status) to branch-1 (was: Expose current xcievers count as metric) > Backport HDFS-2868 (Add number of active transfer threads to the DataNode > status) to branch-1 > - > > Key: HDFS-3647 > URL: https://issues.apache.org/jira/browse/HDFS-3647 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node, performance >Affects Versions: 0.20.2 >Reporter: Steve Hoffman > > Not sure if this is in a newer version of Hadoop, but in CDH3u3 it isn't > there. > There is a lot of mystery surrounding how large to set > dfs.datanode.max.xcievers. Most people say to just up it to 4096, but given > that exceeding this will cause an HBase RegionServer shutdown (see Lars' blog > post here: http://www.larsgeorge.com/2012/03/hadoop-hbase-and-xceivers.html), > it would be nice if we could expose the current count via the built-in > metrics framework (most likely under dfs). In this way we could watch it to > see if we have it set too high, too low, time to bump it up, etc. > Thoughts? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
