[jira] [Updated] (HDFS-14675) Increase Balancer Defaults Further
[ https://issues.apache.org/jira/browse/HDFS-14675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14675: --- Release Note: Increase default bandwidth limit for rebalancing per DataNode (dfs.datanode.balance.bandwidthPerSec) from 10MB/s to 100MB/s. Increase default maximum threads of DataNode balancer (dfs.datanode.balance.max.concurrent.moves) from 50 to 100. > Increase Balancer Defaults Further > -- > > Key: HDFS-14675 > URL: https://issues.apache.org/jira/browse/HDFS-14675 > Project: Hadoop HDFS > Issue Type: Improvement > Components: balancer mover >Affects Versions: 3.3.0 >Reporter: Stephen O'Donnell >Assignee: Stephen O'Donnell >Priority: Major > Fix For: 3.3.0 > > Attachments: HDFS-14675.001.patch > > > HDFS-10297 increased the balancer defaults to 50 for > dfs.datanode.balance.max.concurrent.moves and to 10MB/s for > dfs.datanode.balance.bandwidthPerSec. > We have found that these settings often have to be increased further as users > find the balancer operates too slowly with 50 and 10MB/s. We often recommend > moving concurrent moves to between 200 and 300 and setting the bandwidth to > 100 or even 1000MB/s, and these settings seem to work well in practice. > I would like to suggest we increase the balancer defaults further. I would > suggest 100 for concurrent moves and 100MB/s for the bandwidth, but I would > like to know what others think on this topic too. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14675) Increase Balancer Defaults Further
[ https://issues.apache.org/jira/browse/HDFS-14675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14675: --- Fix Version/s: 3.3.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to trunk. Thanks [~gabor.bota] and [~sodonnell]! > Increase Balancer Defaults Further > -- > > Key: HDFS-14675 > URL: https://issues.apache.org/jira/browse/HDFS-14675 > Project: Hadoop HDFS > Issue Type: Improvement > Components: balancer mover >Affects Versions: 3.3.0 >Reporter: Stephen O'Donnell >Assignee: Stephen O'Donnell >Priority: Major > Fix For: 3.3.0 > > Attachments: HDFS-14675.001.patch > > > HDFS-10297 increased the balancer defaults to 50 for > dfs.datanode.balance.max.concurrent.moves and to 10MB/s for > dfs.datanode.balance.bandwidthPerSec. > We have found that these settings often have to be increased further as users > find the balancer operates too slowly with 50 and 10MB/s. We often recommend > moving concurrent moves to between 200 and 300 and setting the bandwidth to > 100 or even 1000MB/s, and these settings seem to work well in practice. > I would like to suggest we increase the balancer defaults further. I would > suggest 100 for concurrent moves and 100MB/s for the bandwidth, but I would > like to know what others think on this topic too. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14675) Increase Balancer Defaults Further
[ https://issues.apache.org/jira/browse/HDFS-14675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen O'Donnell updated HDFS-14675: - Status: Patch Available (was: Open) > Increase Balancer Defaults Further > -- > > Key: HDFS-14675 > URL: https://issues.apache.org/jira/browse/HDFS-14675 > Project: Hadoop HDFS > Issue Type: Improvement > Components: balancer mover >Affects Versions: 3.3.0 >Reporter: Stephen O'Donnell >Assignee: Stephen O'Donnell >Priority: Major > Attachments: HDFS-14675.001.patch > > > HDFS-10297 increased the balancer defaults to 50 for > dfs.datanode.balance.max.concurrent.moves and to 10MB/s for > dfs.datanode.balance.bandwidthPerSec. > We have found that these settings often have to be increased further as users > find the balancer operates too slowly with 50 and 10MB/s. We often recommend > moving concurrent moves to between 200 and 300 and setting the bandwidth to > 100 or even 1000MB/s, and these settings seem to work well in practice. > I would like to suggest we increase the balancer defaults further. I would > suggest 100 for concurrent moves and 100MB/s for the bandwidth, but I would > like to know what others think on this topic too. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14675) Increase Balancer Defaults Further
[ https://issues.apache.org/jira/browse/HDFS-14675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen O'Donnell updated HDFS-14675: - Attachment: HDFS-14675.001.patch > Increase Balancer Defaults Further > -- > > Key: HDFS-14675 > URL: https://issues.apache.org/jira/browse/HDFS-14675 > Project: Hadoop HDFS > Issue Type: Improvement > Components: balancer mover >Affects Versions: 3.3.0 >Reporter: Stephen O'Donnell >Assignee: Stephen O'Donnell >Priority: Major > Attachments: HDFS-14675.001.patch > > > HDFS-10297 increased the balancer defaults to 50 for > dfs.datanode.balance.max.concurrent.moves and to 10MB/s for > dfs.datanode.balance.bandwidthPerSec. > We have found that these settings often have to be increased further as users > find the balancer operates too slowly with 50 and 10MB/s. We often recommend > moving concurrent moves to between 200 and 300 and setting the bandwidth to > 100 or even 1000MB/s, and these settings seem to work well in practice. > I would like to suggest we increase the balancer defaults further. I would > suggest 100 for concurrent moves and 100MB/s for the bandwidth, but I would > like to know what others think on this topic too. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
