[
https://issues.apache.org/jira/browse/HDFS-15253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072267#comment-17072267
]
Dinesh Chitlangia commented on HDFS-15253:
------------------------------------------
[~kpalanisamy] - Thanks for filing this jira. Yes, restricting the bandwith to
50mb/s makes sense. When I work with customers who are running HDFS at scale,
that is the first thing I recommend them.
Regarding dfs.image.compress, I have fairly little experience and have not seen
much benefit with it other than reduced file size.
dfs.namenode.checkpoint.txns can vary based on the cluster usage. So no matter
what value is set as default, there will always be a large set of users who
would still have to tune it based on their cluster usage. So I would recommend
we leave it at default 1M.
> Set default throttle value on dfs.image.transfer.bandwidthPerSec
> ----------------------------------------------------------------
>
> Key: HDFS-15253
> URL: https://issues.apache.org/jira/browse/HDFS-15253
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Reporter: Karthik Palanisamy
> Assignee: Karthik Palanisamy
> Priority: Major
>
> The default value dfs.image.transfer.bandwidthPerSec is set to 0 so it can
> use maximum available bandwidth for fsimage transfers during checkpoint. I
> think we should throttle this. Many users were experienced namenode failover
> when transferring large image size along with fsimage replication on
> dfs.namenode.name.dir. eg. >25Gb.
> Thought to set,
> dfs.image.transfer.bandwidthPerSec=52428800. (50 MB/s)
> dfs.namenode.checkpoint.txns=2000000 (Default is 1M, good to avoid frequent
> checkpoint. However, the default checkpoint runs every 6 hours once)
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
