[
https://issues.apache.org/jira/browse/HDFS-5517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13823344#comment-13823344
]
Uma Maheswara Rao G commented on HDFS-5517:
-------------------------------------------
+1 for lowering the value. Thanks Aaron for proposing this change. Recently we
also faced this issue. When file having 2L blocks itself NN is behaving
something abnormal.
For every block addition we do update blocks, so this big array will be
persisted every time incrementally I think. This will be overhead really, if
allow to write more and more blocks for same file with the current code But in
practice I am not sure what will be the scenario to configure less block sizes
and more blocks for same file. But we saw this in testing cluster.
> Lower the default maximum number of blocks per file
> ---------------------------------------------------
>
> Key: HDFS-5517
> URL: https://issues.apache.org/jira/browse/HDFS-5517
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 2.2.0
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
>
> We introduced the maximum number of blocks per file in HDFS-4305, but we set
> the default to 1MM. In practice this limit is so high as to never be hit,
> whereas we know that an individual file with 10s of thousands of blocks can
> cause problems. We should lower the default value, in my opinion to 10k.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
