[
https://issues.apache.org/jira/browse/HDFS-7836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14339385#comment-14339385
]
Charles Lamb commented on HDFS-7836:
------------------------------------
bq. it would be useful to see some perf comparison before we add that
complexity.
We definitely plan on getting some baseline measurements and sharing them. We
definitely want to know what the before and after effects are of any changes.
As an aside, I worked on a case where we had to increase the RPC limit to 192MB
in order to get block reports handled correctly so I know this type of
deployments are out there.
bq. I'll see if I can clean up and post what I used on HDFS-7847.
That would be much appreciated. I'm starting to look at HDFS-7847 (subtask of
this Jira) and maybe that could come into play somehow.
bq. These two sound contradictory.
Yes, they do, but aren't meant to be. The first level would be to do concurrent
processing under the FSN lock. That would at least get some parallelism. The
second step would be to make a more "lockless blocksMap" which wouldn't require
the big FSN lock to be held.
BTW, since you have edit privs, if you want to get rid of my reduntent reply to
you above that would be great. My browser suckered me into hitting "Add" twice.
Thanks.
> BlockManager Scalability Improvements
> -------------------------------------
>
> Key: HDFS-7836
> URL: https://issues.apache.org/jira/browse/HDFS-7836
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Charles Lamb
> Assignee: Charles Lamb
> Attachments: BlockManagerScalabilityImprovementsDesign.pdf
>
>
> Improvements to BlockManager scalability.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
