[
https://issues.apache.org/jira/browse/HDFS-9260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15124049#comment-15124049
]
Jing Zhao commented on HDFS-9260:
---------------------------------
Thanks for updating the patch, [~sfriberg].
Yeah, I think we can do #4 and #7 as follow-on.
For compaction, do you think we can track the total number of nodes in a
treeset? We already know the total number of entries (through size), if we can
also track the total number of nodes and keep it updated along with the
create/delete Node operations, the fill ratio calculation is O(1) per tree and
the read lock holding time can be greatly decreased. Also holding the read lock
around the inner most iteration should also work I think.
For the second part when doing real compaction, currently I do not have exact
time limit number. Maybe we can first use 4ms as mentioned by HDFS-9198. Will
be helpful if [~daryn] can comment here.
> Improve performance and GC friendliness of startup and FBRs
> -----------------------------------------------------------
>
> Key: HDFS-9260
> URL: https://issues.apache.org/jira/browse/HDFS-9260
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode, namenode, performance
> Affects Versions: 2.7.1
> Reporter: Staffan Friberg
> Assignee: Staffan Friberg
> Attachments: FBR processing.png, HDFS Block and Replica Management
> 20151013.pdf, HDFS-7435.001.patch, HDFS-7435.002.patch, HDFS-7435.003.patch,
> HDFS-7435.004.patch, HDFS-7435.005.patch, HDFS-7435.006.patch,
> HDFS-7435.007.patch, HDFS-9260.008.patch, HDFS-9260.009.patch,
> HDFS-9260.010.patch, HDFS-9260.011.patch, HDFS-9260.012.patch,
> HDFS-9260.013.patch, HDFS-9260.014.patch, HDFS-9260.015.patch,
> HDFS-9260.016.patch, HDFSBenchmarks.zip, HDFSBenchmarks2.zip
>
>
> This patch changes the datastructures used for BlockInfos and Replicas to
> keep them sorted. This allows faster and more GC friendly handling of full
> block reports.
> Would like to hear peoples feedback on this change.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
