[ https://issues.apache.org/jira/browse/HDFS-7217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14165387#comment-14165387 ]
Yongjun Zhang commented on HDFS-7217: ------------------------------------- Hi [~kihwal], thanks for addressing my question. I think it's a good change. > Better batching of IBRs > ----------------------- > > Key: HDFS-7217 > URL: https://issues.apache.org/jira/browse/HDFS-7217 > Project: Hadoop HDFS > Issue Type: Improvement > Reporter: Kihwal Lee > Assignee: Kihwal Lee > Fix For: 2.6.0 > > Attachments: HDFS-7217.patch > > > After HDFS-2691 (pipeline recovery in HA), the number of IBR(incremental > block report)s have doubled. Since processing of IBR requires exclusive > FSNamesystem write lock, this can be a source of significant overhead on > clusters with high write load. > On one of the busy clusters, we have observed 60 to 70 percent of available > handlers being constantly occupied by IBRs. This degrades throughput greatly > when compared to 0.23. -- This message was sent by Atlassian JIRA (v6.3.4#6332)