[ https://issues.apache.org/jira/browse/HDFS-7217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14166963#comment-14166963 ]
Hudson commented on HDFS-7217: ------------------------------ FAILURE: Integrated in Hadoop-Mapreduce-trunk #1922 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1922/]) HDFS-7217. Better batching of IBRs. Contributed by Kihwal Lee. (kihwal: rev db71bb54bcc75b71c5841b25ceb03fb0218c6d4f) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestIncrementalBlockReports.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java > Better batching of IBRs > ----------------------- > > Key: HDFS-7217 > URL: https://issues.apache.org/jira/browse/HDFS-7217 > Project: Hadoop HDFS > Issue Type: Improvement > Reporter: Kihwal Lee > Assignee: Kihwal Lee > Fix For: 2.6.0 > > Attachments: HDFS-7217.patch > > > After HDFS-2691 (pipeline recovery in HA), the number of IBR(incremental > block report)s have doubled. Since processing of IBR requires exclusive > FSNamesystem write lock, this can be a source of significant overhead on > clusters with high write load. > On one of the busy clusters, we have observed 60 to 70 percent of available > handlers being constantly occupied by IBRs. This degrades throughput greatly > when compared to 0.23. -- This message was sent by Atlassian JIRA (v6.3.4#6332)