[ https://issues.apache.org/jira/browse/HBASE-24757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17165054#comment-17165054 ]
Hudson commented on HBASE-24757: -------------------------------- Results for branch branch-2.3 [build #190 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/190/]: (x) *{color:red}-1 overall{color}* ---- details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/190/General_20Nightly_20Build_20Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- Something went wrong running this stage, please [check relevant console output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/190//console]. (/) {color:green}+1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/190/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/190/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > ReplicationSink should limit the batch size for batch mutations based on > hbase.rpc.rows.warning.threshold > --------------------------------------------------------------------------------------------------------- > > Key: HBASE-24757 > URL: https://issues.apache.org/jira/browse/HBASE-24757 > Project: HBase > Issue Type: Improvement > Reporter: Viraj Jasani > Assignee: Viraj Jasani > Priority: Major > > At times there are quite a large no of WAL Edits to ship as part of > Replication and sometimes replication queues accumulate huge list of Edits to > process. ReplicationSink at the sink server usually goes through all Edits > and creates map of table -> list of rows grouped by clusterIds, and performs > batch mutation of all rows per table level. However, there is no limit to no > of Rows that are sent as part of batch mutate call. If no of rows > limit > threshold defined by hbase.rpc.rows.warning.threshold, we usually get warn > "Large batch operation detected". If hbase.rpc.rows.size.threshold.reject is > turned on, RS will reject the whole batch without processing. > We should let Replication Sink honour this threshold value and accordingly > keep the size lower per batch mutation call. > Replication triggered batch mutations should always be consumed but keeping > limit of mutation low enough will let the system function at the same pace > and without triggering redundant warnings. This will also restrict > exploitation of cpu cycles at the destination server. -- This message was sent by Atlassian Jira (v8.3.4#803005)