[ https://issues.apache.org/jira/browse/HBASE-28672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17864846#comment-17864846 ]
Hudson commented on HBASE-28672: -------------------------------- Results for branch branch-3 [build #247 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/247/]: (x) *{color:red}-1 overall{color}* ---- details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/247/General_20Nightly_20Build_20Report/] (/) {color:green}+1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/230/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/230/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (x) {color:red}-1 jdk17 hadoop3 checks{color} -- For more information [see jdk17 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/247/JDK17_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Ensure large batches are not indefinitely blocked by quotas > ----------------------------------------------------------- > > Key: HBASE-28672 > URL: https://issues.apache.org/jira/browse/HBASE-28672 > Project: HBase > Issue Type: Improvement > Components: Quotas > Affects Versions: 2.6.0 > Reporter: Ray Mattingly > Assignee: Ray Mattingly > Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2, 2.6.1 > > > At my day job we are trying to implement default quotas for a variety of > access patterns. We began by introducing a default read IO limit per-user, > per-machine — this has been very successful in reducing hotspots, even on > clusters with thousands of distinct users. > While implementing a default writes/second throttle, I realized that doing so > would put us in a precarious situation where large-enough batches may never > succeed. If your batch size is greater than your TimeLimiter's max > throughput, then you will always fail in the quota estimation stage. > Meanwhile [IO estimates are more > optimistic|https://github.com/apache/hbase/blob/bdb3f216e864e20eb2b09352707a751a5cf7460f/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/DefaultOperationQuota.java#L192-L193], > deliberately, which can let large requests do targeted oversubscription of > an IO quota: > > {code:java} > // assume 1 block required for reads. this is probably a low estimate, which > is okay > readConsumed = numReads > 0 ? blockSizeBytes : 0;{code} > > This is okay because the Limiter's availability will go negative and force a > longer backoff on subsequent requests. I believe this is preferable UX > compared to a doomed throttling loop. > In my opinion, we should do something similar in batch request estimation, by > estimating a batch request's workload at {{Math.min(batchSize, > limiterMaxThroughput)}} rather than simply {{{}batchSize{}}}. -- This message was sent by Atlassian Jira (v8.20.10#820010)