[
https://issues.apache.org/jira/browse/HBASE-28385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Beaudreault resolved HBASE-28385.
---------------------------------------
Fix Version/s: 3.0.0-beta-2
Release Note: When hbase.quota.use.result.size.bytes is false, we will now
estimate the amount of quota to grab for a scan based on the block bytes
scanned of previous next() requests. This will increase throughput for large
scans which might prefer to wait a little longer for a larger portion of the
quota.
Resolution: Fixed
> Quota estimates are too optimistic for large scans
> --------------------------------------------------
>
> Key: HBASE-28385
> URL: https://issues.apache.org/jira/browse/HBASE-28385
> Project: HBase
> Issue Type: Improvement
> Reporter: Ray Mattingly
> Assignee: Ray Mattingly
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.6.0, 3.0.0-beta-2
>
>
> Let's say you're running a table scan with a throttle of 100MB/sec per
> RegionServer. Ideally your scans are going to pull down large results, often
> containing hundreds or thousands of blocks.
> You will estimate each scan as costing a single block of read capacity, and
> if your quota is already exhausted then the server will evaluate the backoff
> required for your estimated consumption (1 block) to be available. This will
> often be ~1ms, causing your retries to basically be immediate.
> Obviously it will routinely take much longer than 1ms for 100MB of IO to
> become available in the given configuration, so your retries will be destined
> to fail. At worst this can cause a saturation of your server's RPC layer, and
> at best this causes erroneous exhaustion of the client's retries.
> We should find a way to make these estimates a bit smarter for large scans.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
