[
https://issues.apache.org/jira/browse/HBASE-28385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HBASE-28385:
-----------------------------------
Labels: pull-request-available (was: )
> Quota estimates are too optimistic for large scans
> --------------------------------------------------
>
> Key: HBASE-28385
> URL: https://issues.apache.org/jira/browse/HBASE-28385
> Project: HBase
> Issue Type: Improvement
> Reporter: Ray Mattingly
> Assignee: Ray Mattingly
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.6.0
>
>
> Let's say you're running a table scan with a throttle of 100MB/sec per
> RegionServer. Ideally your scans are going to pull down large results, often
> containing hundreds or thousands of blocks.
> You will estimate each scan as costing a single block of read capacity, and
> if your quota is already exhausted then the server will evaluate the backoff
> required for your estimated consumption (1 block) to be available. This will
> often be ~1ms, causing your retries to basically be immediate.
> Obviously it will routinely take much longer than 1ms for 100MB of IO to
> become available in the given configuration, so your retries will be destined
> to fail. At worst this can cause a saturation of your server's RPC layer, and
> at best this causes erroneous exhaustion of the client's retries.
> We should find a way to make these estimates a bit smarter for large scans.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
