[
https://issues.apache.org/jira/browse/HBASE-27558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18039324#comment-18039324
]
Viraj Jasani commented on HBASE-27558:
--------------------------------------
FYI, on HBASE-29722, we encountered data integrity issue on HBase 2.5 which is
not present in HBase 2.6 because of this Jira.
> Scan quotas and limits should account for total block IO
> --------------------------------------------------------
>
> Key: HBASE-27558
> URL: https://issues.apache.org/jira/browse/HBASE-27558
> Project: HBase
> Issue Type: Improvement
> Reporter: Bryan Beaudreault
> Assignee: Bryan Beaudreault
> Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-4
>
>
> Scan and Multi requests pull the byte throughput limit from
> Quotas.getReadAvailable(). Multis validate the result inline in
> RSRpcServices, by checking the accumulated
> {{RpcCallContext.getResponseCellSize}} and {{getResponseBlockSize}} against
> the read available after each action. Scans make use of
> {{{}ScannerContext{}}}, and only checks the total cell serialized size and
> {{{}cell.heapSize(){}}}.
> The handling for Multis was added in HBASE-14978. The block size is checked
> because regardless of the actual cell size, the regionserver needs to retain
> entire blocks backing those cells for the lifetime of a request. If the
> retained blocks grows too large, a regionserver can OOM or experience heavy
> GC pressure.
> So multis validate read available against the actual block size retained for
> the responses, but scans only account for cell sizes. We should extend the
> same block support to scans through ScannerContext tracking block bytes
> scanned.
> Large scans can read over ranges of both returned and filtered rows. Despite
> what's returned the users, the server-side cost of the scan is just as
> impacted by filtered rows as non-filtered.
> Both Scans and Multis take the Math.min of Quotas read available and
> hbase.server.scanner.max.result.size. Scans further take the min of that and
> Scan.setMaxResultSize.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
