[
https://issues.apache.org/jira/browse/HBASE-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14995037#comment-14995037
]
Andrew Purtell commented on HBASE-12790:
----------------------------------------
bq. RPC already has priority. Adding more fields seems really dubious when
we're not even using that one well yet.
Not dubious at all when there's a user lined up to take advantage of it. And
optional protobuf fields cost nothing when not used.
bq. Will the server round-robin'ing amongst the client connections work for
phoenix?
I think this addresses the scenario described way up on the description: client
A issues 100 parallel scans, so does client B, we don't want A's work to delay
B's work. This suggests to me A and B are separate clients hence separate
connections. Therefore dispatching work queued per connection in a round robin
manner would satisfy the problem as stated there. Is that correct
[~giacomotaylor] ? However the description must miss some requirement because
when working up the patch [~ram_krish] then introduces the group ID claiming
it's needed for the Phoenix use case and James follows up saying it's only the
client who knows what work belongs in one "group" versus another. But is that
absolutely necessary?
> Support fairness across parallelized scans
> ------------------------------------------
>
> Key: HBASE-12790
> URL: https://issues.apache.org/jira/browse/HBASE-12790
> Project: HBase
> Issue Type: New Feature
> Reporter: James Taylor
> Assignee: ramkrishna.s.vasudevan
> Labels: Phoenix
> Attachments: AbstractRoundRobinQueue.java, HBASE-12790.patch,
> HBASE-12790_1.patch, HBASE-12790_5.patch, HBASE-12790_callwrapper.patch,
> HBASE-12790_trunk_1.patch, PHOENIX_4.5.3-HBase-0.98-2317-SNAPSHOT.zip
>
>
> Some HBase clients parallelize the execution of a scan to reduce latency in
> getting back results. This can lead to starvation with a loaded cluster and
> interleaved scans, since the RPC queue will be ordered and processed on a
> FIFO basis. For example, if there are two clients, A & B that submit largish
> scans at the same time. Say each scan is broken down into 100 scans by the
> client (broken down into equal depth chunks along the row key), and the 100
> scans of client A are queued first, followed immediately by the 100 scans of
> client B. In this case, client B will be starved out of getting any results
> back until the scans for client A complete.
> One solution to this is to use the attached AbstractRoundRobinQueue instead
> of the standard FIFO queue. The queue to be used could be (maybe it already
> is) configurable based on a new config parameter. Using this queue would
> require the client to have the same identifier for all of the 100 parallel
> scans that represent a single logical scan from the clients point of view.
> With this information, the round robin queue would pick off a task from the
> queue in a round robin fashion (instead of a strictly FIFO manner) to prevent
> starvation over interleaved parallelized scans.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
