Currently, the handlers for processing RPC calls are divided into three
types, they are write, read and scan. Correspondingly, each type of handler
corresponds to a certain number of queues.

The corresponding mode of handler and queue is according to
"hbase.ipc.server.callqueue.handler.factor", the default is 0.1, that is,
the default ratio of the number of queues to the number of handlers is
1:10. The handlers for a call type are divided into handler*0.1 groups,
while each group handlers consume the same queue.

I tested the performance between single queue and multi queue mode, the
result is that, for requests of similar size, multi queue and single queue
do not impact the performance too much, as displayed in
https://issues.apache.org/jira/browse/HBASE-27683.
But for scenarios where the request size or the process time of calls
varies greatly, a single queue has a very big advantage, it can make full
use of all handlers to handle all requests of the same type, such as all
the get handlers to process all the get requests. For the call queues, when
the producing speed is larger than or equal to the speed of consuming, just
several large calls can block the processing of small calls in the same
queue.
We found a scenario for feature stores in our production clusters, while
the requests/responses size of gets varies too much, single queue mode can
help to greatly reduce the queue time of calls.

Thanks.

Reply via email to