Umesh Kumar Kumawat created HBASE-29141:
-------------------------------------------
Summary: Default Size of call queues is too high
Key: HBASE-29141
URL: https://issues.apache.org/jira/browse/HBASE-29141
Project: HBase
Issue Type: Bug
Components: master, regionserver
Affects Versions: 2.6.2, 2.5.11, 3.0.0-beta-1
Reporter: Umesh Kumar Kumawat
Assignee: Umesh Kumar Kumawat
By default there is quadratic relation between total number of handler and
total number of calls in general queues.
Regarding the handler and queues HBase have these config. *
{{hbase.regionserver.handler.count}} => Number of handlers, Default is 30.
* {{hbase.ipc.server.callqueue.handler.factor}} => Queue-to-handler ratio.
Default it is 0.1. Means 10 handler per queue.
** {{number of call queues}} = {{handlerCount}} *
{{callQueuesHandlersFactor(0.1)}}
([code|https://github.com/apache/hbase/blob/777010361abb203b8b17673d84acf4f7f1d0283a/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcExecutor.java#L195])
* {{hbase.ipc.server.max.callqueue.length}} => Max callqueue Length. Default
is {{DEFAULT_MAX_CALLQUEUE_LENGTH_PER_HANDLER(10) * total handlerCount}}
([code|https://github.com/apache/hbase/blob/777010361abb203b8b17673d84acf4f7f1d0283a/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java#L71])
** {*}Issue{*}: I think there’s a small issue here. We are currently treating
this as the length per queue, but it's calculated using the {_}total handler
count{_}, not the number of handlers per queue.
* In addition we also have a hard limit of
({{{}DEFAULT_CALL_QUEUE_SIZE_HARD_LIMIT{}}}) 250
([code|https://github.com/apache/hbase/blob/777010361abb203b8b17673d84acf4f7f1d0283a/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcExecutor.java#L228])
this is hard limit on *minimum* value of queue size. Queue size can not be
less then 250.
{{}}
{{}} # I believe we may have unintentionally created a dependency between the
*total number of handlers* and the call queue length. Should we reconsider this
logic?
# Regarding the hard limit: should we be enforcing a _minimum_ value for the
queue size or a _maximum_ value for the queue size ?
# In codel queue type we are we have a config
{{hbase.ipc.server.callqueue.codel.lifo.threshold}} (default 0.8). Having 100
handlers, by default all the 10 queues will be of size 1000 and queue
processing will not change to lifo unless there are 800 calls in a single
queue.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
