[
https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhuobin zheng updated HBASE-15136:
----------------------------------
Release Note:
Previously RPC request scheduler in HBase had 2 modes in could operate
in:https://translate.google.com/?source=gtx
- simple FIFO
- "partial" deadline, where deadline constraints are only imposed on
long-running scan requests.
This patch adds new type of scheduler to HBase, based on the research around
controlled delay (CoDel) algorithm [1], used in networking to combat
bufferbloat, as well as some analysis on generalizing it to generic request
queues [2]. The purpose of that work is to prevent long standing call queues
caused by discrepancy between request rate and available throughput, caused by
kernel/disk IO/networking stalls.
New RPC scheduler could be enabled by setting
hbase.ipc.server.callqueue.type=codel in configuration. Several additional
params allow to configure algorithm behavior -
hbase.ipc.server.callqueue.codel.target.delay
hbase.ipc.server.callqueue.codel.interval
hbase.ipc.server.callqueue.codel.lifo.threshold
[1] Controlling Queue Delay / A modern AQM is just one piece of the solution to
bufferbloat. http://queue.acm.org/detail.cfm?id=2209336
[2] Fail at Scale / Reliability in the face of rapid change.
http://queue.acm.org/detail.cfm?id=2839461
was:
Previously RPC request scheduler in HBase had 2 modes in could operate in:
- simple FIFO
- "partial" deadline, where deadline constraints are only imposed on
long-running scan requests.
This patch adds new type of scheduler to HBase, based on the research around
controlled delay (CoDel) algorithm [1], used in networking to combat
bufferbloat, as well as some analysis on generalizing it to generic request
queues [2]. The purpose of that work is to prevent long standing call queues
caused by discrepancy between request rate and available throughput, caused by
kernel/disk IO/networking stalls.
New RPC scheduler could be enabled by setting
hbase.ipc.server.callqueue.type=codel in configuration. Several additional
params allow to configure algorithm behavior -
hbase.ipc.server.callqueue.codel.target.delay
hbase.ipc.server.callqueue.codel.interval
hbase.ipc.server.callqueue.codel.lifo.threshold
[1] Controlling Queue Delay / A modern AQM is just one piece of the solution to
bufferbloat. http://queue.acm.org/detail.cfm?id=2209336
[2] Fail at Scale / Reliability in the face of rapid change.
http://queue.acm.org/detail.cfm?id=2839461
> Explore different queuing behaviors while busy
> ----------------------------------------------
>
> Key: HBASE-15136
> URL: https://issues.apache.org/jira/browse/HBASE-15136
> Project: HBase
> Issue Type: New Feature
> Components: IPC/RPC, Scheduler
> Reporter: Elliott Neil Clark
> Assignee: Mikhail Antonov
> Priority: Critical
> Fix For: 1.3.0, 2.0.0
>
> Attachments: HBASE-15136-1.2.v1.patch, HBASE-15136-v2.patch,
> deadline_scheduler_v_0_2.patch
>
>
> http://queue.acm.org/detail.cfm?id=2839461
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
