[jira] [Updated] (HBASE-15136) Explore different queuing behaviors while busy
[ https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhuobin zheng updated HBASE-15136: -- Release Note: Previously RPC request scheduler in HBase had 2 modes in could operate in:https://translate.google.com/?source=gtx - simple FIFO - "partial" deadline, where deadline constraints are only imposed on long-running scan requests. This patch adds new type of scheduler to HBase, based on the research around controlled delay (CoDel) algorithm [1], used in networking to combat bufferbloat, as well as some analysis on generalizing it to generic request queues [2]. The purpose of that work is to prevent long standing call queues caused by discrepancy between request rate and available throughput, caused by kernel/disk IO/networking stalls. New RPC scheduler could be enabled by setting hbase.ipc.server.callqueue.type=codel in configuration. Several additional params allow to configure algorithm behavior - hbase.ipc.server.callqueue.codel.target.delay hbase.ipc.server.callqueue.codel.interval hbase.ipc.server.callqueue.codel.lifo.threshold [1] Controlling Queue Delay / A modern AQM is just one piece of the solution to bufferbloat. http://queue.acm.org/detail.cfm?id=2209336 [2] Fail at Scale / Reliability in the face of rapid change. http://queue.acm.org/detail.cfm?id=2839461 was: Previously RPC request scheduler in HBase had 2 modes in could operate in: - simple FIFO - "partial" deadline, where deadline constraints are only imposed on long-running scan requests. This patch adds new type of scheduler to HBase, based on the research around controlled delay (CoDel) algorithm [1], used in networking to combat bufferbloat, as well as some analysis on generalizing it to generic request queues [2]. The purpose of that work is to prevent long standing call queues caused by discrepancy between request rate and available throughput, caused by kernel/disk IO/networking stalls. New RPC scheduler could be enabled by setting hbase.ipc.server.callqueue.type=codel in configuration. Several additional params allow to configure algorithm behavior - hbase.ipc.server.callqueue.codel.target.delay hbase.ipc.server.callqueue.codel.interval hbase.ipc.server.callqueue.codel.lifo.threshold [1] Controlling Queue Delay / A modern AQM is just one piece of the solution to bufferbloat. http://queue.acm.org/detail.cfm?id=2209336 [2] Fail at Scale / Reliability in the face of rapid change. http://queue.acm.org/detail.cfm?id=2839461 > Explore different queuing behaviors while busy > -- > > Key: HBASE-15136 > URL: https://issues.apache.org/jira/browse/HBASE-15136 > Project: HBase > Issue Type: New Feature > Components: IPC/RPC, Scheduler >Reporter: Elliott Neil Clark >Assignee: Mikhail Antonov >Priority: Critical > Fix For: 1.3.0, 2.0.0 > > Attachments: HBASE-15136-1.2.v1.patch, HBASE-15136-v2.patch, > deadline_scheduler_v_0_2.patch > > > http://queue.acm.org/detail.cfm?id=2839461 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-15136) Explore different queuing behaviors while busy
[ https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Biju Nair updated HBASE-15136: -- Component/s: Scheduler > Explore different queuing behaviors while busy > -- > > Key: HBASE-15136 > URL: https://issues.apache.org/jira/browse/HBASE-15136 > Project: HBase > Issue Type: New Feature > Components: IPC/RPC, Scheduler >Reporter: Elliott Clark >Assignee: Mikhail Antonov >Priority: Critical > Fix For: 1.3.0, 2.0.0 > > Attachments: HBASE-15136-1.2.v1.patch, HBASE-15136-v2.patch, > deadline_scheduler_v_0_2.patch > > > http://queue.acm.org/detail.cfm?id=2839461 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-15136) Explore different queuing behaviors while busy
[ https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15136: -- Priority: Critical (was: Major) > Explore different queuing behaviors while busy > -- > > Key: HBASE-15136 > URL: https://issues.apache.org/jira/browse/HBASE-15136 > Project: HBase > Issue Type: New Feature > Components: IPC/RPC >Reporter: Elliott Clark >Assignee: Mikhail Antonov >Priority: Critical > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15136-1.2.v1.patch, HBASE-15136-v2.patch, > deadline_scheduler_v_0_2.patch > > > http://queue.acm.org/detail.cfm?id=2839461 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15136) Explore different queuing behaviors while busy
[ https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15136: Release Note: Previously RPC request scheduler in HBase had 2 modes in could operate in: - simple FIFO - "partial" deadline, where deadline constraints are only imposed on long-running scan requests. This patch adds new type of scheduler to HBase, based on the research around controlled delay (CoDel) algorithm [1], used in networking to combat bufferbloat, as well as some analysis on generalizing it to generic request queues [2]. The purpose of that work is to prevent long standing call queues caused by discrepancy between request rate and available throughput, caused by kernel/disk IO/networking stalls. New RPC scheduler could be enabled by setting hbase.ipc.server.callqueue.type=codel in configuration. Several additional params allow to configure algorithm behavior - hbase.ipc.server.callqueue.codel.target.delay hbase.ipc.server.callqueue.codel.interval hbase.ipc.server.callqueue.codel.lifo.threshold [1] Controlling Queue Delay / A modern AQM is just one piece of the solution to bufferbloat. http://queue.acm.org/detail.cfm?id=2209336 [2] Fail at Scale / Reliability in the face of rapid change. http://queue.acm.org/detail.cfm?id=2839461 was: Previously RPC request scheduler in HBase had 2 modes in could operate in: - simple FIFO - "partial" deadline, where deadline constraints are only imposed on long-running scan requests. This patch adds new type of scheduler to HBase, based on the research around controlled delay (CoDel) algorithm [1], used in networking to combat bufferbloat, as well as some analysis on generalizing it to generic request queues [2]. The purpose of that work is to prevent long standing call queues caused by discrepancy between request rate and available throughput, caused by kernel/disk IO/networking stalls. New RPC scheduler could be enabled by setting hbase.ipc.server.callqueue.type=codel in configuration. Several additional params allows to configure algorithm behavior - hbase.ipc.server.callqueue.codel.target.delay hbase.ipc.server.callqueue.codel.interval hbase.ipc.server.callqueue.codel.lifo.threshold [1] Controlling Queue Delay / A modern AQM is just one piece of the solution to bufferbloat. http://queue.acm.org/detail.cfm?id=2209336 [2] Fail at Scale / Reliability in the face of rapid change. http://queue.acm.org/detail.cfm?id=2839461 > Explore different queuing behaviors while busy > -- > > Key: HBASE-15136 > URL: https://issues.apache.org/jira/browse/HBASE-15136 > Project: HBase > Issue Type: New Feature > Components: IPC/RPC >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15136-1.2.v1.patch, HBASE-15136-v2.patch, > deadline_scheduler_v_0_2.patch > > > http://queue.acm.org/detail.cfm?id=2839461 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15136) Explore different queuing behaviors while busy
[ https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15136: Release Note: Previously RPC request scheduler in HBase had 2 modes in could operate in: - simple FIFO - "partial" deadline, where deadline constraints are only imposed on long-running scan requests. This patch adds new type of scheduler to HBase, based on the research around controlled delay (CoDel) algorithm [1], used in networking to combat bufferbloat, as well as some analysis on generalizing it to generic request queues [2]. The purpose of that work is to prevent long standing call queues caused by discrepancy between request rate and available throughput, caused by kernel/disk IO/networking stalls. New RPC scheduler could be enabled by setting hbase.ipc.server.callqueue.type=codel in configuration. Several additional params allows to configure algorithm behavior - hbase.ipc.server.callqueue.codel.target.delay hbase.ipc.server.callqueue.codel.interval hbase.ipc.server.callqueue.codel.lifo.threshold [1] Controlling Queue Delay / A modern AQM is just one piece of the solution to bufferbloat. http://queue.acm.org/detail.cfm?id=2209336 [2] Fail at Scale / Reliability in the face of rapid change. http://queue.acm.org/detail.cfm?id=2839461 > Explore different queuing behaviors while busy > -- > > Key: HBASE-15136 > URL: https://issues.apache.org/jira/browse/HBASE-15136 > Project: HBase > Issue Type: New Feature > Components: IPC/RPC >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15136-1.2.v1.patch, HBASE-15136-v2.patch, > deadline_scheduler_v_0_2.patch > > > http://queue.acm.org/detail.cfm?id=2839461 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15136) Explore different queuing behaviors while busy
[ https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15136: Fix Version/s: 1.3.0 2.0.0 > Explore different queuing behaviors while busy > -- > > Key: HBASE-15136 > URL: https://issues.apache.org/jira/browse/HBASE-15136 > Project: HBase > Issue Type: New Feature > Components: IPC/RPC >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15136-1.2.v1.patch, HBASE-15136-v2.patch, > deadline_scheduler_v_0_2.patch > > > http://queue.acm.org/detail.cfm?id=2839461 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15136) Explore different queuing behaviors while busy
[ https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15136: Resolution: Fixed Status: Resolved (was: Patch Available) > Explore different queuing behaviors while busy > -- > > Key: HBASE-15136 > URL: https://issues.apache.org/jira/browse/HBASE-15136 > Project: HBase > Issue Type: New Feature > Components: IPC/RPC >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Attachments: HBASE-15136-1.2.v1.patch, HBASE-15136-v2.patch, > deadline_scheduler_v_0_2.patch > > > http://queue.acm.org/detail.cfm?id=2839461 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15136) Explore different queuing behaviors while busy
[ https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15136: Status: Open (was: Patch Available) > Explore different queuing behaviors while busy > -- > > Key: HBASE-15136 > URL: https://issues.apache.org/jira/browse/HBASE-15136 > Project: HBase > Issue Type: New Feature > Components: IPC/RPC >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Attachments: HBASE-15136-1.2.v1.patch, HBASE-15136-v2.patch, > deadline_scheduler_v_0_2.patch > > > http://queue.acm.org/detail.cfm?id=2839461 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15136) Explore different queuing behaviors while busy
[ https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15136: Status: Patch Available (was: Open) > Explore different queuing behaviors while busy > -- > > Key: HBASE-15136 > URL: https://issues.apache.org/jira/browse/HBASE-15136 > Project: HBase > Issue Type: New Feature > Components: IPC/RPC >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Attachments: HBASE-15136-1.2.v1.patch, HBASE-15136-v2.patch, > deadline_scheduler_v_0_2.patch > > > http://queue.acm.org/detail.cfm?id=2839461 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15136) Explore different queuing behaviors while busy
[ https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15136: Status: Patch Available (was: Open) > Explore different queuing behaviors while busy > -- > > Key: HBASE-15136 > URL: https://issues.apache.org/jira/browse/HBASE-15136 > Project: HBase > Issue Type: New Feature > Components: IPC/RPC >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Attachments: HBASE-15136-1.2.v1.patch, HBASE-15136-v2.patch, > deadline_scheduler_v_0_2.patch > > > http://queue.acm.org/detail.cfm?id=2839461 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15136) Explore different queuing behaviors while busy
[ https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15136: Status: Open (was: Patch Available) > Explore different queuing behaviors while busy > -- > > Key: HBASE-15136 > URL: https://issues.apache.org/jira/browse/HBASE-15136 > Project: HBase > Issue Type: New Feature > Components: IPC/RPC >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Attachments: HBASE-15136-1.2.v1.patch, HBASE-15136-v2.patch, > deadline_scheduler_v_0_2.patch > > > http://queue.acm.org/detail.cfm?id=2839461 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15136) Explore different queuing behaviors while busy
[ https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15136: Attachment: HBASE-15136-v2.patch Some more improvements, dynamically configurable CoDel properties, added test, made sure deadline is still the default. > Explore different queuing behaviors while busy > -- > > Key: HBASE-15136 > URL: https://issues.apache.org/jira/browse/HBASE-15136 > Project: HBase > Issue Type: New Feature > Components: IPC/RPC >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Attachments: HBASE-15136-1.2.v1.patch, HBASE-15136-v2.patch, > deadline_scheduler_v_0_2.patch > > > http://queue.acm.org/detail.cfm?id=2839461 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15136) Explore different queuing behaviors while busy
[ https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15136: Status: Patch Available (was: Open) > Explore different queuing behaviors while busy > -- > > Key: HBASE-15136 > URL: https://issues.apache.org/jira/browse/HBASE-15136 > Project: HBase > Issue Type: New Feature > Components: IPC/RPC >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Attachments: HBASE-15136-1.2.v1.patch, deadline_scheduler_v_0_2.patch > > > http://queue.acm.org/detail.cfm?id=2839461 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15136) Explore different queuing behaviors while busy
[ https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15136: Attachment: HBASE-15136-1.2.v1.patch > Explore different queuing behaviors while busy > -- > > Key: HBASE-15136 > URL: https://issues.apache.org/jira/browse/HBASE-15136 > Project: HBase > Issue Type: New Feature > Components: IPC/RPC >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Attachments: HBASE-15136-1.2.v1.patch, deadline_scheduler_v_0_2.patch > > > http://queue.acm.org/detail.cfm?id=2839461 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15136) Explore different queuing behaviors while busy
[ https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15136: Attachment: (was: HBASE-15136-1.2.v1.branch) > Explore different queuing behaviors while busy > -- > > Key: HBASE-15136 > URL: https://issues.apache.org/jira/browse/HBASE-15136 > Project: HBase > Issue Type: New Feature > Components: IPC/RPC >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Attachments: deadline_scheduler_v_0_2.patch > > > http://queue.acm.org/detail.cfm?id=2839461 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15136) Explore different queuing behaviors while busy
[ https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15136: Attachment: HBASE-15136-1.2.v1.branch First version of patch, for 1.2 branch for now as that's where I test it, will rebase on master soon. > Explore different queuing behaviors while busy > -- > > Key: HBASE-15136 > URL: https://issues.apache.org/jira/browse/HBASE-15136 > Project: HBase > Issue Type: New Feature > Components: IPC/RPC >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Attachments: deadline_scheduler_v_0_2.patch > > > http://queue.acm.org/detail.cfm?id=2839461 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15136) Explore different queuing behaviors while busy
[ https://issues.apache.org/jira/browse/HBASE-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15136: Attachment: deadline_scheduler_v_0_2.patch Attached draft patch sketching deadline-based scheduler (for all calls vs. only long-running scans, as of now) to see changes of what scope we would look for, will update patch when I try controlled delay thing from the paper. > Explore different queuing behaviors while busy > -- > > Key: HBASE-15136 > URL: https://issues.apache.org/jira/browse/HBASE-15136 > Project: HBase > Issue Type: New Feature > Components: IPC/RPC >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Attachments: deadline_scheduler_v_0_2.patch > > > http://queue.acm.org/detail.cfm?id=2839461 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
