Hanifi Gunes created DRILL-4136:
-----------------------------------
Summary: Enhance queue support to take query cost & available
cluster resources into account
Key: DRILL-4136
URL: https://issues.apache.org/jira/browse/DRILL-4136
Project: Apache Drill
Issue Type: Improvement
Components: Execution - Flow
Affects Versions: 1.3.0
Reporter: Hanifi Gunes
Assignee: Hanifi Gunes
Current queue support relies on a distributed semaphore around a fix
pre-defined number. This semaphore indicates the number of queries Drill can
run concurrently. Presently, we define small and large queues where we classify
queries based on a threshold and use two semaphores around small and large
queues individually.
This issue proposes to come up with an enhanced queueing or query dispatch
mechanism where a query is granted execution based on its cost and availability
of system resources(cpu, io, memory etc). Enhancing cost planing and
introducing a distributed resource management should be addressed later to
fully benefit from this enhancement. The proposal is a non-blocking and
asynchronous mechanism that assumes eventual consistency around available
system resources.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
