Letian Jiang created DRILL-8555:
-----------------------------------
Summary: Physical plan cache for parameterized SQL queries
Key: DRILL-8555
URL: https://issues.apache.org/jira/browse/DRILL-8555
Project: Apache Drill
Issue Type: New Feature
Components: Query Planning & Optimization
Reporter: Letian Jiang
h2. Motivation
Repeated queries that differ only in literals are a common workload, yet each
query currently requires full planning. This adds unnecessary latency and
wastes CPU time.
h2. Brief Design
Add a per-Drillbit, in-memory plan cache. Drill automatically parameterizes
eligible SQL queries and reuses a cached physical plan with the current literal
values, without changes to client queries.
h2. Plan reuse safety
* *Engine:* matches the SQL template and checks relevant option changes,
plugin configuration changes, and whether base table versions still match.
* *Storage plugin:* explicitly opts in, provides a plan-compatible table
version, and is responsible for rebuilding its predicate-dependent scan work
from the rebound expressions.
h2. Expected benefit
Short, frequently executed queries such as HBase reads can benefit
substantially. Analytical queries with significant planning cost can benefit as
well.
h2. Implementation status
A working implementation is available, including HBase, Paimon and Iceberg
integrations. A pull request will follow.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
