Stamatis Zampetakis created HIVE-29217:
------------------------------------------
Summary: Add configuration to choose materialization strategy for
CTEs
Key: HIVE-29217
URL: https://issues.apache.org/jira/browse/HIVE-29217
Project: Hive
Issue Type: Improvement
Reporter: Stamatis Zampetakis
Assignee: Stamatis Zampetakis
Currently there are two ways for materializing common table expressions (CTEs)
that are present in SQL queries.
+SQL/AST based materialization+
It was introduced in HIVE-11752 and relies on the syntax of SQL queries
specifically in presence of WITH clauses. It is performed early on during the
semantic analysis and triggers before any kind of optimization.
+CBO based materialization+
It was introduced in HIVE-28259 and relies on plan equivalences. It is
performed during the cost based optimization phase and after the initial
analysis.
Currently the two strategies interfere with each other and in various cases if
they are both enabled they can affect negatively the performance due to
excessive materialization.
Another drawback is that if a query contains a WITH clause we cannot bypass the
AST materialization (and let the materialization decision entirely on CBO) so
we are risking a non-optimal choice purely based on the syntax of the query.
I propose adding a new configuration property for explicitly selecting between
AST and CBO materialization allowing each materialization strategy to be
applied independently of the other.
In order to avoid changes in behavior and retain backward compatibility the
default materialization strategy will use the old AST based approach.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
