[
https://issues.apache.org/jira/browse/HIVE-21344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16904514#comment-16904514
]
Jesus Camacho Rodriguez commented on HIVE-21344:
------------------------------------------------
I uploaded a patch that changes the current logic.
With new logic, we retrieve the materialized views only from the registry (no
access to metastore) and we check whether the materialized view is valid or not
once we have produced the rewriting. This improves compilation times for those
queries for which a materialized view rewritting is not produced (we do not
need to query the metastore) and also for queries for which the rewriting is
produced (we only check the validity for the materialization that is being used
by the query).
Registry is updated proactively for ops happening in the HS2 where it is
contained, and after detecting a materialization is outdated and periodically
for ops happening in a remote HS2.
> CBO: Reduce compilation time in presence of materialized views
> --------------------------------------------------------------
>
> Key: HIVE-21344
> URL: https://issues.apache.org/jira/browse/HIVE-21344
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
> Affects Versions: 4.0.0
> Reporter: Gopal V
> Assignee: Jesus Camacho Rodriguez
> Priority: Major
> Attachments: calcite-planner-after-fix.svg.zip, mv-get-from-remote.png
>
>
> For every query, {{getAllValidMaterializedViews}} still requires a call to
> metastore to verify that the materializations exist, whether they are
> outdated or not, etc. Since this is only useful for active-active HS2
> deployments, we could take a less aggressive approach and check this
> information only after rewriting has been triggered. In addition, we could
> refresh the information in the HS2 registry periodically in a background
> thread.
> {code}
> // This is not a rebuild, we retrieve all the materializations. In turn, we
> do not need
> // to force the materialization contents to be up-to-date, as this is not a
> rebuild, and
> // we apply the user parameters
> (HIVE_MATERIALIZED_VIEW_REWRITING_TIME_WINDOW) instead.
> materializations = db.getAllValidMaterializedViews(getTablesUsed(basePlan),
> false, getTxnMgr());
> {code}
> !mv-get-from-remote.png!
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
