[
https://issues.apache.org/jira/browse/HIVE-18825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16388286#comment-16388286
]
Jesus Camacho Rodriguez commented on HIVE-18825:
------------------------------------------------
bq. Could the predicates you want be inserted at compile time, but bound to
actual values as some post processing after (or at the end of)
Driver.acquireLocks() as currently implemented?
That is somehow similar to parameterized queries, that is quite a lot of work
to implement and falls out of scope I think, at least right now.
Also, I think there is a problem with acquiring the locks after compilation
that we had not realized about beforehand ([~ashutoshc] pointed it out).
Queries use information about available partitions during compilation phase...
What happens when new partitions are added or they are removed between
compilation phase and the moment when locks are acquired and valid txn list is
generated? This might lead to incorrect results if we are not misunderstanding
the whole mechanism.
I think the right fix is to call {{lockAndRespond}} after parsing the query and
before optimization/compilation starts, since at that moment we will have all
the information that we need to proceed. What do you think?
> Define ValidTxnList before starting query optimization
> ------------------------------------------------------
>
> Key: HIVE-18825
> URL: https://issues.apache.org/jira/browse/HIVE-18825
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
> Affects Versions: 3.0.0
> Reporter: Jesus Camacho Rodriguez
> Assignee: Jesus Camacho Rodriguez
> Priority: Major
> Attachments: HIVE-18825.01.patch, HIVE-18825.02.patch,
> HIVE-18825.03.patch, HIVE-18825.04.patch, HIVE-18825.patch
>
>
> Consider a set of tables used by a materialized view where inserts happened
> after the materialization was created. To compute incremental view
> maintenance, we need to be able to filter only new rows from those base
> tables. That can be done by inserting a filter operator with condition e.g.
> {{ROW\_\_ID.transactionId < highwatermark and ROW\_\_ID.transactionId NOT
> IN(<open txns>)}} on top of the MVs query definition and triggering the
> rewriting (which should in turn produce a partial rewriting). However, to do
> that, we need to have a value for {{ValidTxnList}} during query compilation
> so we know the snapshot that we are querying.
> This patch aims to generate {{ValidTxnList}} before query optimization. There
> should not be any visible changes for end user.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
