mahesh kumar behera created HIVE-21774:
------------------------------------------
Summary: Support partition level filtering for events with
multiple partitions
Key: HIVE-21774
URL: https://issues.apache.org/jira/browse/HIVE-21774
Project: Hive
Issue Type: Sub-task
Components: HiveServer2, repl
Affects Versions: 4.0.0
Reporter: mahesh kumar behera
Assignee: mahesh kumar behera
Fix For: 4.0.0
Some of the events in hive can span across multiple partitions, table or even
database. Events related to transactions, can span across multiple databases.
When a transaction does some write operation, it is added to the write
notification log table. During dump of commit transaction event, al the entries
present in the write notification log table for that transaction is read and is
added to the commit transaction message. In case partition filter is supplied
for the dump, only those partitions which are part of the policy should be
added to the commit txn message.
* All the events which are not partition level will be added to the list of
events to be dumped.
* Pass the filter condition for the policy to commit transaction message
handler (events which are not partition level).
* During dump for commit transaction event, extract the events added in the
write notification log table and compare it with the filter condition.
* If the event from write notification log satisfies the filter condition,
then add it to the commit transaction message.
* If filter condition is null, then add all the events from write notification
log table to commit transaction message.
* For events which does not have partition level info like open txn, abort txn
etc, just dump the events without any filtering. So it may happen that some of
events which are not related to any of the satisfying partition, may get
replayed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
