[ https://issues.apache.org/jira/browse/HIVE-21886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
mahesh kumar behera updated HIVE-21886: --------------------------------------- Status: Patch Available (was: Open) > REPL - With table list - Handle rename events during replace policy > ------------------------------------------------------------------- > > Key: HIVE-21886 > URL: https://issues.apache.org/jira/browse/HIVE-21886 > Project: Hive > Issue Type: Sub-task > Components: repl > Reporter: mahesh kumar behera > Assignee: mahesh kumar behera > Priority: Major > Labels: DR, Replication, pull-request-available > Attachments: HIVE-21886.01.patch, HIVE-21886.02.patch, > HIVE-21886.03.patch, HIVE-21886.04.patch, HIVE-21886.04.patch, > HIVE-21886.05.patch > > Time Spent: 11h 10m > Remaining Estimate: 0h > > If some rename events are found to be dumped and replayed while replace > policy is getting executed, it needs to take care of the policy inclusion in > both the policy for each table name. > 1. Create a list of tables to be bootstrapped. > 2. During handling of alter table, if the alter type is rename > 1. If the old table name is present in the list of table to be > bootstrapped, remove it. > 2. If the new table name, matches the new policy, add it to the list > of tables to be bootstrapped. > 3. If the old table does not match the old policy drop it, even if the > table is not present at target. > 3. During handling of drop table > 1. if the table is in the list of tables to be bootstrapped, then > remove it and ignore the event. > 4. During other event handling > 1. if the table is there in the list of tables to be bootstrapped, > then ignore the event. > 2. If the new policy does not match the table name, then ignore the > event. > > Rename handling during replace policy > # Old name not matching old policy – The old table will not be there at the > target cluster. The table will not be returned by get-all-table. > ## Old name is not matching new policy > ### New name not matching old policy > #### New name not matching new policy > ***** Ignore the event, no need to do anything. > #### New name matching new policy > ***** The table will be returned by get-all-table. Replace policy handler > will bootstrap this table as its matching new policy and not matching old > policy. > ***** All the future events will be ignored as part of check added by > replace policy handling. > ***** All the event with old table name will anyways be ignored as the old > name is not matching the new policy. > ### New name matching old policy > #### New name not matching new policy > ***** As the new name is not matching the new policy, the table need not be > replicated. > ***** As the old name is not matching the new policy, the rename events will > be ignored. > ***** So nothing to be done for this scenario. > #### New name matching new policy > ***** As the new name is matching both old and new policy, replace handler > will not bootstrap the table. > ***** Add the table to the list of tables to be bootstrapped. > ***** Ignore all the events with new name. > ***** If there is a drop event for the table (with new name), then remove > the table from the the list of table to be bootstrapped. > ***** In case of rename event (double rename) > ****** If the new name satisfies the table pattern, then add the new name to > the list of tables to be bootstrapped and remove the old name from the list > of tables to be bootstrapped. > ****** If the new name does not satisfies then just removed the table name > from the list of tables to be bootstrapped. > ## Old name is matching new policy – As per replace policy handler, which > checks based on old table, the table should be bootstrapped and event should > be ignored. But rename handler should decide based on new name.The old table > name will not be returned by get-all-table, so replace handler will not d > anything for the old table. > ### New name not matching old policy > #### New name not matching new policy > ***** As the old table is not there at target and new name is not matching > new policy. Ignore the event. > ***** No need to add the table to the list of tables to be bootstrapped. > ***** All the subsequent events will be ignored as the new name is not > matching the new policy. > #### New name matching new policy > ***** As the new name is not matching old policy but matching new policy, > the table will be bootstrapped by replace policy handler. So rename event > need not add this table to list of table to be bootstrapped. > ***** All the future events will be ignored by replace policy handler. > ***** For rename event (double rename) > ****** If there is a rename, the table (with intermittent new name) will not > be present and thus replace handler will not bootstrap the table. > ****** So if the new name (the latest one) is matching the new policy, then > add it to the list of table to be bootstrapped. > ****** And If the new name (the latest one) is not matching the new policy, > then just ignore the event as the intermittent new name would not have added > to the list of table to be bootstrapped. > ### New name matching old policy > #### New name not matching new policy > ***** Dump the event. The table will be dropped by repl load at the target. > #### New name matching new policy > ***** Replace handler will not bootstrap this table as the new name is > matching both policies. > ***** As old name is not matching the old policy, the table will not be > there at target. The rename event should add the new name to the list of > table to be bootstrapped. > ***** Subsequent events with new table name should be ignored. > ***** Drop events should not be ignored as if the table is present during > bootstrapped, then its a new table and thus should be dropped. > ***** In case of rename event (double rename) > ****** If the new name satisfies the table pattern, then add the new name to > the list of tables to be bootstrapped and remove the old name from the list > of tables to be bootstrapped. > ****** If the new name does not satisfies then just removed the table name > from the list of tables to be bootstrapped. > # Old name is matching old policy – The old table will be there at the > target cluster. The table will not be returned by get-all-table. Repl load > should delete the old table as it is not matching the new policy. > ## Old name is not matching new policy > ### New name not matching old policy > #### New name not matching new policy > ***** Nothing to be done. Ignore the event. > #### New name matching new policy > ***** As the new name is not matching old policy but matching new policy, > the table will be bootstrapped by replace policy handler. So rename event > need not add this table to list of table to be bootstrapped. > ***** All the future events will be ignored by replace policy handler. > ***** For rename event (double rename) > ****** If there is a rename, the table (with intermittent new name) will not > be present and thus replace handler will not bootstrap the table. > ****** So if the new name (the latest one) is matching the new policy, then > add it to the list of table to be bootstrapped. > ****** And If the new name (the latest one) is not matching the new policy, > then just ignore the event as the intermittent new name would not have added > to the list of table to be bootstrapped. > ### New name matching old policy > #### New name not matching new policy > ***** Table with new name will be dropped by repl load > ***** Along with other event, ignore the rename event also. > #### New name matching new policy > ***** As the new name is matching both old and new policy, replace handler > will not bootstrap the table. > ***** Add the table to the list of tables to be bootstrapped. > ***** Ignore all the events with new name. > ***** If there is a drop event for the table (with new name), then remove > the table from the the list of table to be bootstrapped. > ***** In case of rename event (double rename) > ****** If the new name satisfies the table pattern, then add the new name to > the list of tables to be bootstrapped and remove the old name from the list > of tables to be bootstrapped. > ****** If the new name does not satisfies then just removed the table name > from the list of tables to be bootstrapped. > ## Old name is matching new policy > ### New name not matching old policy > #### New name not matching new policy > ***** The old table needs to be dropped at target. Ignore this event, as the > old table is not matching the new policy, it will be dropped by repl load. > #### New name matching new policy > ***** Allow the event to dump and replayed at target. > ***** Allow further events to be handled as usual. > ***** In case of rename event (double rename) > ****** If the latest new name is matching the new policy, then keep it as is > it. Let rename event replayed at target. > ****** If the latest new name is not matching the new policy, then change > the rename event to drop event. > ### New name matching old policy > #### New name not matching new policy > ##### Nothing to be done. > #### New name matching new policy > ##### Add the table name to the list of tables to be bootstrapped. -- This message was sent by Atlassian JIRA (v7.6.3#76005)