[
https://issues.apache.org/jira/browse/HIVE-16785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sankar Hariappan updated HIVE-16785:
------------------------------------
Attachment: HIVE-16785.01.patch
Added 01.patch with below updates.
- Added repl.last.id validation for RENAME and TRUNCATE events against
table/partition objects. If the operating object doesn't exist, then the event
is noop because it means, the table/partition is already dropped/renamed and
hence no need to apply this event.
- Updated createReplImportTasks method to validate the eventId against
repl.last.id of table object if exists and if not exists, then validate against
database object. This ensures, table is created only if not exist and is not
dropped or renamed already.
- Removed unnecessary handling of drop+create in-case of partition policy
different from the new table.
- Update table or database object's repl.last.id only if the last applied
eventId is newer than the object itself. In base code, it was always updated.
Request [~anishek]/[~sushanth] to review the patch!
cc [~thejas]
> Ensure replication actions are idempotent if any series of events are applied
> again.
> ------------------------------------------------------------------------------------
>
> Key: HIVE-16785
> URL: https://issues.apache.org/jira/browse/HIVE-16785
> Project: Hive
> Issue Type: Sub-task
> Components: Hive, repl
> Affects Versions: 2.1.0
> Reporter: Sankar Hariappan
> Assignee: Sankar Hariappan
> Labels: DR, replication
> Attachments: HIVE-16785.01.patch
>
>
> Some of the events(ALTER, RENAME, TRUNCATE) are not idempotent and hence
> leads to failure of REPL LOAD if applied twice or applied on an object which
> is latest than current event. For example, if TRUNCATE is applied on a table
> which is already dropped will fail instead of noop.
> Also, need to consider the scenario where the object is missing while
> applying an event. For example, if RENAME_TABLE event is applied on target
> where the old table is missing should validate if table should be recreated
> or should treat the event as noop. This can be done by verifying the DB level
> last repl ID against the current event ID.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
