[ 
https://issues.apache.org/jira/browse/HIVE-20967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Bapat updated HIVE-20967:
----------------------------------
    Description: 
Some of the events from Hive2 may cause conflicts in Hive3 
(hive.strict.managed.tables=true) when applied. So, need to handle them 
properly.
 1. Alter table to convert non-acid to acid.
 - This event should be no-op as the table in target might be already acid or 
MM or external table.

2. Alter table or partition that changes the location.
 - For managed tables at source, the table location shouldn't be changed for 
the given non-partitioned table and partition location shouldn't be changed for 
partitioned table as alter event doesn't capture the new files list. So, it may 
cause data inconsistsency. So, if database is enabled for replication at 
source, then alter location on managed tables should be blocked.
 - For external partitioned tables, if location is changed at source, the the 
location should be changed for the table and any partitions which reside within 
the table location, but not for the partitions which are not within the table 
location. (may be we just need the test).

  was:
Some of the events from Hive2 may cause conflicts in Hive3 
(hive.strict.managed.tables=true) when applied. So, need to handle them 
properly.
 1. Alter table to convert non-acid to acid.
 - This event should be no-op as the table in target might be already acid or 
MM or external table. So, don't dump (and replicate) that event. Anyway, any 
event generated on the source on such an ALTERed table won't be dumped (and 
replicated) to the target.

2. Alter table or partition that changes the location.
 - For managed tables at source, the table location shouldn't be changed for 
the given non-partitioned table and partition location shouldn't be changed for 
partitioned table as alter event doesn't capture the new files list. So, it may 
cause data inconsistsency. So, if database is enabled for replication at 
source, then alter location on managed tables should be blocked.
 - For external partitioned tables, if location is changed at source, the the 
location should be changed for the table and any partitions which reside within 
the table location, but not for the partitions which are not within the table 
location. (may be we just need the test).


> Handle alter events when replicate to cluster with hive.strict.managed.tables 
> enabled.
> --------------------------------------------------------------------------------------
>
>                 Key: HIVE-20967
>                 URL: https://issues.apache.org/jira/browse/HIVE-20967
>             Project: Hive
>          Issue Type: Sub-task
>          Components: repl
>    Affects Versions: 4.0.0
>            Reporter: mahesh kumar behera
>            Assignee: Ashutosh Bapat
>            Priority: Minor
>              Labels: DR
>
> Some of the events from Hive2 may cause conflicts in Hive3 
> (hive.strict.managed.tables=true) when applied. So, need to handle them 
> properly.
>  1. Alter table to convert non-acid to acid.
>  - This event should be no-op as the table in target might be already acid or 
> MM or external table.
> 2. Alter table or partition that changes the location.
>  - For managed tables at source, the table location shouldn't be changed for 
> the given non-partitioned table and partition location shouldn't be changed 
> for partitioned table as alter event doesn't capture the new files list. So, 
> it may cause data inconsistsency. So, if database is enabled for replication 
> at source, then alter location on managed tables should be blocked.
>  - For external partitioned tables, if location is changed at source, the the 
> location should be changed for the table and any partitions which reside 
> within the table location, but not for the partitions which are not within 
> the table location. (may be we just need the test).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to