[jira] [Updated] (HIVE-21500) Replicate conversion of managed table to external at source.
[ https://issues.apache.org/jira/browse/HIVE-21500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21500: Description: Couple of scenarios for Hive2 to Hive3(strict managed tables enabled) replication where managed table is converted to external at source. *Scenario-1: (ACID/MM table converted to external at target)* 1. Create non-ACID ORC format table. 2. Insert some rows 3. Replicate this create event which creates ACID table at target (due to migration rule). Each insert event adds transactional metadata in HMS corresponding to the current table. 4. Convert table to external table using ALTER command at source. *Scenario-2: (External table at target changes table location)* 1. Create non-ACID avro format table. 2. Insert some rows 3. Replicate this create event which creates external table at target (due to migration rule). The data path is chosen under default external warehouse directory. 4. Convert table to external table using ALTER command at source. It is unable to convert an ACID table to external table at target. Also, it is hard to detect what would be the table type at target when perform this ALTER table operation at source. So, it is decided to disable conversion of managed table at source (Hive2) to EXTERNAL or vice-versa. was: Couple of scenarios for Hive2 to Hive3(strict managed tables enabled) replication where managed table is converted to external at source. *Scenario-1: (ACID/MM table converted to external at target)* 1. Create non-ACID ORC format table. 2. Insert some rows 3. Replicate this create event which creates ACID table at target (due to migration rule). Each insert event adds transactional metadata in HMS corresponding to the current table. 4. Convert table to external table using ALTER command at source. 5. Replicating this alter event should convert ACID table to external table and make sure corresponding metadata are removed. *Scenario-2: (External table at target changes table location)* 1. Create non-ACID avro format table. 2. Insert some rows 3. Replicate this create event which creates external table at target (due to migration rule). The data path is chosen under default external warehouse directory. 4. Convert table to external table using ALTER command at source. 5. Replicating this alter event should update the table/partitions location as data moved under external tables base directory. > Replicate conversion of managed table to external at source. > > > Key: HIVE-21500 > URL: https://issues.apache.org/jira/browse/HIVE-21500 > Project: Hive > Issue Type: Sub-task >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, Replication > > Couple of scenarios for Hive2 to Hive3(strict managed tables enabled) > replication where managed table is converted to external at source. > *Scenario-1: (ACID/MM table converted to external at target)* > 1. Create non-ACID ORC format table. > 2. Insert some rows > 3. Replicate this create event which creates ACID table at target (due to > migration rule). Each insert event adds transactional metadata in HMS > corresponding to the current table. > 4. Convert table to external table using ALTER command at source. > *Scenario-2: (External table at target changes table location)* > 1. Create non-ACID avro format table. > 2. Insert some rows > 3. Replicate this create event which creates external table at target (due to > migration rule). The data path is chosen under default external warehouse > directory. > 4. Convert table to external table using ALTER command at source. > It is unable to convert an ACID table to external table at target. Also, it > is hard to detect what would be the table type at target when perform this > ALTER table operation at source. > So, it is decided to disable conversion of managed table at source (Hive2) to > EXTERNAL or vice-versa. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21500) Replicate conversion of managed table to external at source.
[ https://issues.apache.org/jira/browse/HIVE-21500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21500: Description: Couple of scenarios for Hive2 to Hive3(strict managed tables enabled) replication where managed table is converted to external at source. *Scenario-1: (ACID/MM table converted to external at target)* 1. Create non-ACID ORC format table. 2. Insert some rows 3. Replicate this create event which creates ACID table at target (due to migration rule). Each insert event adds transactional metadata in HMS corresponding to the current table. 4. Convert table to external table using ALTER command at source. 5. Replicating this alter event should convert ACID table to external table and make sure corresponding metadata are removed. *Scenario-2: (External table at target changes table location)* 1. Create non-ACID avro format table. 2. Insert some rows 3. Replicate this create event which creates external table at target (due to migration rule). The data path is chosen under default external warehouse directory. 4. Convert table to external table using ALTER command at source. 5. Replicating this alter event should update the table/partitions location as data moved under external tables base directory. was: For the below scenario of Hive2 to Hive3 replication (with strict managed=true), the managed ACID table at target should be converted to external table. 1. Create non-ACID ORC format table. 2. Insert some rows 3. Replicate this create event which creates ACID table at target (due to migration rule). Each insert event adds metadata in HMS corresponding to the current table. 4. Convert table to external table using ALTER command. 5. Replicating this alter event should convert ACID table to external table and make sure corresponding metadata are removed. > Replicate conversion of managed table to external at source. > > > Key: HIVE-21500 > URL: https://issues.apache.org/jira/browse/HIVE-21500 > Project: Hive > Issue Type: Sub-task >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, Replication > > Couple of scenarios for Hive2 to Hive3(strict managed tables enabled) > replication where managed table is converted to external at source. > *Scenario-1: (ACID/MM table converted to external at target)* > 1. Create non-ACID ORC format table. > 2. Insert some rows > 3. Replicate this create event which creates ACID table at target (due to > migration rule). Each insert event adds transactional metadata in HMS > corresponding to the current table. > 4. Convert table to external table using ALTER command at source. > 5. Replicating this alter event should convert ACID table to external table > and make sure corresponding metadata are removed. > *Scenario-2: (External table at target changes table location)* > 1. Create non-ACID avro format table. > 2. Insert some rows > 3. Replicate this create event which creates external table at target (due to > migration rule). The data path is chosen under default external warehouse > directory. > 4. Convert table to external table using ALTER command at source. > 5. Replicating this alter event should update the table/partitions location > as data moved under external tables base directory. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21500) Replicate conversion of managed table to external at source.
[ https://issues.apache.org/jira/browse/HIVE-21500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21500: Summary: Replicate conversion of managed table to external at source. (was: Support converting managed ACID table to external if the corresponding non-ACID table is converted to external at source.) > Replicate conversion of managed table to external at source. > > > Key: HIVE-21500 > URL: https://issues.apache.org/jira/browse/HIVE-21500 > Project: Hive > Issue Type: Sub-task >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, Replication > > For the below scenario of Hive2 to Hive3 replication (with strict > managed=true), the managed ACID table at target should be converted to > external table. > 1. Create non-ACID ORC format table. > 2. Insert some rows > 3. Replicate this create event which creates ACID table at target (due to > migration rule). Each insert event adds metadata in HMS corresponding to the > current table. > 4. Convert table to external table using ALTER command. > 5. Replicating this alter event should convert ACID table to external table > and make sure corresponding metadata are removed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
