[
https://issues.apache.org/jira/browse/HIVE-27728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17768963#comment-17768963
]
Naveen Gangam commented on HIVE-27728:
--------------------------------------
[~dengzh] Would you be able to take a look at this?
For tables that are converted/translated to EXTERNAL_TABLE, we add
"TRANSLATED_TO_EXTERNAL" to the table properties. So any partitions (or even
table) that are being renamed for such tables, we can treat them as the same
for MANAGED tables and move the data as well.
https://issues.apache.org/jira/browse/HIVE-26158 might give some guidance.
> Changed behavior for alter table rename partition from legacy tables
> --------------------------------------------------------------------
>
> Key: HIVE-27728
> URL: https://issues.apache.org/jira/browse/HIVE-27728
> Project: Hive
> Issue Type: Bug
> Reporter: Naveen Gangam
> Assignee: Zhihua Deng
> Priority: Major
>
> set hive.create.as.external.legacy=true;
> CREATE TABLE default.metadata_test1(
> emp_number int,
> emp_name string,
> city string)
> PARTITIONED BY(state string)
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ‘,’;
> INSERT INTO default.metadata_test1 PARTITION(state=‘A’) VALUES (11, ‘ABC’,
> ‘AA’);
> INSERT INTO default.metadata_test1 PARTITION(state=‘B’) VALUES (12, ‘XYZ’,
> ‘BX’);
> INSERT INTO default.metadata_test1 PARTITION(state=‘B’) VALUES (13, ‘UVW’,
> ‘BU’);
> from hdfs
> $ hdfs dfs -ls /warehouse/tablespace/external/hive/metadata_test1
> Found 2 items
> drwxr-xr-x - hive hive 0 2023-08-11 14:24
> /warehouse/tablespace/external/hive/metadata_test1/state=A
> drwxr-xr-x - hive hive 0 2023-08-11 14:25
> /warehouse/tablespace/external/hive/metadata_test1/state=B
> Now when we alter the partition.
> ALTER TABLE default.metadata_test1 PARTITION (state=‘A’) RENAME TO PARTITION
> (state=‘C’);
> select * from default.metadata_test1;
> +----------------------------+--------------------------+----------------------+-----------------------+
> | metadata_test1.emp_number | metadata_test1.emp_name | metadata_test1.city |
> metadata_test1.state |
> +----------------------------+--------------------------+----------------------+-----------------------+
> | 12 | XYZ | BX | B |
> | 13 | UVW | BU | B |
> | 11 | ABC | AA | C |
> +----------------------------+--------------------------+----------------------+-----------------------+
> show partitions default.metadata_test1;
> +------------+
> | partition |
> +------------+
> | state=B |
> | state=C |
> +------------+
> But from HDFS
> hdfs dfs -ls /warehouse/tablespace/external/hive/metadata_test1
> Found 2 items
> drwxr-xr-x - hive hive 0 2023-08-11 14:24
> /warehouse/tablespace/external/hive/metadata_test1/state=A
> drwxr-xr-x - hive hive 0 2023-08-11 14:25
> /warehouse/tablespace/external/hive/metadata_test1/state=B
> Hive only relocates/renames the partition locations to match the new
> partition value if it is a MANAGED table. For external tables, we assume that
> ETL pipelines are writing to the old location and thus will not rename the
> dir.
> But for legacy tables, (create table) that would have created non-acid
> managed tables, we convert this tables to EXTERNAL_TABLE but with auto-purge
> enabled. For such tables, we should take the liberty to managed these
> locations as well to match the legacy MANAGED table behavior.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
