[
https://issues.apache.org/jira/browse/HIVE-30234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-30234:
----------------------------------
Description:
HiveCatalog.renameTableOrView reads the HMS table, changes its db/name, and
writes it back with
alter_table, which replaces all table parameters. It takes neither the HMS
lock nor the
metadata_location compare-and-swap that HiveTableOperations.doCommit uses. A
commit that lands
between the read and the write is reverted: metadata_location goes back to
the file the rename
read, and the new snapshot is lost.
Hive's ALTER TABLE ... RENAME on a HiveCatalog table goes through this path
(the meta hook sets
SKIP_METASTORE_ALTER and delegates to Catalogs.renameTable), as does a rename
through the HMS
Iceberg REST catalog. Upstream Iceberg's HiveCatalog has the same code.
Fix: send the CAS on the metadata_location that was read
(expected_parameter_key /
expected_parameter_value) with the alter, as doCommit does without the lock.
A table without
metadata_location sends no CAS.
Reproducer: a commit between the rename's getTable and alterTable; after the
rename the table no
longer has the commit's snapshot.
> Iceberg: HiveCatalog rename can revert a concurrent commit's metadata_location
> ------------------------------------------------------------------------------
>
> Key: HIVE-30234
> URL: https://issues.apache.org/jira/browse/HIVE-30234
> Project: Hive
> Issue Type: Bug
> Reporter: Denys Kuzmenko
> Priority: Major
>
> HiveCatalog.renameTableOrView reads the HMS table, changes its db/name, and
> writes it back with
>
> alter_table, which replaces all table parameters. It takes neither the HMS
> lock nor the
>
> metadata_location compare-and-swap that HiveTableOperations.doCommit uses.
> A commit that lands
>
> between the read and the write is reverted: metadata_location goes back to
> the file the rename
>
> read, and the new snapshot is lost.
>
>
>
>
>
> Hive's ALTER TABLE ... RENAME on a HiveCatalog table goes through this path
> (the meta hook sets
>
> SKIP_METASTORE_ALTER and delegates to Catalogs.renameTable), as does a
> rename through the HMS
>
> Iceberg REST catalog. Upstream Iceberg's HiveCatalog has the same code.
>
>
>
>
>
> Fix: send the CAS on the metadata_location that was read
> (expected_parameter_key /
>
> expected_parameter_value) with the alter, as doCommit does without the
> lock. A table without
>
> metadata_location sends no CAS.
>
>
>
>
>
> Reproducer: a commit between the rename's getTable and alterTable; after
> the rename the table no
>
> longer has the commit's snapshot.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
