[
https://issues.apache.org/jira/browse/HIVE-30234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-30234:
----------------------------------
Description:
HiveCatalog.renameTableOrView reads the HMS table, changes its db/name, and
writes it back with alter_table, which replaces all table parameters. It takes
neither the HMS lock nor the metadata_location compare-and-swap that
HiveTableOperations.doCommit uses. A commit that lands between the read and the
write is reverted: metadata_location goes back to the file the rename read, and
the new snapshot is lost.
Hive's ALTER TABLE ... RENAME on a HiveCatalog table goes through this path
(the meta hook sets SKIP_METASTORE_ALTER and delegates to
Catalogs.renameTable), as does a rename through the HMS Iceberg REST catalog.
Upstream Iceberg's HiveCatalog has the same code.
Fix: send the CAS on the metadata_location that was read
(expected_parameter_key / expected_parameter_value) with the alter, as
doCommit does without the lock. A table without metadata_location sends no CAS.
Reproducer: a commit between the rename's getTable and alterTable; after the
rename the table no longer has the commit's snapshot.
was:
HiveCatalog.renameTableOrView reads the HMS table, changes its db/name, and
writes it back with
alter_table, which replaces all table parameters. It takes neither the HMS
lock nor the
metadata_location compare-and-swap that HiveTableOperations.doCommit uses. A
commit that lands
between the read and the write is reverted: metadata_location goes back to
the file the rename
read, and the new snapshot is lost.
Hive's ALTER TABLE ... RENAME on a HiveCatalog table goes through this path
(the meta hook sets
SKIP_METASTORE_ALTER and delegates to Catalogs.renameTable), as does a rename
through the HMS
Iceberg REST catalog. Upstream Iceberg's HiveCatalog has the same code.
Fix: send the CAS on the metadata_location that was read
(expected_parameter_key /
expected_parameter_value) with the alter, as doCommit does without the lock.
A table without
metadata_location sends no CAS.
Reproducer: a commit between the rename's getTable and alterTable; after the
rename the table no
longer has the commit's snapshot.
> Iceberg: HiveCatalog rename can revert a concurrent commit's metadata_location
> ------------------------------------------------------------------------------
>
> Key: HIVE-30234
> URL: https://issues.apache.org/jira/browse/HIVE-30234
> Project: Hive
> Issue Type: Bug
> Components: Iceberg integration, Locking
> Affects Versions: 4.2.0, 4.2.1
> Reporter: Denys Kuzmenko
> Priority: Major
>
> HiveCatalog.renameTableOrView reads the HMS table, changes its db/name, and
> writes it back with alter_table, which replaces all table parameters. It
> takes neither the HMS lock nor the metadata_location compare-and-swap that
> HiveTableOperations.doCommit uses. A commit that lands between the read and
> the write is reverted: metadata_location goes back to the file the rename
> read, and the new snapshot is lost.
>
>
>
>
>
> Hive's ALTER TABLE ... RENAME on a HiveCatalog table goes through this path
> (the meta hook sets SKIP_METASTORE_ALTER and delegates to
> Catalogs.renameTable), as does a rename through the HMS Iceberg REST catalog.
> Upstream Iceberg's HiveCatalog has the same code.
>
>
>
>
>
> Fix: send the CAS on the metadata_location that was read
> (expected_parameter_key / expected_parameter_value) with the alter, as
> doCommit does without the lock. A table without metadata_location sends no
> CAS.
>
>
>
>
>
> Reproducer: a commit between the rename's getTable and alterTable; after the
> rename the table no longer has the commit's snapshot.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
