[
https://issues.apache.org/jira/browse/HIVE-30235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-30235:
----------------------------------
Description:
With hive.txn.ext.locking.enabled=true, HiveIcebergStorageHandler.getLockType
rejects DML on a table with
engine.hive.lock-enabled=true before any lock is taken.
AcidUtils.makeLockComponents doesn't consult the storage
handler for DDL_EXCLUSIVE, though, so DDL (ADD/REPLACE/CHANGE COLUMNS, UNSET
TBLPROPERTIES, TRUNCATE, DROP
PARTITION) takes its EXCLUSIVE Hive lock on db.table. The meta hook then
requests the Iceberg commit lock on the
same table (the same HMS resource since HIVE-30231) and waits on the
statement's own lock until
iceberg.hive.lock-timeout-ms (3 min), failing with "Timed out ... waiting for
lock on <db>.<table>". Without the
hook's lock, the Iceberg commit would wait the same way.
Proposed: have makeLockComponents ask the storage handler for DDL_EXCLUSIVE
of non-native tables too, so the DDL is
rejected like DML; handlers overriding getLockType (Iceberg, Kafka) return
EXCLUSIVE for it
was:
With hive.txn.ext.locking.enabled=true, DDL that takes a DDL_EXCLUSIVE Hive
lock (ADD/REPLACE/CHANGE COLUMNS,
UNSET TBLPROPERTIES, TRUNCATE, DROP PARTITION) on an Iceberg table with
engine.hive.lock-enabled=true holds an
EXCLUSIVE lock on db.table, and its Iceberg commit then requests an EXCLUSIVE
HMS lock on the same table (same
resource since HIVE-30231). The statement waits on itself until
iceberg.hive.lock-timeout-ms (3 min) and fails with
"LockException: Timed out ... waiting for lock on <db>.<table>".
DML on such tables is already rejected up front by
HiveIcebergStorageHandler.getLockType. Proposed: have
AcidUtils.makeLockComponents ask the storage handler for DDL_EXCLUSIVE of
non-native tables too, so the same check
rejects the DDL before any lock is taken; handlers overriding getLockType
(Iceberg, Kafka) return EXCLUSIVE for it.
> Iceberg: exclusive DDL on a lock-enabled table waits on its own Hive lock
> under external locking
> -------------------------------------------------------------------------------------------------
>
> Key: HIVE-30235
> URL: https://issues.apache.org/jira/browse/HIVE-30235
> Project: Hive
> Issue Type: Bug
> Components: Iceberg integration, Locking
> Affects Versions: 4.2.0, 4.2.1
> Reporter: Denys Kuzmenko
> Priority: Major
>
> With hive.txn.ext.locking.enabled=true,
> HiveIcebergStorageHandler.getLockType rejects DML on a table with
>
> engine.hive.lock-enabled=true before any lock is taken.
> AcidUtils.makeLockComponents doesn't consult the storage
>
> handler for DDL_EXCLUSIVE, though, so DDL (ADD/REPLACE/CHANGE COLUMNS,
> UNSET TBLPROPERTIES, TRUNCATE, DROP
>
> PARTITION) takes its EXCLUSIVE Hive lock on db.table. The meta hook then
> requests the Iceberg commit lock on the
>
> same table (the same HMS resource since HIVE-30231) and waits on the
> statement's own lock until
>
> iceberg.hive.lock-timeout-ms (3 min), failing with "Timed out ... waiting
> for lock on <db>.<table>". Without the
>
> hook's lock, the Iceberg commit would wait the same way.
>
>
>
>
>
> Proposed: have makeLockComponents ask the storage handler for DDL_EXCLUSIVE
> of non-native tables too, so the DDL is
>
> rejected like DML; handlers overriding getLockType (Iceberg, Kafka) return
> EXCLUSIVE for it
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
