[
https://issues.apache.org/jira/browse/HIVE-26922?focusedWorklogId=839962&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839962
]
ASF GitHub Bot logged work on HIVE-26922:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 18/Jan/23 13:31
Start Date: 18/Jan/23 13:31
Worklog Time Spent: 10m
Work Description: kasakrisz commented on PR #3934:
URL: https://github.com/apache/hive/pull/3934#issuecomment-1387079817
Lock requests in the code:
1.
https://github.com/apache/hive/blob/55471330426c2e0a52101c2e535a66f751be76ee/ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java#L3029
2.
https://github.com/apache/hive/blob/55471330426c2e0a52101c2e535a66f751be76ee/iceberg/iceberg-catalog/src/main/java/org/apache/iceberg/hive/HiveCommitLock.java#L118
The first lock is acquired only if these setting are present
```
set hive.support.concurrency=true;
set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
```
These are required for native acid operations.
The deadlock happens in a few lines later in the 2. place: after the lock
request is issued we check periodically the state of the lock whether it is
acquired or not. Unfortunately in this use case it is always in a waiting state
since another exclusive write lock already acquired on the same table in the 1.
place and that one is released only when the current txn ends.
In this patch my goal was to copy the behavior from the insert case: let the
table's storage handler determine the type of lock in the 1. place.
https://github.com/apache/hive/blob/55471330426c2e0a52101c2e535a66f751be76ee/ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java#L3068
In case of Iceberg it is `SHARED_READ`. Iceberg uses optimistic concurrency
control so multiple transactions can write the same table parallel only the
Iceberg commit step is serialized using exclusive write locks.
Currently we support only full rebuild of Iceberg Materialized views which
is technically an insert overwrite. So a plain insert overwrite statement is
also affected.
Issue Time Tracking
-------------------
Worklog Id: (was: 839962)
Time Spent: 1h 50m (was: 1h 40m)
> Deadlock when rebuilding Materialized view stored by Iceberg
> ------------------------------------------------------------
>
> Key: HIVE-26922
> URL: https://issues.apache.org/jira/browse/HIVE-26922
> Project: Hive
> Issue Type: Bug
> Reporter: Krisztian Kasa
> Assignee: Krisztian Kasa
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> {code}
> create table tbl_ice(a int, b string, c int) stored by iceberg stored as orc
> tblproperties ('format-version'='1');
> insert into tbl_ice values (1, 'one', 50), (2, 'two', 51), (3, 'three', 52),
> (4, 'four', 53), (5, 'five', 54);
> create materialized view mat1 stored by iceberg stored as orc tblproperties
> ('format-version'='1') as
> select tbl_ice.b, tbl_ice.c from tbl_ice where tbl_ice.c > 52;
> insert into tbl_ice values (10, 'ten', 60);
> alter materialized view mat1 rebuild;
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
