[
https://issues.apache.org/jira/browse/HIVE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nickSoul updated HIVE-19950:
----------------------------
Description:
Hi,
When using Streaming Mutation recently, I found LockComponents were not locked
correctly by current transaction. Below is my test case:
# Begin the transaction with a transactionId 126, and the transaction locks a
table. Then hangs the transaction. The lock information were correctly restored
in mariaDB
{code:java}
MariaDB [hive]> select
HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
from HIVE_LOCKS;
+----------------+----------------+----------+-----------+-----------+----
| HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE | HL_PARTITION
| HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT | HL_BLOCKEDBY_EXT_ID |
HL_BLOCKEDBY_INT_ID |
+----------------+----------------+----------+-----------+-----------+----
| 384 | 1 | 126 | test_acid | acid_test | NULL | a | w | 1529512857000 | NULL |
NULL |
+----------------+----------------+----------+-----------+-----------+----{code}
#
{code:java}
{code}
Begin the other transaction with a transactionId 127 before previous
transaction 126 finished. Transaction 127 tries to lock the same table too, but
failed at first attempt. The lock information were correctly restored in
mariaDB, Lock 385 was blocked by Lock 384.
{code:java}
MariaDB [hive]> select
HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
from HIVE_LOCKS;
+----------------+----------------+----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+
| HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE | HL_PARTITION
| HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT | HL_BLOCKEDBY_EXT_ID |
HL_BLOCKEDBY_INT_ID |
+----------------+----------------+----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+
| 384 | 1 | 126 | test_acid | acid_test | NULL | a | w | 1529512857000 | NULL |
NULL |
| 385 | 1 | 127 | test_acid | acid_test | NULL | w | w | NULL | 384 | 1 |
+----------------+----------------+----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+{code}
3. Then transaction 127 tries to lock the table for a second retry after
30s, this time it successfully locked the table, whereas transaction 126 is
still holding the lock. Lock informations in MetaStore DB:
{code:java}
MariaDB [hive]> select
HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
from HIVE_LOCKS;
+----------------+----------------+----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+
| HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE | HL_PARTITION
| HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT | HL_BLOCKEDBY_EXT_ID |
HL_BLOCKEDBY_INT_ID |
+----------------+----------------+----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+
| 384 | 1 | 126 | test_acid | acid_test | NULL | a | w | 1529512857000 | NULL |
NULL |
| 385 | 1 | 127 | test_acid | acid_test | NULL | w | w | NULL | 384 | 1 |
| 386 | 1 | 127 | test_acid | acid_test | NULL | a | w | 1529513069000 | NULL |
NULL |
+----------------+----------------+----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+
{code}
I wonder if i use it in a wrong way, or misunderstand sth about ACID in
hive.
After going through the code, If that is a problem, maybe it's better to
delete the lock information of a failed attempt in matastore DB ?
was:
Hi,
When using Streaming Mutation recently, I found LockComponents were not locked
correctly by current transaction. I wonder if i use in a wrong way or
misunderstand about ACID in hive. Below is my test case:
# Begin the transaction with a transactionId 126, and the transaction locks a
table. Then hangs the transaction. The lock information were correctly restored
in mariaDB
{code:java}
MariaDB [hive]> select
HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
from HIVE_LOCKS;
+----------------+----------------+----------+-----------+-----------+----
| HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE | HL_PARTITION
| HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT | HL_BLOCKEDBY_EXT_ID |
HL_BLOCKEDBY_INT_ID |
+----------------+----------------+----------+-----------+-----------+----
| 384 | 1 | 126 | test_acid | acid_test | NULL | a | w | 1529512857000 | NULL |
NULL |
+----------------+----------------+----------+-----------+-----------+----{code}
# Begin the other transaction with a transactionId 127 before previous
transaction 126 finished. Transaction 127 tries to lock the same table too, but
failed at first attempt.
{code:java}
//Log Information{code}
#
> Hive ACID NOT LOCK LockComponent Correctly
> ------------------------------------------
>
> Key: HIVE-19950
> URL: https://issues.apache.org/jira/browse/HIVE-19950
> Project: Hive
> Issue Type: Bug
> Affects Versions: 2.3.2
> Reporter: nickSoul
> Priority: Major
>
> Hi,
> When using Streaming Mutation recently, I found LockComponents were not
> locked correctly by current transaction. Below is my test case:
> # Begin the transaction with a transactionId 126, and the transaction locks
> a table. Then hangs the transaction. The lock information were correctly
> restored in mariaDB
> {code:java}
> MariaDB [hive]> select
> HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
> from HIVE_LOCKS;
> +----------------+----------------+----------+-----------+-----------+----
> | HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE |
> HL_PARTITION | HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT |
> HL_BLOCKEDBY_EXT_ID | HL_BLOCKEDBY_INT_ID |
> +----------------+----------------+----------+-----------+-----------+----
> | 384 | 1 | 126 | test_acid | acid_test | NULL | a | w | 1529512857000 | NULL
> | NULL |
> +----------------+----------------+----------+-----------+-----------+----{code}
> #
> {code:java}
> {code}
> Begin the other transaction with a transactionId 127 before previous
> transaction 126 finished. Transaction 127 tries to lock the same table too,
> but failed at first attempt. The lock information were correctly restored in
> mariaDB, Lock 385 was blocked by Lock 384.
> {code:java}
> MariaDB [hive]> select
> HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
> from HIVE_LOCKS;
> +----------------+----------------+----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+
> | HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE |
> HL_PARTITION | HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT |
> HL_BLOCKEDBY_EXT_ID | HL_BLOCKEDBY_INT_ID |
> +----------------+----------------+----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+
> | 384 | 1 | 126 | test_acid | acid_test | NULL | a | w | 1529512857000 | NULL
> | NULL |
> | 385 | 1 | 127 | test_acid | acid_test | NULL | w | w | NULL | 384 | 1 |
> +----------------+----------------+----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+{code}
> 3. Then transaction 127 tries to lock the table for a second retry after
> 30s, this time it successfully locked the table, whereas transaction 126 is
> still holding the lock. Lock informations in MetaStore DB:
>
> {code:java}
> MariaDB [hive]> select
> HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID
> from HIVE_LOCKS;
> +----------------+----------------+----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+
> | HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE |
> HL_PARTITION | HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT |
> HL_BLOCKEDBY_EXT_ID | HL_BLOCKEDBY_INT_ID |
> +----------------+----------------+----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+
> | 384 | 1 | 126 | test_acid | acid_test | NULL | a | w | 1529512857000 | NULL
> | NULL |
> | 385 | 1 | 127 | test_acid | acid_test | NULL | w | w | NULL | 384 | 1 |
> | 386 | 1 | 127 | test_acid | acid_test | NULL | a | w | 1529513069000 | NULL
> | NULL |
> +----------------+----------------+----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+
> {code}
>
> I wonder if i use it in a wrong way, or misunderstand sth about ACID in
> hive.
> After going through the code, If that is a problem, maybe it's better to
> delete the lock information of a failed attempt in matastore DB ?
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
