[ https://issues.apache.org/jira/browse/HIVE-26472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sherman updated HIVE-26472: -------------------------------- Priority: Critical (was: Major) > Concurrent UPDATEs can cause duplicate rows > ------------------------------------------- > > Key: HIVE-26472 > URL: https://issues.apache.org/jira/browse/HIVE-26472 > Project: Hive > Issue Type: Bug > Components: HiveServer2 > Affects Versions: 4.0.0-alpha-1 > Reporter: John Sherman > Priority: Critical > Attachments: debug.diff > > > Concurrent UPDATEs to the same table can cause duplicate rows when the > following occurs: > Two UPDATEs get assigned txnIds and writeIds like this: > UPDATE #1 = txnId: 100 writeId: 50 <--- commits first > UPDATE #2 = txnId: 101 writeId: 49 > To replicate the issue: > I applied the attach debug.diff patch which adds hive.lock.sleep.writeid > (which controls the amount to sleep before acquiring a writeId) and > hive.lock.sleep.post.writeid (which controls the amount to sleep after > acquiring a writeId). > {code:java} > CREATE TABLE test_update(i int) STORED AS ORC > TBLPROPERTIES('transactional'="true"); > INSERT INTO test_update VALUES (1); > Start two beeline connections. > In connection #1 - run: > set hive.driver.parallel.compilation = true; > set hive.lock.sleep.writeid=5s; > update test_update set i = 1 where i = 1; > Wait one second and in connection #2 - run: > set hive.driver.parallel.compilation = true; > set hive.lock.sleep.post.writeid=10s; > update test_update set i = 1 where i = 1; > After both updates complete - it is likely that test_update contains two rows > now. > {code} > HIVE-24211 seems to address the case when: > UPDATE #1 = txnId: 100 writeId: 50 > UPDATE #2 = txnId: 101 writeId: 49 <--- commits first (I think this causes > UPDATE #1 to detect the snapshot is out of date because commitedTxn > UPDATE > #1s txnId) > A possible work around is to set hive.driver.parallel.compilation = false, > but this would only help in cases there is only one HS2 instance. -- This message was sent by Atlassian Jira (v8.20.10#820010)