Hello Norbert Luksa, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/14444
to look at the new patch set (#4).
Change subject: IMPALA-8648: Add stress tests for ACID INSERTs/SELECTs
......................................................................
IMPALA-8648: Add stress tests for ACID INSERTs/SELECTs
Added different stress tests for ACID operations. We already had one
stress test where an Impala client issued SELECTs while another process
inserted data through Hive. The test checked wether some invariants are
held.
I added the following tests:
* one Impala client inserts data, one Impala client reads and checks
invariants
* multiple Impala clients issue INSERTs and TRUNCATEs, multiple Impala
clients issue SELECTs and check invariants
* multiple Impala clients insert data, sometimes failures are injected
during inserts to create aborted transactions. In the meantime
multiple Impala clients read the data and check that if they see
data from aborted transactions
During the tests I found and fixed a temporal deadlock.
CatalogOpExecutor.truncateTable() locked catalog_.versionLock_ and the
HMS ACID lock in a different order than the INSERT statements, it could
lead to a deadlock for 30 seconds, after that TRUNCATE failed.
Also, in the Frontend I switched the order of write id allocation and
locking. From now on at first we lock the table then allocate a write
id.
Change-Id: I066652bfa7d924742af01aef8df4512e00620c7d
---
M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/main/java/org/apache/impala/catalog/Transaction.java
M fe/src/main/java/org/apache/impala/common/TransactionKeepalive.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M tests/common/impala_test_suite.py
M tests/stress/test_acid_stress.py
7 files changed, 380 insertions(+), 104 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/44/14444/4
--
To view, visit http://gerrit.cloudera.org:8080/14444
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I066652bfa7d924742af01aef8df4512e00620c7d
Gerrit-Change-Number: 14444
Gerrit-PatchSet: 4
Gerrit-Owner: Zoltan Borok-Nagy <borokna...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Norbert Luksa <norbert.lu...@cloudera.com>
