Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/13968 )
Change subject: IMPALA-8637: Implement transaction handling and locking for ACID queries ...................................................................... Patch Set 7: (1 comment) http://gerrit.cloudera.org:8080/#/c/13968/7/tests/query_test/test_acid.py File tests/query_test/test_acid.py: http://gerrit.cloudera.org:8080/#/c/13968/7/tests/query_test/test_acid.py@124 PS7, Line 124: def _any_open_heartbeated_transaction(self): : hive_transactions = self.run_stmt_in_hive("SHOW TRANSACTIONS") : for transaction_line in hive_transactions.split('\n'): : transaction_columns = transaction_line.split(',') : if len(transaction_columns) < 4: : continue : transaction_state = transaction_columns[1] : transaction_start_time = transaction_columns[2] : transaction_last_heartbeat = transaction_columns[3] : if transaction_state == 'OPEN': : if transaction_start_time != transaction_last_heartbeat: : return True : return False My concern here is that as our experience is that HMS never aborts transactions due to missing heartbeats, so there can be quite old open transactions, and if there was ever any heartbeat sent for them, they will satisfy this condition. This seems possible to me for long running Hive inserts/compactions, so this function can return true even if Impala does not send heartbeats at all. I would do the following changes to make the test stronger: - get the count or set of open_heartbeated transacations instead of just checking their existence - make the test sequential - check the count/set before starting the query, and then expect it to be increased/have a new transaction at one point -- To view, visit http://gerrit.cloudera.org:8080/13968 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iaa37899b24aa114be642bf8772b4e0f882865cfa Gerrit-Change-Number: 13968 Gerrit-PatchSet: 7 Gerrit-Owner: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Gabor Kaszab <gaborkas...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Yongzhi Chen <yc...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Comment-Date: Wed, 07 Aug 2019 14:03:12 +0000 Gerrit-HasComments: Yes
