Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/11815 )
Change subject: KUDU-2584: Prevent flaky off-by-one errors in backup tests ...................................................................... Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/11815/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11815/1//COMMIT_MSG@13 PS1, Line 13: : The off-by-one errors are possibly due to very fast : test runs where the writes and backups start : in under 1ms. They could also be due to some : underlying issue that is not well understood. So I thought the underlying issue was due to a different in granularity: Kudu's HT timestamps are specified in micros while the default value for the HT timestamp to backup (System.currentTimeMillis) is specified in millis. The issue being: if the data inserted for the test completes at HT timestamp of "x millis + y micros", the use of currentTimeMillis configures the backup to read at HT timestamp of "x millis", which means the backup misses some of the inserts. After restoring this backup, the number of rows in the table doesn't match the number of rows originally inserted. If this is true, I'd like to see a few changes here: 1) Spell out the issue in detail, both in the commit message and in the corresponding workaround in the test code. 2) Work around the issue in a different way, by calling currentTimeMillis in the Spark job and passing that argument (plus 1 ms) in as the HT timestamp to use for the backup. It's not fundamentally different, but it doesn't slow down the test in the way that Thread.sleep does. -- To view, visit http://gerrit.cloudera.org:8080/11815 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia0f1b4a4138cc8c913543a68fad748927cdc439d Gerrit-Change-Number: 11815 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Hao Hao <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mike Percy <[email protected]> Gerrit-Reviewer: Will Berkeley <[email protected]> Gerrit-Comment-Date: Mon, 29 Oct 2018 18:53:55 +0000 Gerrit-HasComments: Yes
