[kudu-CR] [java] Deflake TestKuduScanner.testDiffScan
Will Berkeley has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13495 ) Change subject: [java] Deflake TestKuduScanner.testDiffScan .. [java] Deflake TestKuduScanner.testDiffScan I noticed a funny failure in TestKuduScanner.testDiffScan: 21:55:41.681 [DEBUG - New I/O worker #13] (AsyncKuduScanner.java:556) Can not open scanner org.apache.kudu.client.NonRecoverableException: snapshot scan start timestamp is earlier than the ancient history mark. Consider increasing the value of the configuration parameter --tablet_history_max_age_sec. Snapshot timestamp: P: 0 usec, L: 0 Ancient History Mark: P: 1558907741677535 usec, L: 0 Physical time difference: -1558907741.678s The client logs the request it sent, which indeed has a 'snap_start_timestamp' of 0. This timestamp is supposed to come from incrementing the propagated timestamp received from the client after some random inserts and mutations. Fortunately, this test has nice logging: it logs the operations it did and the timestamp: 21:55:40.874 [INFO - Time-limited test] (TestKuduScanner.java:250) Before: {} 21:55:40.874 [INFO - Time-limited test] (TestKuduScanner.java:255) startHT: 0 'generateMutationOperations' could generate 0 mutations, causing this test to fail. This requires the random number generator to generate 0 3 times in a row when sampling uniformly from {0, 1, 2, 3, 4}, so the odds of failure due to this defect in the test were only 1/125. I fixed this by adjusting the number of initial inserts generated so it is always at least 1. I ran the test 1250 times with the fix and saw zero failures. Without the fix, I saw 9/1250 failures. Change-Id: Iadb46e5dae71724aa8ff88d04c40ef4eaf1ddd2a Reviewed-on: http://gerrit.cloudera.org:8080/13495 Tested-by: Will Berkeley Reviewed-by: Alexey Serbin --- M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java M java/kudu-test-utils/src/main/java/org/apache/kudu/test/RandomUtils.java 2 files changed, 32 insertions(+), 14 deletions(-) Approvals: Will Berkeley: Verified Alexey Serbin: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/13495 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Iadb46e5dae71724aa8ff88d04c40ef4eaf1ddd2a Gerrit-Change-Number: 13495 Gerrit-PatchSet: 3 Gerrit-Owner: Will Berkeley Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Will Berkeley
[kudu-CR] [java] Deflake TestKuduScanner.testDiffScan
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/13495 ) Change subject: [java] Deflake TestKuduScanner.testDiffScan .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13495 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iadb46e5dae71724aa8ff88d04c40ef4eaf1ddd2a Gerrit-Change-Number: 13495 Gerrit-PatchSet: 2 Gerrit-Owner: Will Berkeley Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Mon, 03 Jun 2019 06:54:52 + Gerrit-HasComments: No
[kudu-CR] [java] Deflake TestKuduScanner.testDiffScan
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/13495 ) Change subject: [java] Deflake TestKuduScanner.testDiffScan .. Patch Set 2: Verified+1 When will the flakes end :'( -- To view, visit http://gerrit.cloudera.org:8080/13495 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iadb46e5dae71724aa8ff88d04c40ef4eaf1ddd2a Gerrit-Change-Number: 13495 Gerrit-PatchSet: 2 Gerrit-Owner: Will Berkeley Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Mon, 03 Jun 2019 06:02:43 + Gerrit-HasComments: No
[kudu-CR] [java] Deflake TestKuduScanner.testDiffScan
Will Berkeley has removed a vote on this change. Change subject: [java] Deflake TestKuduScanner.testDiffScan .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/13495 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: Iadb46e5dae71724aa8ff88d04c40ef4eaf1ddd2a Gerrit-Change-Number: 13495 Gerrit-PatchSet: 2 Gerrit-Owner: Will Berkeley Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [java] Deflake TestKuduScanner.testDiffScan
Hello Alexey Serbin, Kudu Jenkins, Adar Dembo, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13495 to look at the new patch set (#2). Change subject: [java] Deflake TestKuduScanner.testDiffScan .. [java] Deflake TestKuduScanner.testDiffScan I noticed a funny failure in TestKuduScanner.testDiffScan: 21:55:41.681 [DEBUG - New I/O worker #13] (AsyncKuduScanner.java:556) Can not open scanner org.apache.kudu.client.NonRecoverableException: snapshot scan start timestamp is earlier than the ancient history mark. Consider increasing the value of the configuration parameter --tablet_history_max_age_sec. Snapshot timestamp: P: 0 usec, L: 0 Ancient History Mark: P: 1558907741677535 usec, L: 0 Physical time difference: -1558907741.678s The client logs the request it sent, which indeed has a 'snap_start_timestamp' of 0. This timestamp is supposed to come from incrementing the propagated timestamp received from the client after some random inserts and mutations. Fortunately, this test has nice logging: it logs the operations it did and the timestamp: 21:55:40.874 [INFO - Time-limited test] (TestKuduScanner.java:250) Before: {} 21:55:40.874 [INFO - Time-limited test] (TestKuduScanner.java:255) startHT: 0 'generateMutationOperations' could generate 0 mutations, causing this test to fail. This requires the random number generator to generate 0 3 times in a row when sampling uniformly from {0, 1, 2, 3, 4}, so the odds of failure due to this defect in the test were only 1/125. I fixed this by adjusting the number of initial inserts generated so it is always at least 1. I ran the test 1250 times with the fix and saw zero failures. Without the fix, I saw 9/1250 failures. Change-Id: Iadb46e5dae71724aa8ff88d04c40ef4eaf1ddd2a --- M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java M java/kudu-test-utils/src/main/java/org/apache/kudu/test/RandomUtils.java 2 files changed, 32 insertions(+), 14 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/95/13495/2 -- To view, visit http://gerrit.cloudera.org:8080/13495 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iadb46e5dae71724aa8ff88d04c40ef4eaf1ddd2a Gerrit-Change-Number: 13495 Gerrit-PatchSet: 2 Gerrit-Owner: Will Berkeley Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [java] Deflake TestKuduScanner.testDiffScan
Will Berkeley has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13495 Change subject: [java] Deflake TestKuduScanner.testDiffScan .. [java] Deflake TestKuduScanner.testDiffScan I noticed a funny failure in TestKuduScanner.testDiffScan: 21:55:41.681 [DEBUG - New I/O worker #13] (AsyncKuduScanner.java:556) Can not open scanner org.apache.kudu.client.NonRecoverableException: snapshot scan start timestamp is earlier than the ancient history mark. Consider increasing the value of the configuration parameter --tablet_history_max_age_sec. Snapshot timestamp: P: 0 usec, L: 0 Ancient History Mark: P: 1558907741677535 usec, L: 0 Physical time difference: -1558907741.678s The client logs the request it sent, which indeed has a 'snap_start_timestamp' of 0. This timestamp is supposed to come from incrementing the propagated timestamp received from the client after some random inserts and mutations. Fortunately, this test has nice logging: it logs the operations it did and the timestamp: 21:55:40.874 [INFO - Time-limited test] (TestKuduScanner.java:250) Before: {} 21:55:40.874 [INFO - Time-limited test] (TestKuduScanner.java:255) startHT: 0 'generateMutationOperations' could generate 0 mutations, causing this test to fail. This requires the random number generate to generate 0 3 times in a row when sample uniformly from {0, 1, 2, 3, 4}, so the odds of failure due to this defect in the tests were only 1/125. I fixed this by adjusting the number of inserts generated so it is always at least 1. I ran the test 1250 times with the fix and saw zero failures. Without the fix, I saw 9/1250 failures. Change-Id: Iadb46e5dae71724aa8ff88d04c40ef4eaf1ddd2a --- M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java M java/kudu-test-utils/src/main/java/org/apache/kudu/test/RandomUtils.java 2 files changed, 32 insertions(+), 14 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/95/13495/1 -- To view, visit http://gerrit.cloudera.org:8080/13495 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Iadb46e5dae71724aa8ff88d04c40ef4eaf1ddd2a Gerrit-Change-Number: 13495 Gerrit-PatchSet: 1 Gerrit-Owner: Will Berkeley