[kudu-CR] [KUDU-2521] Java Implementation for BloomFilter
jinxing6...@126.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/11333 ) Change subject: [KUDU-2521] Java Implementation for BloomFilter .. Patch Set 8: Thanks Adar~ I updated, please take another look :) -- To view, visit http://gerrit.cloudera.org:8080/11333 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32673c008f9958088d281c2c198c543bfea2eb8e Gerrit-Change-Number: 11333 Gerrit-PatchSet: 8 Gerrit-Owner: jinxing6...@126.com Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Dan Burkert Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: jinxing6...@126.com Gerrit-Comment-Date: Fri, 14 Sep 2018 01:50:24 + Gerrit-HasComments: No
[kudu-CR] [KUDU-2521] Java Implementation for BloomFilter
Hello Dan Burkert, Kudu Jenkins, Adar Dembo, Grant Henke, Todd Lipcon, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11333 to look at the new patch set (#8). Change subject: [KUDU-2521] Java Implementation for BloomFilter .. [KUDU-2521] Java Implementation for BloomFilter Change-Id: I32673c008f9958088d281c2c198c543bfea2eb8e --- A java/kudu-client/src/main/java/org/apache/kudu/util/BloomFilter.java A java/kudu-client/src/test/java/org/apache/kudu/client/TestBloomFilter.java 2 files changed, 588 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/33/11333/8 -- To view, visit http://gerrit.cloudera.org:8080/11333 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I32673c008f9958088d281c2c198c543bfea2eb8e Gerrit-Change-Number: 11333 Gerrit-PatchSet: 8 Gerrit-Owner: jinxing6...@126.com Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Dan Burkert Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: jinxing6...@126.com
[kudu-CR](gh-pages) Blogpost describing index skip scan optimization.
Hello Alexey Serbin, Mike Percy, Attila Bukor, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11263 to look at the new patch set (#8). Change subject: Blogpost describing index skip scan optimization. .. Blogpost describing index skip scan optimization. Link to the version with images: https://github.com/AnupamaGupta01/kudu/blob/blogpost-2/_posts/2018-08-17-index-skip-scan-optimization-in-kudu.md Change-Id: I2250652dcba3d1b0a06f1ffb7f23c11bf533d35e --- A _posts/2018-08-17-index-skip-scan-optimization-in-kudu.md A img/index-skip-scan/example-table.png A img/index-skip-scan/skip-scan-example-table.png A img/index-skip-scan/skip-scan-performance-graph.png 4 files changed, 113 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/63/11263/8 -- To view, visit http://gerrit.cloudera.org:8080/11263 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2250652dcba3d1b0a06f1ffb7f23c11bf533d35e Gerrit-Change-Number: 11263 Gerrit-PatchSet: 8 Gerrit-Owner: Anupama Gupta Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Anupama Gupta Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Mike Percy
[kudu-CR](gh-pages) Blogpost describing index skip scan optimization.
Anupama Gupta has posted comments on this change. ( http://gerrit.cloudera.org:8080/11263 ) Change subject: Blogpost describing index skip scan optimization. .. Patch Set 8: (7 comments) http://gerrit.cloudera.org:8080/#/c/11263/7/_posts/2018-08-17-index-skip-scan-optimization-in-kudu.md File _posts/2018-08-17-index-skip-scan-optimization-in-kudu.md: http://gerrit.cloudera.org:8080/#/c/11263/7/_posts/2018-08-17-index-skip-scan-optimization-in-kudu.md@7 PS7, Line 7: team > nit: lower-case "team" Done http://gerrit.cloudera.org:8080/#/c/11263/7/_posts/2018-08-17-index-skip-scan-optimization-in-kudu.md@27 PS7, Line 27: table > nit: lower-case "table" Done http://gerrit.cloudera.org:8080/#/c/11263/7/_posts/2018-08-17-index-skip-scan-optimization-in-kudu.md@45 PS7, Line 45: . We will refer to it as the : "prefix column" and its specific value as the "prefix k > nit: drop the parens and start a new sentence instead. Done http://gerrit.cloudera.org:8080/#/c/11263/7/_posts/2018-08-17-index-skip-scan-optimization-in-kudu.md@47 PS7, Line 47: > nit: remove comma, maybe replace with "that" Done http://gerrit.cloudera.org:8080/#/c/11263/7/_posts/2018-08-17-index-skip-scan-optimization-in-kudu.md@82 PS7, Line 82: : Conclusion : == > No where does this mention that, as implemented, this works for equality pr Done http://gerrit.cloudera.org:8080/#/c/11263/7/_posts/2018-08-17-index-skip-scan-optimization-in-kudu.md@98 PS7, Line 98: roughly enjo > nit: full-fledged Done http://gerrit.cloudera.org:8080/#/c/11263/7/_posts/2018-08-17-index-skip-scan-optimization-in-kudu.md@99 PS7, Line 99: right from underst > nit: "of the skip scan approach" or "of the skip scan optimization" Done -- To view, visit http://gerrit.cloudera.org:8080/11263 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I2250652dcba3d1b0a06f1ffb7f23c11bf533d35e Gerrit-Change-Number: 11263 Gerrit-PatchSet: 8 Gerrit-Owner: Anupama Gupta Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Anupama Gupta Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Mike Percy Gerrit-Comment-Date: Thu, 13 Sep 2018 22:16:44 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-2095: [java] Add scanner keepAlive API to the Java client
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/11436 ) Change subject: KUDU-2095: [java] Add scanner keepAlive API to the Java client .. Patch Set 3: Code-Review+2 Though please loop this on dist-test in TSAN; curious if the new tests are robust. -- To view, visit http://gerrit.cloudera.org:8080/11436 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic802f556c8860cdd43ef5f794c8f3658259bd0be Gerrit-Change-Number: 11436 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Comment-Date: Thu, 13 Sep 2018 19:36:08 + Gerrit-HasComments: No
[kudu-CR] KUDU-2095: [java] Add scanner keepAlive API to the Java client
Adar Dembo has posted comments on this change. (
http://gerrit.cloudera.org:8080/11436 )
Change subject: KUDU-2095: [java] Add scanner keepAlive API to the Java client
..
Patch Set 2:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java
File java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java:
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java@85
PS1, Line 85: if ("testKeepAlive".equals(testName.getMethodName()) ||
: "testScannerExpiration".equals(testName.getMethodName())
: )
> I think as long as the result is a broken test we should be okay in the mea
In this case it does look like the tests would fail, but in the general case,
tests might just become more flaky, so I think it's important for us to get
this right.
Handling it in a follow-up is fine.
http://gerrit.cloudera.org:8080/#/c/11436/2/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java
File java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java:
http://gerrit.cloudera.org:8080/#/c/11436/2/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java@66
PS2, Line 66: import org.junit.rules.TestName;
Why did this move?
--
To view, visit http://gerrit.cloudera.org:8080/11436
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic802f556c8860cdd43ef5f794c8f3658259bd0be
Gerrit-Change-Number: 11436
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke
Gerrit-Reviewer: Adar Dembo
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy
Gerrit-Comment-Date: Thu, 13 Sep 2018 19:27:17 +
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2095: [java] Add scanner keepAlive API to the Java client
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/11436 ) Change subject: KUDU-2095: [java] Add scanner keepAlive API to the Java client .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/11436/2/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java File java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java: http://gerrit.cloudera.org:8080/#/c/11436/2/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java@66 PS2, Line 66: import org.junit.rules.TestName; > Why did this move? oops. A mistake from experimenting with other options. -- To view, visit http://gerrit.cloudera.org:8080/11436 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic802f556c8860cdd43ef5f794c8f3658259bd0be Gerrit-Change-Number: 11436 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Comment-Date: Thu, 13 Sep 2018 19:31:00 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-2095: [java] Add scanner keepAlive API to the Java client
Hello Mike Percy, Kudu Jenkins, Adar Dembo, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11436 to look at the new patch set (#3). Change subject: KUDU-2095: [java] Add scanner keepAlive API to the Java client .. KUDU-2095: [java] Add scanner keepAlive API to the Java client This patch adds keepAlive methods to the AsyncKuduScanner and KuduScanner. These methods leverage a package private method added to the AsyncKuduClient using a similar implementation pattern to existing scan related RPCs. The behavior of this implementation mimics the C++ client. Change-Id: Ic802f556c8860cdd43ef5f794c8f3658259bd0be --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduScanner.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduScanner.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestRemoteTablet.java M src/kudu/client/client.h 6 files changed, 347 insertions(+), 63 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/36/11436/3 -- To view, visit http://gerrit.cloudera.org:8080/11436 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic802f556c8860cdd43ef5f794c8f3658259bd0be Gerrit-Change-Number: 11436 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy
[kudu-CR] KUDU-2095: [java] Add scanner keepAlive API to the Java client
Grant Henke has posted comments on this change. (
http://gerrit.cloudera.org:8080/11436 )
Change subject: KUDU-2095: [java] Add scanner keepAlive API to the Java client
..
Patch Set 2:
(10 comments)
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java:
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java@1046
PS1, Line 1046: final ServerInfo info =
tablet.getReplicaSelectedServerInfo(keepAliveRequest.getReplicaSelection());
> Is it possible for 'info' to be null? Seems like closeScanner handles that
It really shouldn't be possible given a scan would need to have succeeded. And
if a scan succeeded we already have looked up the ServerInfo for that remote
tablet.
I think we can handle this the same way closeScanner does and result in a no-op.
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduScanner.java
File
java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduScanner.java:
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduScanner.java@717
PS1, Line 717:*/
> Need a @return for the Deferred.
Done
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduScanner.java@824
PS1, Line 824:
> Short Javadoc.
Done
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java
File java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java:
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java@85
PS1, Line 85: if ("testKeepAlive".equals(testName.getMethodName()) ||
: "testScannerExpiration".equals(testName.getMethodName())
: )
> Is there a more robust way to do this, such that renaming one of these test
I think as long as the result is a broken test we should be okay in the mean
time. I added comments above each test method to ensure it is clear that the
name is significant.
As a follow up, it looks like there is a way I could write an annotation that
adds flags to the mini-cluster on a per-test basis. I will look to do that as a
follow up change.
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java@186
PS1, Line 186:*/
> Is this long enough in TSAN environments? Could you loop your new tests in
This was an accident from experimentation. I will set to match all the other
tests. We cam look to use a global setting in a follow up patch.
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java@201
PS1, Line 201: }
> Isn't the default flush mode AUTO_FLUSH_SYNC, thus obviating the need for t
Done
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java@212
PS1, Line 212: // Wait for the scanner to time out.
: Thread.sleep(SHORT_SCANNER_TTL_MS * 2);
:
: try {
: scanner.nextRows();
: fail("Exception was not thrown when accessing an expired
scanner");
: } catch (NonRecoverableException ex) {
: assertThat(ex.getMessage(), containsString("Scanner not
found"));
: }
> Would be nice to wrap this up in an equivalent to C++'s AssertEventually. E
We do have AssertHelpers.assertEventuallyTrue. The problem is if we call
scanner.nextRows() too many times we will run out of rows.
I would also need to write a version thats "assertEventuallyException" I think.
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java@246
PS1, Line 246: KuduScanner scanner = new
KuduScanner.KuduScannerBuilder(client, table)
> Nit: move this to L255.
Done
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java@265
PS1, Line 265: if (accum == numRows) {
> tablets
Done
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java@281
PS1, Line 281: scanner.keepAlive();
> It's possible for poor scheduling to lead to a full TTL of elapsed time bet
Done
--
To view, visit http://gerrit.cloudera.org:8080/11436
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic802f556c8860cdd43ef5f794c8f3658259bd0be
Gerrit-Change-Number: 11436
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke
[kudu-CR] KUDU-2095: [java] Add scanner keepAlive API to the Java client
Hello Mike Percy, Kudu Jenkins, Adar Dembo, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11436 to look at the new patch set (#2). Change subject: KUDU-2095: [java] Add scanner keepAlive API to the Java client .. KUDU-2095: [java] Add scanner keepAlive API to the Java client This patch adds keepAlive methods to the AsyncKuduScanner and KuduScanner. These methods leverage a package private method added to the AsyncKuduClient using a similar implementation pattern to existing scan related RPCs. The behavior of this implementation mimics the C++ client. Change-Id: Ic802f556c8860cdd43ef5f794c8f3658259bd0be --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduScanner.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduScanner.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestRemoteTablet.java M src/kudu/client/client.h 6 files changed, 347 insertions(+), 63 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/36/11436/2 -- To view, visit http://gerrit.cloudera.org:8080/11436 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic802f556c8860cdd43ef5f794c8f3658259bd0be Gerrit-Change-Number: 11436 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy
[kudu-CR] [docs] Add basic advice on setting block cache size
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/11420 ) Change subject: [docs] Add basic advice on setting block cache size .. Patch Set 3: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/11420 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idc7411c38b6fcc8694509ec89c32e2fe74e6c0db Gerrit-Change-Number: 11420 Gerrit-PatchSet: 3 Gerrit-Owner: Will Berkeley Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Thu, 13 Sep 2018 19:03:31 + Gerrit-HasComments: No
[kudu-CR] [docs] Add basic advice on setting block cache size
Hello Alexey Serbin, Attila Bukor, Kudu Jenkins, Andrew Wong, Adar Dembo, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11420 to look at the new patch set (#3). Change subject: [docs] Add basic advice on setting block cache size .. [docs] Add basic advice on setting block cache size This adds a short section to the troubleshooting guide about improving the performance of the block cache. It's fuzzy since the effectiveness of the cache and the efficacy of enlarging it are so workload dependent (e.g. consider a workload doing full table scans vs. one mostly re-scanning a small range checking for updates), but I tried to provide a starting point for users to evaluate their cache size since we've totally lacked any advice on that up to this point. I also added information about the change due to release in 1.8 that servers won't start when the block cache capacity is set too large relative to the memory limit. Change-Id: Idc7411c38b6fcc8694509ec89c32e2fe74e6c0db --- M docs/troubleshooting.adoc 1 file changed, 78 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/20/11420/3 -- To view, visit http://gerrit.cloudera.org:8080/11420 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Idc7411c38b6fcc8694509ec89c32e2fe74e6c0db Gerrit-Change-Number: 11420 Gerrit-PatchSet: 3 Gerrit-Owner: Will Berkeley Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Will Berkeley
[kudu-CR] [docs] Add basic advice on setting block cache size
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/11420 ) Change subject: [docs] Add basic advice on setting block cache size .. Patch Set 2: (5 comments) http://gerrit.cloudera.org:8080/#/c/11420/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11420/1//COMMIT_MSG@11 PS1, Line 11: are so : workload dependent (e.g. consider a workload doing full table scans vs. : one mostly re-scanning a small rang > I was just letting you know. To the extent that it affects this patch...you (thumbsup) http://gerrit.cloudera.org:8080/#/c/11420/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11420/2//COMMIT_MSG@10 PS2, Line 10: the performance of the block cache size > This still doesn't make sense; what is the "performance of the block cache Oops. http://gerrit.cloudera.org:8080/#/c/11420/1/docs/troubleshooting.adoc File docs/troubleshooting.adoc: http://gerrit.cloudera.org:8080/#/c/11420/1/docs/troubleshooting.adoc@557 PS1, Line 557: metrics > I was confused about whether there was some other endpoint I didn't know ab (thumbsup) http://gerrit.cloudera.org:8080/#/c/11420/1/docs/troubleshooting.adoc@602 PS1, Line 602: Kudu expected to read from cache but which weren't found in the cache. If a > Ah, I misinterpreted the above definition of `block_cache_misses_caching` a :/ That's the correct interpretation...we just expect to find every block in the cache always unless the cache is off, I think :). Maybe that would change, but phrased this way the docs are relatively future-proofed, even if the part about checking eviction vs insertion numbers becomes redundant. http://gerrit.cloudera.org:8080/#/c/11420/2/docs/troubleshooting.adoc File docs/troubleshooting.adoc: http://gerrit.cloudera.org:8080/#/c/11420/2/docs/troubleshooting.adoc@596 PS2, Line 596: on a tablet server > Doesn't the same guidance apply to the masters as well? Though I guess it's It sure does, but if you are struggling with cache performance on your masters you are in a realm of Kudu scalability where we don't know how to help you. -- To view, visit http://gerrit.cloudera.org:8080/11420 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idc7411c38b6fcc8694509ec89c32e2fe74e6c0db Gerrit-Change-Number: 11420 Gerrit-PatchSet: 2 Gerrit-Owner: Will Berkeley Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Thu, 13 Sep 2018 18:56:35 + Gerrit-HasComments: Yes
[kudu-CR] [docs] Add basic advice on setting block cache size
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/11420 ) Change subject: [docs] Add basic advice on setting block cache size .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/11420/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11420/1//COMMIT_MSG@11 PS1, Line 11: are so : workload dependent (e.g. consider a workload doing full table scans vs. : one mostly re-scanning a small rang > In the commit message, in the docs, or you are just letting me know? (Thank I was just letting you know. To the extent that it affects this patch...you could argue that a well-written full table scan will NOT cache blocks and thus have no effect on the block cache. But that's not a given, and I don't even know if Impala uses that API. http://gerrit.cloudera.org:8080/#/c/11420/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11420/2//COMMIT_MSG@10 PS2, Line 10: the performance of the block cache size This still doesn't make sense; what is the "performance of the block cache size"? Did you mean "the performance of the block cache"? -- To view, visit http://gerrit.cloudera.org:8080/11420 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idc7411c38b6fcc8694509ec89c32e2fe74e6c0db Gerrit-Change-Number: 11420 Gerrit-PatchSet: 2 Gerrit-Owner: Will Berkeley Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Thu, 13 Sep 2018 18:47:07 + Gerrit-HasComments: Yes
[kudu-CR] [docs] Add basic advice on setting block cache size
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11420 ) Change subject: [docs] Add basic advice on setting block cache size .. Patch Set 2: (3 comments) http://gerrit.cloudera.org:8080/#/c/11420/1/docs/troubleshooting.adoc File docs/troubleshooting.adoc: http://gerrit.cloudera.org:8080/#/c/11420/1/docs/troubleshooting.adoc@557 PS1, Line 557: metrics > Oops. Tangentially, why comment as a question if you know the text is wrong I was confused about whether there was some other endpoint I didn't know about :) http://gerrit.cloudera.org:8080/#/c/11420/1/docs/troubleshooting.adoc@602 PS1, Line 602: Kudu expected to read from cache but which weren't found in the cache. If a > Looking at how the metrics behave on my local cluster, loading a block fres Ah, I misinterpreted the above definition of `block_cache_misses_caching` as "the number of blocks Kudu expected to read from cache because it put it there, but which weren't found" i.e. those that were evicted from the cache that we maybe shouldn't have evicted. Maybe reword the above definition to just "the number of blocks that Kudu tried to read from the cache but weren't found" or something? http://gerrit.cloudera.org:8080/#/c/11420/2/docs/troubleshooting.adoc File docs/troubleshooting.adoc: http://gerrit.cloudera.org:8080/#/c/11420/2/docs/troubleshooting.adoc@596 PS2, Line 596: on a tablet server Doesn't the same guidance apply to the masters as well? Though I guess it's not as important given how little activity there is on the masters. -- To view, visit http://gerrit.cloudera.org:8080/11420 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idc7411c38b6fcc8694509ec89c32e2fe74e6c0db Gerrit-Change-Number: 11420 Gerrit-PatchSet: 2 Gerrit-Owner: Will Berkeley Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Thu, 13 Sep 2018 18:34:12 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-2095: [java] Add scanner keepAlive API to the Java client
Adar Dembo has posted comments on this change. (
http://gerrit.cloudera.org:8080/11436 )
Change subject: KUDU-2095: [java] Add scanner keepAlive API to the Java client
..
Patch Set 1:
(10 comments)
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java:
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java@1046
PS1, Line 1046: final ServerInfo info =
tablet.getReplicaSelectedServerInfo(keepAliveRequest.getReplicaSelection());
Is it possible for 'info' to be null? Seems like closeScanner handles that case.
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduScanner.java
File
java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduScanner.java:
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduScanner.java@717
PS1, Line 717: public Deferred keepAlive() {
Need a @return for the Deferred.
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduScanner.java@824
PS1, Line 824: final class KeepAliveRequest extends KuduRpc {
Short Javadoc.
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java
File java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java:
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java@85
PS1, Line 85: if ("testKeepAlive".equals(testName.getMethodName()) ||
: "testScannerExpiration".equals(testName.getMethodName())
: )
Is there a more robust way to do this, such that renaming one of these tests
won't cause this code to be skipped? Perhaps we could annotate the two tests
and look for the corresponding annotation here?
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java@186
PS1, Line 186: @Test(timeout = 1)
Is this long enough in TSAN environments? Could you loop your new tests in
dist-test?
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java@201
PS1, Line 201: session.flush();
Isn't the default flush mode AUTO_FLUSH_SYNC, thus obviating the need for this?
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java@212
PS1, Line 212: // Wait for the scanner to time out.
: Thread.sleep(SHORT_SCANNER_TTL_MS * 2);
:
: try {
: scanner.nextRows();
: fail("Exception was not thrown when accessing an expired
scanner");
: } catch (NonRecoverableException ex) {
: assertThat(ex.getMessage(), containsString("Scanner not
found"));
: }
Would be nice to wrap this up in an equivalent to C++'s AssertEventually. Even
though the test waits twice as long as the TTL, it's possible (especially in
TSAN environments) for the scanner GC thread to not yet be scheduled, and for
the test to fail.
Anyway, if you can jury-rig an AssertEventually, you could omit the
Thread.sleep() altogether.
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java@246
PS1, Line 246: int accum = 0;
Nit: move this to L255.
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java@265
PS1, Line 265: // Ensure we actually end up between tablet.
tablets
http://gerrit.cloudera.org:8080/#/c/11436/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java@281
PS1, Line 281: Thread.sleep(SHORT_SCANNER_TTL_MS / 2);
It's possible for poor scheduling to lead to a full TTL of elapsed time between
keep alive requests. Consider raising the TTL to more than 1s for this test,
sleeping for less time, and looping for more iterations.
Loop in dist-test with TSAN binaries to see how robust it is.
--
To view, visit http://gerrit.cloudera.org:8080/11436
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic802f556c8860cdd43ef5f794c8f3658259bd0be
Gerrit-Change-Number: 11436
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke
Gerrit-Reviewer: Adar Dembo
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy
Gerrit-Comment-Date: Thu, 13 Sep 2018 18:29:11 +
Gerrit-HasComments: Yes
[kudu-CR] Add helper macro for tool invocations
Will Berkeley has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11411 ) Change subject: Add helper macro for tool invocations .. Add helper macro for tool invocations Using `ASSERT_OK` to test the results of the `kudu` tool is normal, but it results in lousy test failure output: ../../src/kudu/tools/kudu-admin-test.cc:235: Failure Failed Bad status: Runtime error: /Users/wdberkeley/src/kudu/build/debug/bin/kudu: process exited with non-zero status 1 This adds a new macro, `ASSERT_TOOL_OK`, that also logs the stdout and stderr of a `kudu` tool invocation: ../../src/kudu/tools/kudu-admin-test.cc:235: Failure Failed Runtime error: /Users/wdberkeley/src/kudu/build/debug/bin/kudu: process exited with non-zero status 1 stdout: stderr: W0910 12:39:07.483736 2830984064 flags.cc:406] Enabled unsafe flag: --never_fsync=true Invalid argument: Unrecognized peer type: FOOVOTER Change-Id: I7ffd357d79982ee5c93f8d3c7cfd7cc1f0863f07 Reviewed-on: http://gerrit.cloudera.org:8080/11411 Reviewed-by: Alexey Serbin Tested-by: Kudu Jenkins --- M src/kudu/tools/kudu-admin-test.cc 1 file changed, 47 insertions(+), 32 deletions(-) Approvals: Alexey Serbin: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/11411 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I7ffd357d79982ee5c93f8d3c7cfd7cc1f0863f07 Gerrit-Change-Number: 11411 Gerrit-PatchSet: 5 Gerrit-Owner: Will Berkeley Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Will Berkeley
[kudu-CR] [docs] Add basic advice on setting block cache size
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/11420 ) Change subject: [docs] Add basic advice on setting block cache size .. Patch Set 2: (8 comments) http://gerrit.cloudera.org:8080/#/c/11420/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11420/1//COMMIT_MSG@9 PS1, Line 9: improving > Nit: "improving" is an odd choice here, since the subject is "the block cac Done http://gerrit.cloudera.org:8080/#/c/11420/1//COMMIT_MSG@11 PS1, Line 11: are so : workload dependent (e.g. consider a workload doing full table scans vs. : one mostly re-scanning a small rang > Worth noting that the scanner API has a SetCacheBlocks() method to control In the commit message, in the docs, or you are just letting me know? (Thanks, btw, I knew we didn't cache blocks when doing a checksum but I didn't know we had an scanner API for the same thing.) http://gerrit.cloudera.org:8080/#/c/11420/1/docs/troubleshooting.adoc File docs/troubleshooting.adoc: http://gerrit.cloudera.org:8080/#/c/11420/1/docs/troubleshooting.adoc@538 PS1, Line 538: _cache_capacity_mb` above the memory pressure > Hrm, at first glance I read this as either: I removed this clause and joined the first part of this sentence to the second instead. http://gerrit.cloudera.org:8080/#/c/11420/1/docs/troubleshooting.adoc@542 PS1, Line 542: re threshold is recommended. With the defaults, thi > Nit: clearer as "`--memory_pressure_percentage` of `--memory_limit_hard_byt Done (but I also rewrote the surrounding text a little) http://gerrit.cloudera.org:8080/#/c/11420/1/docs/troubleshooting.adoc@543 PS1, Line 543: `--block_cache_capacity_mb` should not exceed 30% of : `--memory_limit_hard_bytes`. On Ku > Nit: wouldn't it be clearer to use "block cache capacity" and "memory hard No, I think this is clearest when referring the concrete values of the gflags. I rewrote the text before this sentence a little, which may have helped. http://gerrit.cloudera.org:8080/#/c/11420/1/docs/troubleshooting.adoc@557 PS1, Line 557: metrics > metrics? Oops. Tangentially, why comment as a question if you know the text is wrong? http://gerrit.cloudera.org:8080/#/c/11420/1/docs/troubleshooting.adoc@596 PS1, Line 596: first wait until > Won't block_cache_usage be 100% of block_cache_capacity_mb given enough upt Right, and we don't want users to extrapolate steady-state cache behavior from that case. I rewrote the section to emphasize users should look at the cache once the server has been running for a while, when it's expected that the cache is full. http://gerrit.cloudera.org:8080/#/c/11420/1/docs/troubleshooting.adoc@602 PS1, Line 602: Kudu expected to read from cache but which weren't found in the cache. If a > I'm trying to think of a case where this isn't true, but the above is true. Looking at how the metrics behave on my local cluster, loading a block fresh counts as a miss, so the initial load of N blocks counts as N misses. Ergo we need to see evictions to confirm there is churn because pure miss numbers aren't sufficient. -- To view, visit http://gerrit.cloudera.org:8080/11420 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idc7411c38b6fcc8694509ec89c32e2fe74e6c0db Gerrit-Change-Number: 11420 Gerrit-PatchSet: 2 Gerrit-Owner: Will Berkeley Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Thu, 13 Sep 2018 18:01:59 + Gerrit-HasComments: Yes
[kudu-CR](gh-pages) Blogpost describing index skip scan optimization.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11263 ) Change subject: Blogpost describing index skip scan optimization. .. Patch Set 7: Code-Review+1 (7 comments) Some tiny nits and one real suggestion, but otherwise LGTM. http://gerrit.cloudera.org:8080/#/c/11263/7/_posts/2018-08-17-index-skip-scan-optimization-in-kudu.md File _posts/2018-08-17-index-skip-scan-optimization-in-kudu.md: http://gerrit.cloudera.org:8080/#/c/11263/7/_posts/2018-08-17-index-skip-scan-optimization-in-kudu.md@7 PS7, Line 7: Team nit: lower-case "team" http://gerrit.cloudera.org:8080/#/c/11263/7/_posts/2018-08-17-index-skip-scan-optimization-in-kudu.md@27 PS7, Line 27: Table nit: lower-case "table" http://gerrit.cloudera.org:8080/#/c/11263/7/_posts/2018-08-17-index-skip-scan-optimization-in-kudu.md@45 PS7, Line 45: (we will refer to it as : "prefix column" and its specific value as "prefix key") nit: drop the parens and start a new sentence instead. Also nit: "as the 'prefix column' and its specific value as the 'prefix key.'" http://gerrit.cloudera.org:8080/#/c/11263/7/_posts/2018-08-17-index-skip-scan-optimization-in-kudu.md@47 PS7, Line 47: , nit: remove comma, maybe replace with "that" http://gerrit.cloudera.org:8080/#/c/11263/7/_posts/2018-08-17-index-skip-scan-optimization-in-kudu.md@82 PS7, Line 82: : Conclusion : == No where does this mention that, as implemented, this works for equality predicates. Should probably mention that. http://gerrit.cloudera.org:8080/#/c/11263/7/_posts/2018-08-17-index-skip-scan-optimization-in-kudu.md@98 PS7, Line 98: full fledged nit: full-fledged http://gerrit.cloudera.org:8080/#/c/11263/7/_posts/2018-08-17-index-skip-scan-optimization-in-kudu.md@99 PS7, Line 99: skip scan approach nit: "of the skip scan approach" or "of the skip scan optimization" -- To view, visit http://gerrit.cloudera.org:8080/11263 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I2250652dcba3d1b0a06f1ffb7f23c11bf533d35e Gerrit-Change-Number: 11263 Gerrit-PatchSet: 7 Gerrit-Owner: Anupama Gupta Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Anupama Gupta Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Mike Percy Gerrit-Comment-Date: Thu, 13 Sep 2018 17:57:03 + Gerrit-HasComments: Yes
[kudu-CR] [KUDU-2521] Java Implementation for BloomFilter
Adar Dembo has posted comments on this change. (
http://gerrit.cloudera.org:8080/11333 )
Change subject: [KUDU-2521] Java Implementation for BloomFilter
..
Patch Set 7:
(4 comments)
Almost there, just a few more stylistic comments.
http://gerrit.cloudera.org:8080/#/c/11333/6/java/kudu-client/src/main/java/org/apache/kudu/util/BloomFilter.java
File java/kudu-client/src/main/java/org/apache/kudu/util/BloomFilter.java:
http://gerrit.cloudera.org:8080/#/c/11333/6/java/kudu-client/src/main/java/org/apache/kudu/util/BloomFilter.java@73
PS6, Line 73: this.nBits = bitSet.size();
> it's:
Okay, that's constant time, so can you drop nBits and call bitSet.size() as
needed?
http://gerrit.cloudera.org:8080/#/c/11333/7/java/kudu-client/src/main/java/org/apache/kudu/util/BloomFilter.java
File java/kudu-client/src/main/java/org/apache/kudu/util/BloomFilter.java:
http://gerrit.cloudera.org:8080/#/c/11333/7/java/kudu-client/src/main/java/org/apache/kudu/util/BloomFilter.java@71
PS7, Line 71: bitSet.size() > 8
Should be >=, right?
http://gerrit.cloudera.org:8080/#/c/11333/7/java/kudu-client/src/main/java/org/apache/kudu/util/BloomFilter.java@84
PS7, Line 84: public static BloomFilter BySize(int nBytes) {
In our Java code we use lowerCamelCase to name functions and variables. So
"BySize" should actually be "bySize", and "BySizeAndFPRate" should be
"bySizeAndFPRate". Etc.
http://gerrit.cloudera.org:8080/#/c/11333/7/java/kudu-client/src/main/java/org/apache/kudu/util/BloomFilter.java@92
PS7, Line 92:* ever been {@code put} into the {@code
BloomFilter}.
For @params that span multiple lines, please align the continuation lines with
the first letter of the explanation on the first line:
@param fpRate the probability ...
ever been put into the ...
That's the Javadoc style we follow.
Below too.
--
To view, visit http://gerrit.cloudera.org:8080/11333
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32673c008f9958088d281c2c198c543bfea2eb8e
Gerrit-Change-Number: 11333
Gerrit-PatchSet: 7
Gerrit-Owner: jinxing6...@126.com
Gerrit-Reviewer: Adar Dembo
Gerrit-Reviewer: Dan Burkert
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon
Gerrit-Reviewer: jinxing6...@126.com
Gerrit-Comment-Date: Thu, 13 Sep 2018 17:57:43 +
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2095: [java] Add scanner keepAlive API to the Java client
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11436 Change subject: KUDU-2095: [java] Add scanner keepAlive API to the Java client .. KUDU-2095: [java] Add scanner keepAlive API to the Java client This patch adds keepAlive methods to the AsyncKuduScanner and KuduScanner. These methods leverage a package private method added to the AsyncKuduClient using a similar implementation pattern to existing scan related RPCs. The behavior of this implementation mimics the C++ client. Change-Id: Ic802f556c8860cdd43ef5f794c8f3658259bd0be --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduScanner.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduScanner.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestRemoteTablet.java M src/kudu/client/client.h 6 files changed, 341 insertions(+), 63 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/36/11436/1 -- To view, visit http://gerrit.cloudera.org:8080/11436 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ic802f556c8860cdd43ef5f794c8f3658259bd0be Gerrit-Change-Number: 11436 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] Add helper macro for tool invocations
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/11411 ) Change subject: Add helper macro for tool invocations .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11411 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7ffd357d79982ee5c93f8d3c7cfd7cc1f0863f07 Gerrit-Change-Number: 11411 Gerrit-PatchSet: 4 Gerrit-Owner: Will Berkeley Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Thu, 13 Sep 2018 17:27:18 + Gerrit-HasComments: No
[kudu-CR] Add helper macro for tool invocations
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/11411 ) Change subject: Add helper macro for tool invocations .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/11411/3/src/kudu/tools/kudu-admin-test.cc File src/kudu/tools/kudu-admin-test.cc: http://gerrit.cloudera.org:8080/#/c/11411/3/src/kudu/tools/kudu-admin-test.cc@140 PS3, Line 140: "kudu", > I think this should be Done -- To view, visit http://gerrit.cloudera.org:8080/11411 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7ffd357d79982ee5c93f8d3c7cfd7cc1f0863f07 Gerrit-Change-Number: 11411 Gerrit-PatchSet: 3 Gerrit-Owner: Will Berkeley Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Thu, 13 Sep 2018 17:26:27 + Gerrit-HasComments: Yes
[kudu-CR] Add helper macro for tool invocations
Hello Alexey Serbin, Kudu Jenkins, Adar Dembo, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11411 to look at the new patch set (#4). Change subject: Add helper macro for tool invocations .. Add helper macro for tool invocations Using `ASSERT_OK` to test the results of the `kudu` tool is normal, but it results in lousy test failure output: ../../src/kudu/tools/kudu-admin-test.cc:235: Failure Failed Bad status: Runtime error: /Users/wdberkeley/src/kudu/build/debug/bin/kudu: process exited with non-zero status 1 This adds a new macro, `ASSERT_TOOL_OK`, that also logs the stdout and stderr of a `kudu` tool invocation: ../../src/kudu/tools/kudu-admin-test.cc:235: Failure Failed Runtime error: /Users/wdberkeley/src/kudu/build/debug/bin/kudu: process exited with non-zero status 1 stdout: stderr: W0910 12:39:07.483736 2830984064 flags.cc:406] Enabled unsafe flag: --never_fsync=true Invalid argument: Unrecognized peer type: FOOVOTER Change-Id: I7ffd357d79982ee5c93f8d3c7cfd7cc1f0863f07 --- M src/kudu/tools/kudu-admin-test.cc 1 file changed, 47 insertions(+), 32 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/11/11411/4 -- To view, visit http://gerrit.cloudera.org:8080/11411 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7ffd357d79982ee5c93f8d3c7cfd7cc1f0863f07 Gerrit-Change-Number: 11411 Gerrit-PatchSet: 4 Gerrit-Owner: Will Berkeley Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Will Berkeley
[kudu-CR] [docs] Add basic advice on setting block cache size
Hello Alexey Serbin, Attila Bukor, Kudu Jenkins, Andrew Wong, Adar Dembo, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11420 to look at the new patch set (#2). Change subject: [docs] Add basic advice on setting block cache size .. [docs] Add basic advice on setting block cache size This adds a short section to the troubleshooting guide about improving the performance of the block cache size. It's fuzzy since the effectiveness of the cache and the efficacy of enlarging it are so workload dependent (e.g. consider a workload doing full table scans vs. one mostly re-scanning a small range checking for updates), but I tried to provide a starting point for users to evaluate their cache size since we've totally lacked any advice on that up to this point. I also added information about the change due to release in 1.8 that servers won't start when the block cache capacity is set too large relative to the memory limit. Change-Id: Idc7411c38b6fcc8694509ec89c32e2fe74e6c0db --- M docs/troubleshooting.adoc 1 file changed, 78 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/20/11420/2 -- To view, visit http://gerrit.cloudera.org:8080/11420 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Idc7411c38b6fcc8694509ec89c32e2fe74e6c0db Gerrit-Change-Number: 11420 Gerrit-PatchSet: 2 Gerrit-Owner: Will Berkeley Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins
